Data processing method and device, equipment and storage medium

Abstract

The invention discloses a data processing method and device, equipment and a storage medium, and relates to the technical field of data processing. The method comprises the steps of multiplexing a working coroutine in a coroutine resource pool, and obtaining a task execution object by changing an original structure of an initial task execution object to which the working coroutine belongs, namely newly adding a running resource block configured for the working coroutine and used for storing running resources of an online application in the initial task execution object; and based on the running state of the task execution object, updating running resources stored in the running resource block in the task execution object. Therefore, the working coroutine is provided with the running resources, the process that the working coroutine obtains the running resources from the application resource pool in a locking and unlocking mode and releases the running resources back to the application resource pool is not needed, the time for calling the resources is saved, the resource consumption for maintaining the application resource pool is reduced, and the working coroutine does not need to be frequently created and destroyed; and concurrency performance of an application system is improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a data processing method, device, equipment and storage medium. Background Art

[0002] As the service scenarios of microservice systems become more complex and the application scenarios become more extensive, higher requirements are placed on the concurrency, availability, and stability of online applications. Among them, online applications are applications that need to run continuously on the Internet, and usually need to handle a large number of concurrent requests and resource interactions, which requires the application to have the ability to efficiently handle concurrency. In this case, work coroutines, as a lightweight concurrent processing mechanism, can effectively utilize running resources and improve the response speed and processing power of applications. Through work coroutines, applications can achieve high concurrency processing without increasing system resource consumption, thereby improving the overall performance of the application system and user experience.

[0003] In the related art, the process of online applications in processing concurrent requests and resource interactions is that the application service of the online application creates a work coroutine for the received request, the resource pool initializes a preset number of running resources, the work coroutine obtains and runs resources from the application resource pool until the cycle of responding to the request ends, destroys the work coroutine, and recycles its related resources to the application resource pool. In this process, if there is an update to the resources in the application resource pool, the backup application resource pool is used to update the resources, and after the backup application resource pool has completed the resource update, the backup application resource pool is switched with its aforementioned application resource pool, so that when a request is received again, the latest resources are obtained from the backup application resource pool through the newly created work coroutine to respond to its corresponding request. However, the above process will reduce the concurrent performance of the application system due to the processing of massive requests, the frequent creation and destruction of work coroutines, and the backup application resource pool will only be started when the resources are updated. The daily maintenance of the backup application resource pool requires a large amount of maintenance resources, which increases the resource consumption of the application service. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device and storage medium, which can solve the problems of low application system concurrency and serious application service resource consumption in the process of online applications processing concurrent requests and resource interactions in the related art.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, which may include:

[0006] Get the working coroutine from the coroutine resource pool;

[0007] According to the work coroutine and the running resource block configured for the work coroutine, a task execution object corresponding to the work coroutine is constructed, wherein the running resource block is used to store the running resources of the online application, and the task execution object is used to process the service request received by the online application based on the running resources;

[0008] Based on the running state of the task execution object, the running resources stored in the running resource block in the task execution object are updated. The running state includes the state of the task execution object in the process of processing the service request.

[0009] In a second aspect, an embodiment of the present application provides a data processing device, which may include:

[0010] The acquisition module is used to obtain the working coroutine in the coroutine resource pool;

[0011] A construction module, used to construct a task execution object corresponding to the work coroutine according to the work coroutine and the running resource block configured for the work coroutine, wherein the running resource block is used to store the running resources of the online application, and the task execution object is used to process the service request received by the online application based on the running resources;

[0012] The updating module is used to update the running resources stored in the running resource block in the task execution object based on the running status of the task execution object. The running status includes the status of the task execution object in the process of processing the service request.

[0013] In a third aspect, an embodiment of the present application provides a computer device, the computer device comprising: a processor and a memory storing computer program instructions;

[0014] When the processor executes the computer program instructions, the data processing method shown in the first aspect is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the data processing method shown in the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the data processing method shown in the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the data processing method shown in the first aspect.

[0018] The data processing method, apparatus, device and storage medium of the embodiment of the present application, by reusing the working coroutine in the coroutine resource pool, and according to the working coroutine and the running resource block configured for the working coroutine for storing the running resources of the online application, construct a task execution object corresponding to the working coroutine for processing the service request received by the online application based on the running resources. In this way, the working coroutine has its own running resources, and there is no need for the working coroutine to obtain the running resources from the application resource pool by means of encryption and unlocking and release them back to the application resource pool after execution, which saves the time of calling resources and reduces the resource consumption of frequently calling resources in the application resource pool. Based on the running status of the task execution object, the running resources stored in the running resource block in the task execution object are updated. The running status includes the status of the task execution object in the process of processing the service request. Therefore, by reusing the working coroutine in the coroutine resource pool, there is no need to frequently create and destroy the working coroutine, which improves the concurrency performance of the application system. In addition, by changing the structure of the task execution object to which the working coroutine belongs, that is, adding a running resource block configured for the working coroutine in the task execution object to which the working coroutine belongs to store the running resources of the online application, there is no need to set up a backup resource pool, which saves a lot of maintenance resources for maintaining the backup application resource pool and reduces the resource consumption of the application service. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of a data processing method for an application service;

[0021] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the structure of a coroutine resource pool in a data processing method provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the architecture of an execution subject in a data processing method provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of data processing effect in a data processing method provided in an embodiment of the present application;

[0025] Figure 6 is a structural schematic diagram of a data processing device provided by an embodiment of the present application;

[0026] Figure 7It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0028] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0029] The acquisition, storage, use, and processing of data (including but not limited to the features and information in the text) in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0030] As the business scenarios of modern microservice systems become more complex and the application scenarios become more extensive, higher requirements are placed on the concurrency, availability, and stability of online applications of multiple microservices. In scenarios where some business parameters or business processes need to change frequently, services need to support dynamic loading of business parameters, business scripts and other operating resources to avoid interruptions when loading parameters and scripts, and to achieve smooth upgrades and maintenance of services. At the same time, in actual system applications, the load may fluctuate with time and the number of requests, and services also need to be able to cope with different numbers of concurrent business requests. Among them, online applications are applications that need to run continuously on the Internet, and usually need to handle a large number of concurrent requests and resource interactions, which requires the application to have the ability to efficiently handle concurrency. In this case, work coroutines, as a lightweight concurrent processing mechanism, can effectively utilize operating resources and improve the response speed and processing power of applications. Through work coroutines, applications can achieve high concurrency processing without increasing system resource consumption, thereby improving the overall performance and user experience of the application system.

[0031] In related technologies, such as Figure 1 As shown in the figure, the process of processing concurrent requests and resource interactions in the online application is that the main process in the application service of the online application creates a work coroutine for the received request, and the application resource pool (such as Figure 1 The currently used resource pool in the application resource pool initializes a preset number of running resources, and the worker coroutine obtains and runs resources from the application resource pool with a lock until the cycle of responding to the request ends, destroys the worker coroutine, and recycles its related resources to the application resource pool. Here, the worker coroutine obtains and runs resources from the application resource pool through the lock and unlock operation, which means that the worker coroutine performs the lock and unlock operation when obtaining resources from the resource pool. The lock and unlock operation means that the resource pool manages the access and allocation of resources through the lock and unlock mechanism to ensure the correct allocation of resources and avoid resource conflicts.

[0032] Based on this, in the above process, if there is an update in the resource in the application resource pool, the backup application resource pool (such as Figure 1 The standby application resource pool (loaded resource pool in the standby application resource pool) updates resources, and after the standby application resource pool has completed updating resources, the standby application resource pool is switched with its aforementioned application resource pool, so that when a request is received again, the latest resources are obtained from the standby application resource pool through the newly created work coroutine to respond to its corresponding request. However, the above process will reduce the concurrency performance of the application system due to the frequent creation and destruction of work coroutines due to the processing of massive requests, and the standby application resource pool will only be started when the resources are updated. The daily maintenance of the standby application resource pool requires a large amount of maintenance resources, which increases the resource consumption of the application service. In addition, the work coroutine needs to perform encryption and unlocking operations when obtaining resources from the resource pool, which will not only increase the resource consumption of the application service, but also reduce the concurrency performance of the application system.

[0033] Based on this, in order to solve the problems in the related technology, the embodiment of the present application provides a data processing method based on the coroutine resource pool that supports dynamic loading of resources and flexible expansion and contraction in a complex online service scenario. Figures 2 to 5 , the methods, devices, computer equipment and storage media of the embodiments of the present application are described in detail. It should be noted that these embodiments are not intended to limit the scope of the disclosure of the present application.

[0034] first, Figure 2 A flowchart of a data processing method provided in an embodiment of the present application.

[0035] like Figure 2 As shown, the data processing method can be applied to a data processing system, and the data processing method can specifically include the following steps:

[0036] Step 210, obtaining a working coroutine from a coroutine resource pool; Step 220, constructing a task execution object corresponding to the working coroutine according to the working coroutine and the running resource block configured for the working coroutine, wherein the running resource block is used to store the running resources of the online application, and the task execution object is used to process the service request received by the online application based on the running resources; Step 230, updating the running resources stored in the running resource block in the task execution object based on the running status of the task execution object, wherein the running status includes the status of the task execution object in the process of processing the service request.

[0037] For example, Figure 3 As shown, at least two working coroutines are initialized in the coroutine resource pool (worker pool), and the working resource coroutines are loaded one by one through an additional resource overloading coroutine, that is, the initial task execution object configuration of each working coroutine in at least two working coroutines is configured to obtain a task execution object (worker), so that the task execution object to which the working coroutine belongs includes both the working coroutine and the running resource block, and the running resource block can load the latest version of the running resource of the online application. Then, the resource overloading coroutine can load the latest version of the running resource without affecting the task execution object processing service requests by judging the running status of each task execution object, ensuring that the running resource blocks in all task execution objects can be loaded into the latest version of the running resource, and no additional loading resource pool is required as a standby switch. Here, the existing working coroutines are reused in the coroutine resource pool, which reduces the additional resource overhead caused by the frequent creation and destruction of coroutines and improves the resource utilization efficiency.

[0038] In this way, by reusing the working coroutine in the coroutine resource pool, and according to the working coroutine and the running resource block configured for the working coroutine for storing the running resources of the online application, a task execution object corresponding to the working coroutine is constructed for processing the service request received by the online application based on the running resources. In this way, the working coroutine has its own running resources, and there is no need for the working coroutine to obtain the running resources from the application resource pool by locking and unlocking and releasing them back to the application resource pool after execution, which saves the time for calling resources and reduces the resource consumption of frequently calling resources in the application resource pool. Then, based on the operation of the task execution object, The running status is updated, and the running resources stored in the running resource block in the task execution object are updated. The running status includes the status of the task execution object in the process of processing the service request. Therefore, by reusing the working coroutine in the coroutine resource pool, there is no need to frequently create and destroy the working coroutine, which improves the concurrency performance of the application system. In addition, by changing the structure of the task execution object to which the working coroutine belongs, that is, adding a running resource block configured for the working coroutine in the task execution object to which the working coroutine belongs to store the running resources of the online application, there is no need to set up a backup resource pool, which saves a lot of maintenance resources for maintaining the backup application resource pool and reduces the resource consumption of application services.

[0039] The above steps are described in detail below, as shown below.

[0040] First, involving step 210, in some embodiments of the present application, a preset number, such as at least two working coroutines, can be obtained from the coroutine resource pool.

[0041] Next, step 220 is involved. In some embodiments of the present application, step 220 may specifically include:

[0042] Add the running resource block configured for the work coroutine to the initial task execution object to which the work coroutine belongs, and the task execution object corresponding to the work coroutine.

[0043] For example, Figure 4 As shown, a task execution object (worker) can include a working coroutine (goroutine) and a running resource block (data).

[0044] Therefore, by adding a running resource block configured for the working coroutine in the initial task execution object to which the working coroutine belongs, it is possible to realize that the task execution object where the working coroutine is located has its own running resources. Dynamic loading of running resources is achieved by an additional coroutine judging the state machine of the working coroutine to update and switch the running resources. The coroutine does not need to obtain running resources from the resource pool by locking and unlocking, nor does it need to load the resource pool as a backup switch, thereby reducing the consumption of application service resources.

[0045] Then, in step 230, as the online application is updated and upgraded, the resource version also needs to be adjusted accordingly to ensure that the new version of the application can correctly access and use these resources. Based on this, when loading the running resources in the running resource block in the task execution object, the loading behavior needs to be determined according to the different running state combinations of the task execution object.

[0046] Among them, the task execution object has two types of running states. The first type is the use state, including the ready state and the processing state, that is, S1 = {idle, inuse}. This state will only change when the task execution object is locked. The second type is the switching state, including the blocking state and the updating state, that is, S2 = {needswitch, switching}. This state change does not require locking. The resource loading coroutine needs to handle four separate running states of the task execution object or a combination of at least two of the four running states.

[0047] The running state in the embodiment of the present application may include at least one of the following: ready state (idle), processing state (inuse), blocking state (inuse|needswitch) and updating state (inuse|switching). Among them, the ready state (idle) is used to characterize that the task execution object is in a state of being unprocessed and waiting for processing service requests; the processing state (inuse) is used to characterize that the task execution object is in a state of processing service requests; the blocking state (inuse|needswitch) is used to characterize that the task execution object is in a state of processing service requests, and the running resources in the running resource block of the task execution object have been loaded and not updated; the updating state (inuse|switching) is used to characterize that the task execution object is in a state of updating the running resources in the running resource block of the task execution object.

[0048] The above-mentioned operating states are described below respectively.

[0049] In some embodiments of the present application, the running state includes a ready state, and the ready state is used to represent that the task execution object is in an unprocessed state and is waiting for a service request to be processed. Based on this, step 230 may specifically include step 2301 and step 2302.

[0050] Step 2301 : Based on the ready state of the task execution object, load the target running resources of the online application, where the target running resources include the running resources of the latest version of the online application.

[0051] Step 2302: Replace the running resources in the running resource block of the task execution object with the target running resources.

[0052] For example, Figure 4As shown, when the online application is in normal working state, after receiving the service request, it will put the service request into the task queue, and the coroutine resource pool will provide <idle>The worker in the running state receives the task in the task queue, and the worker state is set to <inuse>After processing a request task, it is put back into the coroutine pool and the status is set to <idle>, at this time, when the worker status is <idle>When the task is executed, the latest version of the running resources of the online application is directly loaded, and then the old version of the running resources in the running resource block of the task execution object is replaced and recycled.

[0053] It should be noted that the resource version in the embodiment of the present application refers to the version of the resource files that the online application or service relies on. These resources may include database models, configuration files, code libraries, etc.

[0054] In this way, it can be ensured that the version of the running resources in the running resource block of the task execution object is the latest version of the running resources, thereby ensuring the stable operation and compatibility of the online application.

[0055] In other embodiments of the present application, the running status includes a processing status and a ready status. The processing status is used to characterize that the task execution object is in a state of processing a service request, and the ready status is used to characterize that the task execution object is in an unprocessed state and waiting for processing a service request. Based on this, step 230 can specifically include step 2303 and step 2304.

[0056] Step 2303: based on the processing state of the task execution object, determine the switching status of the running state of the task execution object within the first preset time period, wherein the switching status of the running state of the task execution object includes switching from the processing state to the ready state.

[0057] Step 2304: Update the operating resources in the operating resource block of the task execution object according to the switching of the operating state of the task execution object within the first preset time period.

[0058] Among them, when a task execution object is polled and the task execution object is in a processing state, the method of updating the running resources in the running resource block of the task execution object can be determined based on whether the task execution object switches from the processing state to the ready state within a first preset time period, such as 10s, as shown below.

[0059] Specifically, in some embodiments, the running state of the task execution object is switched from the processing state to the ready state within the first preset time period. Based on this, step 2304 may specifically include step 23041 and step 23042.

[0060] Step 23041, when the running state of the task execution object is switched from the processing state to the ready state within the first preset time period, the target running resources of the online application are loaded, and the target running resources include the running resources of the latest version of the online application.

[0061] Step 23042: Replace the running resources in the running resource block of the task execution object with the target running resources.

[0062] For example, when the worker's running status is <inuse>If the worker status changes from <inuse>Switch to <idle>, continue to load the target running resources according to the steps when the running state of the task execution object is in the ready state, that is, the above steps 2303 to 2304.

[0063] In other embodiments, the running state of the task execution object is not switched from the processing state to the ready state within the first preset time period. Based on this, step 2304 may specifically include steps 23043 to 23045.

[0064] Step 23043, when the running state of the task execution object is not switched from the processing state to the ready state within the first preset time period, load the target running resources of the online application, the target running resources including the running resources of the latest application version of the online application.

[0065] Step 23044, monitor the progress of the task execution object in processing the service request.

[0066] Step 23045: When the task execution object has processed the service request, the running resources in the running resource block of the task execution object are replaced with the target running resources.

[0067] For example, still taking the above example, if the worker's running status does not change from <inuse>Switch to <idle>, you can change the worker's running status from <inuse>Switch to<inuse|needswitch> And monitor the progress of its task execution object in processing service requests. At this time, only the latest version of the running resources of the online application are loaded in the worker's running resource block. Because the worker is still working with the old version of the running resources, no replacement operation is performed, that is, the running resources in the running resource block of the task execution object are not replaced with the target running resources for the time being. Then, when the worker finishes its work and is put back into the coroutine resource pool, it will detect that the worker is in<inuse|needswitch> state, at this time, the step of replacing the running resources in the running resource block of the task execution object with the target running resources and recycling the old version of the running resources can be directly executed.

[0068] It should be noted that during the process of replacing the running resources in the running resource block of the task execution object with the target running resources and recycling the old version of the running resources, the running state of the worker will temporarily change from<inuse|needswitch> Status switched to<inuse|switching> After the running resources in the running resource block of the task execution object are replaced with the target running resources and the steps of recycling the old version of the running resources are completed, you can<inuse|switching> State switching will <idle>state.

[0069] In some other embodiments of the present application, the running state also includes a blocking state, which is used to characterize that the task execution object is in a state where the running resources in the running resource block of the task execution object are being processed and the service request has been loaded but not updated. Based on this, step 230 may specifically include step 2305 and step 2306.

[0070] Step 2305: based on the blocking state of the task execution object, delete the loaded running resources in the running resource block of the task execution object.

[0071] For example, when the worker status is<inuse|needswitch> , it means that the worker has been working until this loading after the last loading of the running resource timed out. At this time, the running resource loaded last time can be recycled (the running resource version is outdated and not used), and then the running state of the worker is adjusted to <inuse>Status, and then update the running resources through the steps of updating the running resources stored in the running resource block in the task execution object corresponding to the processing status, that is, updating the running resources through the steps 2303 and 2304 involved above.

[0072] Specifically, in order to reduce the process of updating the target running resources, before deleting the running resources loaded in the running resource block of the task execution object, it can be determined whether the running resources loaded in the running resource block of the deleted task execution object are old versions. If they are old versions, the following steps can be executed; if not, they can be retained and the following step 23053 can be executed.

[0073] Based on this, step 2305 may specifically include step 23051 and step 23053.

[0074] Step 23051, based on the blocking state of the task execution object, obtain the first resource version of the running resource loaded in the running resource block of the task execution object.

[0075] Step 23052, when the first resource version does not match the preset resource version, delete the loaded running resources in the running resource block of the task execution object; wherein the preset resource version is the resource version of the target running resource corresponding to the latest version of the online application.

[0076] Step 23053, adjusting the running state of the task execution object from the blocking state to the processing state, so as to update the running resources by the step of updating the running resources stored in the running resource block in the task execution object corresponding to the processing state.

[0077] In some further embodiments of the present application, the running state includes an update state, and the update state is used to represent that the task execution object is in a state of updating the running resources in the running resource block of the task execution object.

[0078] For example, when the worker status is<inuse|switching> , it means that the worker has finished its work and is executing the running resource switch (the running resource version is outdated). The worker is skipped without any processing. After the worker completes the switch, it changes to other states and waits for the resource loading coroutine to check its outdated version next time before loading and switching.

[0079] Therefore, through the above state control, the coroutine pool can complete the switching of all working coroutine running resources in the normal business processing process, and use the state S1 mechanism to ensure that the update is in <idle>The running resources of the state worker are increased, and the switching state S2 mechanism is added to ensure that the update is in <inuse>The running resources of the state worker do not need to occupy a large amount of memory in the loading resource pool for standby switching, which is conducive to saving resources.

[0080] In addition, in order to further reduce resource consumption, before step 130, the data processing method may further include step 140, obtaining a second resource version of the running resource in the running resource block of the task execution object through a resource overloading coroutine.

[0081] Based on this, step 130 may specifically include:

[0082] When the second resource version does not match the preset resource version, the running resources stored in the running resource block in the task execution object are updated based on the running status of the task execution object, wherein the preset resource version is the resource version of the target running resources corresponding to the latest version of the online application.

[0083] In addition, in order to prevent an excessive number of workers from occupying a large amount of resources or an insufficient number of workers from being unable to process requests in the task queue in a timely manner, the data processing method provided in the embodiment of the present application provides steps for flexibly expanding or shrinking the number of workers. In this way, a preset number of working coroutines are created when the coroutine resource pool is initialized. During operation, an additional management coroutine is used to adjust the number of workers based on the amount of service requests in the next time period predicted by a prediction algorithm. The maximum number of objects is set to prevent an excessive number of workers from occupying a large amount of resources, and the minimum number of objects is set to prevent an insufficient number of workers from processing service requests in a timely manner.

[0084] Based on this, in some embodiments of the present application, after step 130 , the data processing method may further include steps 210 and 220 .

[0085] Step 210, generating a second number of service requests that the online application is expected to process within a third preset time period according to a first number of service requests to be processed by the online application within a second preset time period; wherein the second preset time period occurs before the third preset time period.

[0086] Exemplarily, the Holt trend prediction algorithm can be used to generate the second number of service requests that the online application is expected to process within the third preset time period according to the first number of service requests to be processed by the online application within the second preset time period. Here, the Holt trend prediction algorithm is an extension of the exponential smoothing algorithm and is applicable to time series data containing linear trends, such as the first number of service requests to be processed by the online application within the second preset time period. The second preset time period may include at least two preset sub-time periods, so that the management coroutine can record the number of service requests to be processed by the online application within each preset sub-time period of at least two preset sub-time periods, that is, M t ={N1,N2....N w }, the iterative trend component coefficient and the horizontal component coefficient are continuously updated through this set data, and then the second number of service requests that the online application is expected to process in the next time period, that is, the third preset time period, is predicted through the prediction formula.

[0087] It should be noted that the Holt trend prediction algorithm in the embodiment of the present application may adopt other trend prediction algorithms, and the trend prediction algorithm is not limited here.

[0088] Step 220: Based on the second quantity, adjust the number of task execution objects in the coroutine resource pool within a third preset time period.

[0089] Furthermore, the above step 210 may specifically include steps 2101 to 2103.

[0090] Step 2101, based on the first number of service requests received by the online application within the second preset time period, the target dynamic horizontal smoothing parameter, the first horizontal component coefficient corresponding to the service requests within the fourth preset time period, and the first trend component evaluation coefficient, generate a second horizontal component coefficient corresponding to the service requests within the second preset time period; wherein the fourth preset time period occurs before the second preset time period, the first horizontal component coefficient includes the average number of service requests corresponding to the baseline of the service requests within the fourth preset time period, the first trend component evaluation coefficient includes the amount of data corresponding to the speed of change of the service requests within the fourth preset time period in the direction of the rising and falling trend over time, the second horizontal component coefficient includes the average number of service requests corresponding to the baseline of the service requests within the second preset time period, and the target dynamic horizontal smoothing parameter is used to characterize the sensitivity to the number of service requests to be processed by the online application in the latest preset time period among N preset time periods, and N is a positive integer.

[0091] Exemplarily, the second horizontal component coefficient can be calculated by the following formula (1):

[0092] L t =σN t +(1 - σ)(L t-1 +ΔT t-1 ) (1)

[0093] Among them, L t is the second horizontal component coefficient of the second preset time period t, σ is the target dynamic horizontal smoothing parameter, which determines the sensitivity to new data, N t is the number of tasks in the task queue at the current time t, i.e., the first number of service requests, L t-1 is the first horizontal component coefficient of the fourth preset time period t-1, ΔT t-1 is the first trend component evaluation coefficient of the fourth preset time period t-1.

[0094] Step 2102, based on the first horizontal component coefficient, the second horizontal component coefficient, the first trend component evaluation coefficient corresponding to the service requests within the fourth preset time period, and the target dynamic trend smoothing parameter, generate a second trend component evaluation coefficient corresponding to the service requests within the second preset time period, the second trend component evaluation coefficient includes a data volume corresponding to the speed of change of the service requests within the second preset time period in the rising and falling trend direction over time, and the target dynamic trend smoothing parameter is used to characterize the sensitivity to the trend change of the number of service requests to be processed by online applications within N preset time periods.

[0095] For example, the second trend component coefficient represents the speed and direction of data change over time. The initial value is N2, which indicates whether the data is rising, falling or remaining unchanged as a whole. It can be calculated by the following formula (2):

[0096] ΔT t =θ(L t -L t-1 )+(1 - θ)ΔT t-1 (2)

[0097] Where, ΔT t is the coefficient of the second trend component in the second time period t, θ is the target dynamic trend smoothing parameter, which determines the sensitivity of the model to trend changes, and L t is the second horizontal component coefficient of the second time period t, L t-1 is the first horizontal component coefficient of the fourth preset time period t-1, ΔT t-1 is the first trend component evaluation coefficient of the fourth preset time period t-1.

[0098] Step 2103: Generate a second number of service requests that are expected to be processed by the online application within a third preset time period based on a second horizontal component coefficient and a second trend component evaluation coefficient corresponding to the service requests within the second preset time period.

[0099] For example, according to the second horizontal component coefficient and the second trend component evaluation coefficient obtained by iteration above, the number of tasks in the task queue in the next time period, i.e., the third preset time period, can be predicted by the following formula (3), i.e., the second number of service requests that the online application is expected to process:

[0100] N pt =L t +ΔT t (3)

[0101] Among them, N pt A second number of service requests that the online application is expected to process within a third preset time period.

[0102] It should be noted that the target dynamic level smoothing parameters and target dynamic trend smoothing parameters involved above can be determined in the following manner.

[0103] In some embodiments of the present application, before step 210 , the data processing method may further include steps 2104 to 2106 .

[0104] Step 2104, obtaining a first sample number of service requests actually processed by the online application in the i-th preset time period among N preset time periods and a second sample number of service requests expected to be processed by the online application in the i-th preset time period; wherein the N preset time periods include at least one preset time period prior to the second preset time period.

[0105] Step 2105: Generate a first sample error of the online application processing service request within the i-th preset time period based on the first sample quantity and the second sample quantity.

[0106] Step 2106, when the first sample error is less than or equal to the preset sample error, the first dynamic horizontal smoothing parameter of the previous preset time period of the second preset time period is determined as the target dynamic horizontal smoothing parameter, and the first dynamic trend smoothing parameter of the previous preset time period of the second preset time period is determined as the target dynamic trend smoothing parameter.

[0107] For example, the idea of ​​Bayesian optimization can be used to adjust the target dynamic level smoothing parameter σ and the target dynamic trend smoothing parameter θ. Specifically, the management coroutine predicts the result M based on the number of task queues in the most recent t time periods every half an hour. p = {N pt ,N p(t-1) ...N pnow }, and the real task queue number M t The sample error can be calculated by the following formula (4):

[0108]

[0109] Among them, when the first sample error J≤the preset sample error H, the parameters σ and θ are not adjusted, and the first dynamic horizontal smoothing parameter of the previous preset time period can be determined as the target dynamic horizontal smoothing parameter, and the first dynamic trend smoothing parameter of the previous preset time period of the second preset time period can be determined as the target dynamic trend smoothing parameter.

[0110] After the above step 2105, the data processing method may further include steps 2107 to 2111.

[0111] Step 2107, when the first sample error is greater than the preset sample error, an adjustment strategy is generated according to the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter, the adjustment strategy including a first adjustment strategy for simultaneously increasing the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter according to a preset adjustment amplitude; a second adjustment strategy for simultaneously decreasing the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter according to a preset adjustment amplitude; a third adjustment strategy for increasing the first dynamic horizontal smoothing parameter and decreasing the first dynamic trend smoothing parameter according to a preset adjustment amplitude; and a fourth adjustment strategy for decreasing the first dynamic horizontal smoothing parameter and increasing the first dynamic trend smoothing parameter according to a preset adjustment amplitude.

[0112] Step 2108, according to the adjustment strategy, adjust the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter respectively to obtain the second dynamic horizontal smoothing parameter and the second dynamic trend smoothing parameter corresponding to each adjustment strategy.

[0113] Step 2109, according to the second dynamic level smoothing parameter and the second dynamic trend smoothing parameter corresponding to each adjustment strategy, respectively calculate the number of service requests that the online application is expected to process within the second preset time period corresponding to each adjustment strategy.

[0114] Step 2110, based on the third sample number of service requests that the online application is expected to process within the second preset time period corresponding to each adjustment strategy and the fourth sample number of service requests that the online application actually processes within the second preset time period, generate a second sample error of the service requests processed by the online application within the second preset time period corresponding to each adjustment strategy; based on the second sample error corresponding to each adjustment strategy, screen out a target sample error, the error value of the target sample error being less than or equal to the preset error value.

[0115] Step 2111, determine the second dynamic horizontal smoothing parameter of the target adjustment strategy corresponding to the target sample error as the target dynamic horizontal smoothing parameter, and determine the second dynamic trend smoothing parameter of the target adjustment strategy corresponding to the target sample error as the target dynamic trend smoothing parameter.

[0116] For example, when J>H, the changes of the four parameters according to σ and θ are as follows:

[0117]

[0118] Based on this, we train the task queue number set M for the last 2t to t time periods respectively. 2t-t = {N 2t ,N 2t-1 ...N t }, four different prediction formulas are obtained, and then the four prediction formulas are used to calculate the number of task queues M in the last t time periods. t Iterate and calculate the number of task queues M in the last t time periods respectively pi , and then use the error score formula to calculate the respective error scores J i , select the one that produces the smallest J min The parameters σ and θ are used as new horizontal smoothing parameters and trend smoothing parameters.

[0119] In some embodiments of the present application, the embodiments of the present application provide a step for flexibly expanding the number of workers, so that a preset number of working coroutines are created when the coroutine resource pool is initialized. During operation, an additional management coroutine is used to adjust the number of workers by using the service request volume of the next time period predicted by the prediction algorithm, and the maximum number of objects is set to prevent workers from occupying too many resources. Based on this, the above step 220 can include steps 2201 to 2203.

[0120] Step 2201, calculate the first object expansion quantity of the task execution object in the third preset time period according to the object quantity of the task execution object in the coroutine resource pool in the second preset time period and the preset expansion ratio.

[0121] Step 2202, when the second number is greater than the first object expansion number, based on the second number, the preset expansion ratio, the number of task execution objects in the coroutine resource pool within the third preset time period and the preset offset value, calculate the second object expansion number of the newly added task execution objects within the third preset time period.

[0122] Exemplarily, according to the above-obtained second number N of task execution objects predicted for the next time period pt , design the corresponding expansion logic, and set the number of task execution objects in the coroutine resource pool within the third preset time period to N w , the preset expansion ratio is α (0<α<1), and the maximum number of workers is N maxw .

[0123] Based on this, when N pt >N w When α is equal to α, the second object expansion quantity of the newly added task execution objects within the third preset time period is calculated by the following formula (5):

[0124]

[0125] Among them, N pt is based on the second quantity, α is a preset expansion ratio, and μ is a preset offset value,

[0126] Step 2203, when the third object expansion number which is the sum of the first object expansion number and the second object expansion number is greater than the preset maximum object number, the preset maximum object number is determined as the object number of the task execution objects in the coroutine resource pool within the third preset time period.

[0127] For example, if the third object expansion number is greater than N maxw , then the maximum number of objects N will be preset maxw The number of objects of the task execution objects in the coroutine resource pool within the third preset time period is determined.

[0128] In addition, after step 2202, step 220 may further include step 2204, in which, when the third object expansion number which is the sum of the first object expansion number and the second object expansion number is greater than the preset maximum object number, and the third object expansion number is the same as the preset maximum object number, the object number of the task execution objects in the coroutine resource pool within the second preset time period is determined as the object number of the task execution objects in the coroutine resource pool within the third preset time period.

[0129] For example, when N pt >N w α and N w =N maxw There is no need to adjust the number of workers.

[0130] In some other embodiments of the present application, a minimum number of objects is set to prevent a situation where too few workers process service requests in a timely manner. Based on this, the above step 220 may include steps 2204 to 2206.

[0131] Step 2204, calculating a first reduced number of task execution objects within a third preset time period according to the number of task execution objects in the coroutine resource pool within the second preset time period and the preset reduction ratio.

[0132] Step 2205, when the second number is less than the first reduced object number, based on the object number of task execution objects in the coroutine resource pool within the second preset time period, the second number and the preset reduction ratio, calculate the second object reduction number of task execution objects within the third preset time period.

[0133] For example, when the second number N pt <N w When γ is equal to γ, and this state lasts for a period of time Δτ, the number of reductions of the second object can be calculated by the following formula (6):

[0134]

[0135] Step 2206, when the third object reduction number which is the cumulative number of the first reduced objects and the second reduced objects is less than the preset minimum object number, the preset minimum object number is determined as the object number of the task execution objects in the coroutine resource pool within the third preset time period.

[0136] For example, if the number of the third object reduction is less than N minw , then N w Equal to N minw .

[0137] In addition, after step 2205, step 220 may further include step 2207, in which the object number of task execution objects in the coroutine resource pool within the second preset time period is determined as the object number of task execution objects in the coroutine resource pool within the third preset time period when the third object reduction number which is the sum of the first reduced object number and the second object reduction number is less than the preset minimum object number, and the third object reduction number is the same as the preset minimum object number.

[0138] For example, when N pt <N w γ and N w =N minw There is no need to adjust the number of workers.

[0139] Based on this, we can refer to Figure 5 As shown in the figure, compared with the operation of doubling the number of coroutines directly when the performance bottleneck is reached, the time series prediction algorithm can adjust the number of coroutines accordingly according to the real-time request volume, which can more flexibly respond to sudden traffic and load changes, achieve better scaling effects, and save a lot of idle working coroutines. In addition, the Holt trend prediction algorithm and Bayesian optimization ideas are combined to predict the number of requests to achieve the effect of smoothly scaling the size of the coroutine pool. At the same time, there is less data demand, and only the request volume in t time periods is analyzed, so the algorithm has better adaptability and interpretability.

[0140] Therefore, in the embodiment of the present application, the management coroutine can simultaneously manage the linear trend characteristics of the business request data, use the algorithm to predict the future request data volume, and then adjust the number of working coroutines in the coroutine pool accordingly. The management coroutine collects the number of tasks in the task queue of t time periods, uses the Holt trend algorithm to predict the business request volume in the next time period, and then adjusts the number of working coroutines in the coroutine pool by comparing it with the current number of working coroutines. At the same time, the algorithm parameters are adjusted in combination with the Bayesian optimization idea to achieve better prediction results.

[0141] The present application also provides a data processing device, specifically in combination with Figure 6 Provide detailed explanation.

[0142] Figure 6 It is a structural diagram of a data processing device provided by an embodiment of the present application.

[0143] In some embodiments of the present application, Figure 6 The data processing device shown can be arranged in the data processing system provided in the embodiment of the present application.

[0144] like Figure 6 As shown, the data processing device 60 may specifically include:

[0145] The acquisition module 601 is used to acquire a working coroutine from a coroutine resource pool;

[0146] A construction module 602 is used to construct a task execution object corresponding to the work coroutine according to the work coroutine and the running resource block configured for the work coroutine, wherein the running resource block is used to store the running resources of the online application, and the task execution object is used to process the service request received by the online application based on the running resources;

[0147] The updating module 603 is used to update the running resources stored in the running resource block in the task execution object based on the running status of the task execution object. The running status includes the status of the task execution object in the process of processing the service request.

[0148] In this way, the data processing device in the embodiment of the present application can construct a task execution object corresponding to the working coroutine for processing the service request received by the online application based on the running resources by reusing the working coroutine in the coroutine resource pool and according to the working coroutine and the running resource block configured for the working coroutine for storing the running resources of the online application. In this way, the working coroutine has its own running resources, and there is no need for the working coroutine to obtain the running resources from the application resource pool by locking and unlocking and releasing them back to the application resource pool after execution, which saves the time for calling resources and reduces the resource consumption of frequently calling resources in the application resource pool. Then, based on The running status of the task execution object updates the running resources stored in the running resource block in the task execution object. The running status includes the status of the task execution object in the process of processing the service request. Therefore, by reusing the working coroutine in the coroutine resource pool, there is no need to frequently create and destroy the working coroutine, which improves the concurrency performance of the application system. In addition, by changing the structure of the task execution object to which the working coroutine belongs, that is, adding a running resource block configured for the working coroutine in the task execution object to which the working coroutine belongs to store the running resources of the online application, there is no need to set up a backup resource pool, which saves a lot of maintenance resources for maintaining the backup application resource pool and reduces the resource consumption of application services.

[0149] The data processing device 60 in the embodiment of the present application is described in detail below.

[0150] In some embodiments of the present application, the update module 603 may be specifically used to, when the running state includes a ready state, which is used to indicate that the task execution object is in an unprocessed state and is waiting for a service request to be processed, load the target running resource of the online application based on the ready state of the task execution object, where the target running resource includes the running resource of the latest version of the online application;

[0151] Replace the running resource in the running resource block of the task execution object with the target running resource.

[0152] In some embodiments of the present application, the update module 603 may be specifically used to determine the switching of the running state of the task execution object within the first preset time period based on the processing state of the task execution object, when the running state includes a processing state and a ready state, the processing state is used to characterize that the task execution object is in a state of processing a service request, and the ready state is used to characterize that the task execution object is in a state of not processing and waiting for processing a service request, wherein the switching of the running state of the task execution object includes switching from the processing state to the ready state;

[0153] According to the switching of the running state of the task execution object within the first preset time period, the running resources in the running resource block of the task execution object are updated.

[0154] In some embodiments of the present application, the data processing device 60 in the embodiment of the present application may further include a loading module, which is used to load the target running resources of the online application when the running state of the task execution object is switched from the processing state to the ready state within the first preset time period, and the target running resources include the running resources of the latest version of the online application;

[0155] In the embodiment of the present application, the data processing device 60 may further include a replacement module, which is used to replace the running resources in the running resource block of the task execution object with the target running resources.

[0156] In some embodiments of the present application, the data processing device 60 in the embodiment of the present application may further include a loading module, which is used to load the target running resources of the online application when the running state of the task execution object is not switched from the processing state to the ready state within the first preset time period, and the target running resources include the running resources of the latest application version of the online application;

[0157] In the embodiment of the present application, the data processing device 60 may further include a monitoring module for monitoring the progress of the task execution object in processing the service request;

[0158] In the embodiment of the present application, the data processing device 60 may further include a replacement module, which is used to replace the running resources in the running resource block of the task execution object with the target running resources when the task execution object completes processing the service request.

[0159] In some embodiments of the present application, the data processing device 60 in the embodiment of the present application may further include a deletion module, which is used to delete the loaded running resources in the running resource block of the task execution object based on the blocking state of the task execution object when the running state also includes a blocking state, and the blocking state is used to represent that the task execution object is in a state of processing a service request and the running resources in the running resource block of the task execution object have been loaded and not updated;

[0160] In the embodiment of the present application, the data processing device 60 may further include an adjustment module for adjusting the running state of the task execution object from a blocking state to a processing state, so as to update the running resources by updating the running resources stored in the running resource block in the task execution object corresponding to the processing state.

[0161] In some embodiments of the present application, the acquisition module 601 is used to acquire the first resource version of the running resource loaded in the running resource block of the task execution object based on the blocking state of the task execution object;

[0162] In the embodiment of the present application, the data processing device 60 may also include a deletion module, which is used to delete the loaded running resources in the running resource block of the task execution object when the first resource version does not match the preset resource version; wherein the preset resource version is the resource version of the target running resource corresponding to the latest version of the online application.

[0163] In some embodiments of the present application, the acquisition module 601 is used to acquire the second resource version of the running resource in the running resource block of the task execution object through the resource overloading coroutine;

[0164] The update module 603 can be specifically used to update the running resources stored in the running resource block in the task execution object based on the running status of the task execution object when the second resource version does not match the preset resource version, wherein the preset resource version is the resource version of the target running resources corresponding to the latest version of the online application.

[0165] In some embodiments of the present application, the data processing device 60 in the embodiment of the present application may further include a generating module, which is used to generate a second number of service requests that the online application is expected to process within a third preset time period according to a first number of service requests to be processed by the online application within a second preset time period; wherein the second preset time period occurs before the third preset time period;

[0166] In the embodiment of the present application, the data processing device 60 may further include an adjustment module for adjusting the number of task execution objects in the coroutine resource pool within a third preset time period based on the second number.

[0167] In some embodiments of the present application, the data processing device 60 in the embodiment of the present application may further include a generating module, which is used to generate a second horizontal component coefficient corresponding to the service request within the second preset time period according to the first number of service requests received by the online application within the second preset time period, the target dynamic horizontal smoothing parameter, the first horizontal component coefficient corresponding to the service request within the fourth preset time period, and the first trend component evaluation coefficient; wherein the fourth preset time period occurs before the second preset time period, the first horizontal component coefficient includes the average number of service requests corresponding to the baseline of the service request within the fourth preset time period, the first trend component evaluation coefficient includes the data volume corresponding to the speed of change of the service request within the fourth preset time period in the ascending and descending trend direction over time, the second horizontal component coefficient includes the average number of service requests corresponding to the baseline of the service request within the second preset time period, and the target dynamic horizontal smoothing parameter is used to characterize the sensitivity to the number of service requests to be processed by the online application within the latest preset time period among the N preset time periods, and N is a positive integer;

[0168] In the embodiment of the present application, the data processing device 60 may further include a generating module, which is used to generate a second trend component evaluation coefficient corresponding to the service request within the second preset time period based on the first horizontal component coefficient, the second horizontal component coefficient, the first trend component evaluation coefficient corresponding to the service request within the fourth preset time period, and the target dynamic trend smoothing parameter, wherein the second trend component evaluation coefficient includes a data volume corresponding to a speed of change of the service request within the second preset time period in the ascending and descending trend direction over time, and the target dynamic trend smoothing parameter is used to characterize the sensitivity to the trend change of the number of service requests to be processed by the online application within N preset time periods;

[0169] In the embodiment of the present application, the data processing device 60 may further include a generation module for generating a second number of service requests expected to be processed by the online application within a third preset time period based on a second horizontal component coefficient and a second trend component evaluation coefficient corresponding to the service requests within the second preset time period.

[0170] In some embodiments of the present application, the acquisition module 601 is used to acquire a first sample number of service requests actually processed by the online application in the i-th preset time period among N preset time periods and a second sample number of service requests expected to be processed by the online application in the i-th preset time period; wherein the N preset time periods include at least one preset time period prior to the second preset time period;

[0171] In the embodiment of the present application, the data processing device 60 may further include a generating module for generating a first sample error of the online application processing service request within the i-th preset time period based on the first sample quantity and the second sample quantity;

[0172] In the embodiment of the present application, the data processing device 60 may further include a determination module for determining, when the first sample error is less than or equal to the preset sample error, a first dynamic horizontal smoothing parameter of a preset time period before the second preset time period as a target dynamic horizontal smoothing parameter, and determining a first dynamic trend smoothing parameter of a preset time period before the second preset time period as a target dynamic trend smoothing parameter.

[0173] In some embodiments of the present application, the data processing device 60 in the embodiment of the present application may also include a generation module, which is used to generate an adjustment strategy according to the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter when the first sample error is greater than the preset sample error, and the adjustment strategy includes a first adjustment strategy for simultaneously increasing the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter according to a preset adjustment amplitude; a second adjustment strategy for simultaneously decreasing the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter according to a preset adjustment amplitude; a third adjustment strategy for increasing the first dynamic horizontal smoothing parameter and decreasing the first dynamic trend smoothing parameter according to a preset adjustment amplitude; and a fourth adjustment strategy for decreasing the first dynamic horizontal smoothing parameter and increasing the first dynamic trend smoothing parameter according to a preset adjustment amplitude;

[0174] In the embodiment of the present application, the data processing device 60 may further include an adjustment module, which is used to adjust the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter respectively according to the adjustment strategy, and obtain the second dynamic horizontal smoothing parameter and the second dynamic trend smoothing parameter corresponding to each adjustment strategy;

[0175] In the embodiment of the present application, the data processing device 60 may further include a calculation module, which is used to calculate the number of service requests that are expected to be processed by the online application within the second preset time period corresponding to each adjustment strategy, according to the second dynamic level smoothing parameter and the second dynamic trend smoothing parameter corresponding to each adjustment strategy;

[0176] The generating module may also be used to generate a second sample error of the service requests processed by the online application within the second preset time period corresponding to each adjustment strategy based on a third sample number of service requests expected to be processed by the online application within the second preset time period corresponding to each adjustment strategy and a fourth sample number of service requests actually processed by the online application within the second preset time period; and screen out a target sample error based on the second sample error corresponding to each adjustment strategy, wherein the error value of the target sample error is less than or equal to the preset error value;

[0177] In the embodiment of the present application, the data processing device 60 may further include a determination module for determining the second dynamic horizontal smoothing parameter of the target adjustment strategy corresponding to the target sample error as the target dynamic horizontal smoothing parameter, and determining the second dynamic trend smoothing parameter of the target adjustment strategy corresponding to the target sample error as the target dynamic trend smoothing parameter.

[0178] In some embodiments of the present application, the data processing device 60 in the embodiment of the present application may further include a calculation module, which calculates the first object expansion quantity of the task execution object in the third preset time period according to the object quantity and the preset expansion ratio of the task execution object in the coroutine resource pool in the second preset time period;

[0179] The calculation module may also be used to calculate the second object expansion number of the newly added task execution objects within the third preset time period based on the second number, the preset expansion ratio, the object number of the task execution objects in the coroutine resource pool within the third preset time period, and the preset offset value when the second number is greater than the first object expansion number;

[0180] In the embodiment of the present application, the data processing device 60 may further include a determination module for determining the preset maximum number of objects as the number of objects of the task execution objects in the coroutine resource pool within a third preset time period when the third object expansion number obtained by summing the first object expansion number and the second object expansion number is greater than the preset maximum number of objects.

[0181] In some embodiments of the present application, the data processing device 60 in the embodiments of the present application may also include a determination module for determining the object number of task execution objects in the coroutine resource pool within the second preset time period as the object number of task execution objects in the coroutine resource pool within the third preset time period when the third object expansion number which is the sum of the first object expansion number and the second object expansion number is greater than the preset maximum object number, and the third object expansion number is the same as the preset maximum object number.

[0182] In some embodiments of the present application, the data processing device 60 in the embodiment of the present application may further include a calculation module, which is used to calculate the first reduced number of objects of the task execution objects in the third preset time period according to the number of objects of the task execution objects in the coroutine resource pool in the second preset time period and the preset reduction ratio;

[0183] A calculation module, configured to calculate a second object reduction number of task execution objects to be reduced within a third preset time period based on the object number of task execution objects in the coroutine resource pool within a second preset time period, the second number, and a preset reduction ratio when the second number is less than the first reduced object number;

[0184] In the embodiment of the present application, the data processing device 60 may also include a determination module, which is used to determine the preset minimum number of objects as the number of objects of the task execution objects in the coroutine resource pool within the third preset time period when the third object reduction number obtained by accumulating the first reduced number of objects and the second reduced number of objects is less than the preset minimum number of objects.

[0185] In some embodiments of the present application, the data processing device 60 in the embodiments of the present application may also include a determination module for determining the object number of task execution objects in the coroutine resource pool within the second preset time period as the object number of task execution objects in the coroutine resource pool within the third preset time period when the third object reduction number which is the sum of the first reduced object number and the second object reduction number is less than the preset minimum object number, and the third object reduction number is the same as the preset minimum object number.

[0186] In some embodiments of the present application, the data processing device 60 in the embodiments of the present application may also include an adding module for adding a running resource block configured for the working coroutine to the initial task execution object to which the working coroutine belongs, and a task execution object corresponding to the working coroutine.

[0187] Based on the same inventive concept, the present application also provides a computer device. Figure 7 Provide detailed explanation.

[0188] Figure 7 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application.

[0189] like Figure 7 As shown, the computer device may include at least one of the following involved in the embodiments of the present application: a user terminal device, a server terminal device. The computer device may include a processor 701 and a memory 702 storing computer program instructions.

[0190] Specifically, the processor 701 may include a central processing unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit (ASTC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0191] The memory 702 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 702 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 702 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 702 is a non-volatile solid-state memory. In a specific embodiment, the memory 702 includes a solid-state storage (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0192] The processor 701 implements any one of the data processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 702 .

[0193] In one example, the computer device may further include a communication interface 703 and a bus 710. Figure 7 As shown, the processor 701, the memory 702, and the communication interface 703 are connected via a bus 710 and communicate with each other.

[0194] The communication interface 703 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0195] Bus 710 includes hardware, software or both, and the parts of flow control device are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard system (ETSA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard system (TSA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel system (MCA) bus, peripheral component interconnection (PCT) bus, PCT-Express (PCT-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 710 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0196] The data processing device can execute the data processing method in the embodiment of the present application, thereby realizing the combination Figures 2 to 7 The data processing method and device described.

[0197] In addition, in combination with the data processing method in the above embodiments, the present application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data processing methods in the above embodiments is implemented.

[0198] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0199] The functional blocks shown in the above structured block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0200] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0201] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.< / inuse> < / idle> < / inuse> < / idle> < / inuse> < / idle> < / inuse> < / idle> < / inuse> < / inuse> < / idle> < / idle> < / inuse> < / idle>

Claims

1. A data processing method, comprising: Get the working coroutine from the coroutine resource pool; According to the working coroutine and the running resource block configured for the working coroutine, construct a task execution object corresponding to the working coroutine, wherein the running resource block is used to store the running resources of the online application, and the task execution object is used to process the service request received by the online application based on the running resources; Based on the running state of the task execution object, the running resources stored in the running resource block in the task execution object are updated, wherein the running state includes the state of the task execution object in the process of processing the service request.

2. The method according to claim 1, wherein: The running state includes a ready state, and the ready state is used to indicate that the task execution object is in a state of being unprocessed and waiting for a service request to be processed; The updating of the running resources stored in the running resource block in the task execution object based on the running state of the task execution object includes: Based on the ready state of the task execution object, loading the target running resources of the online application, the target running resources including the running resources of the latest version of the online application; The running resources in the running resource block of the task execution object are replaced with the target running resources.

3. The method according to claim 1, wherein: The running state includes a processing state and a ready state, wherein the processing state is used to indicate that the task execution object is in a state of processing a service request, and the ready state is used to indicate that the task execution object is in a state of not processing and waiting for processing a service request; The updating of the running resources stored in the running resource block in the task execution object based on the running state of the task execution object includes: Based on the processing state of the task execution object, determining the switching condition of the running state of the task execution object within a first preset time period, wherein the switching condition of the running state of the task execution object includes switching from the processing state to the ready state; According to the switching of the running state of the task execution object within the first preset time period, the running resources in the running resource block of the task execution object are updated.

4. The method according to claim 3, wherein: The updating of the operating resources in the operating resource block of the task execution object according to the switching of the operating state of the task execution object within the first preset time period includes: When the running state of the task execution object switches from the processing state to the ready state within the first preset time period, loading the target running resources of the online application, wherein the target running resources include the running resources of the latest version of the online application; The running resources in the running resource block of the task execution object are replaced with the target running resources.

5. The method according to claim 3, wherein: The updating of the operating resources in the operating resource block of the task execution object according to the switching of the operating state of the task execution object within the first preset time period includes: If the running state of the task execution object is not switched from the processing state to the ready state within the first preset time period, loading the target running resources of the online application, wherein the target running resources include the running resources of the latest application version of the online application; Monitoring the progress of the task execution object in processing the service request; When the task execution object completes processing the service request, the running resources in the running resource block of the task execution object are replaced with the target running resources.

6. The method according to claim 3, wherein: The running state also includes a blocking state, and the blocking state is used to indicate that the task execution object is in a state where a service request is being processed and the running resources in the running resource block of the task execution object have been loaded but not updated; The updating of the running resources stored in the running resource block in the task execution object based on the running state of the task execution object includes: Based on the blocking state of the task execution object, deleting the loaded running resources in the running resource block of the task execution object; The running state of the task execution object is adjusted from the blocking state to the processing state, so as to update the running resources by a step of updating the running resources stored in the running resource block in the task execution object corresponding to the processing state.

7. The method according to claim 6, wherein: The deleting the loaded running resources in the running resource block of the task execution object based on the blocking state of the task execution object includes: Based on the blocking state of the task execution object, obtaining a first resource version of the running resource loaded in the running resource block of the task execution object; In the case where the first resource version does not match the preset resource version, the loaded running resources in the running resource block of the task execution object are deleted; wherein the preset resource version is the resource version of the target running resources corresponding to the latest version of the online application.

8. The method according to claim 1, wherein: Before updating the running resources stored in the running resource block in the task execution object based on the running state of the task execution object, the method further includes: Obtaining, through the resource reloading coroutine, a second resource version of the running resource in the running resource block of the task execution object; The updating of the running resources stored in the running resource block in the task execution object based on the running state of the task execution object includes: When the second resource version does not match the preset resource version, the running resources stored in the running resource block in the task execution object are updated based on the running status of the task execution object, wherein the preset resource version is the resource version of the target running resources corresponding to the latest version of the online application.

9. The method according to claim 1, wherein: The method further comprises: generating a second number of service requests that the online application is expected to process within a third preset time period according to a first number of service requests to be processed by the online application within a second preset time period; wherein the second preset time period occurs before the third preset time period; Based on the second number, the number of task execution objects in the coroutine resource pool within the third preset time period is adjusted.

10. The method according to claim 9, wherein: The step of generating a second number of service requests that the online application is expected to process within a third preset time period according to the first number of service requests to be processed by the online application within the second preset time period includes: A second horizontal component coefficient corresponding to the service requests within the second preset time period is generated according to the first number of service requests received by the online application within the second preset time period, the target dynamic horizontal smoothing parameter, the first horizontal component coefficient corresponding to the service requests within the fourth preset time period, and the first trend component evaluation coefficient; wherein the fourth preset time period occurs before the second preset time period, the first horizontal component coefficient includes the average number of service requests corresponding to the baseline of the service requests within the fourth preset time period, the first trend component evaluation coefficient includes the amount of data corresponding to the speed of change of the service requests within the fourth preset time period in the direction of the rising and falling trend over time, the second horizontal component coefficient includes the average number of service requests corresponding to the baseline of the service requests within the second preset time period, and the target dynamic horizontal smoothing parameter is used to characterize the sensitivity to the number of service requests to be processed by the online application in the latest preset time period among N preset time periods, where N is a positive integer; Based on the first horizontal component coefficient, the second horizontal component coefficient, the first trend component evaluation coefficient corresponding to the service request within the fourth preset time period, and the target dynamic trend smoothing parameter, a second trend component evaluation coefficient corresponding to the service request within the second preset time period is generated, wherein the second trend component evaluation coefficient includes a data volume corresponding to a speed of change of the service request within the second preset time period in an ascending and descending trend direction over time, and the target dynamic trend smoothing parameter is used to characterize a sensitivity to a trend change of the number of service requests to be processed by the online application within N preset time periods; Based on the second horizontal component coefficient and the second trend component evaluation coefficient corresponding to the service requests within the second preset time period, a second number of service requests that the online application is expected to process within a third preset time period is generated.

11. The method according to claim 10, wherein: The method further comprises: Obtaining a first sample number of service requests actually processed by the online application in an i-th preset time period among the N preset time periods and a second sample number of service requests expected to be processed by the online application in the i-th preset time period; wherein the N preset time periods include at least one preset time period prior to the second preset time period; generating a first sample error of the online application processing service request within the i-th preset time period based on the first sample quantity and the second sample quantity; When the first sample error is less than or equal to the preset sample error, the first dynamic horizontal smoothing parameter of the previous preset time period of the second preset time period is determined as the target dynamic horizontal smoothing parameter, and the first dynamic trend smoothing parameter of the previous preset time period of the second preset time period is determined as the target dynamic trend smoothing parameter.

12. The method according to claim 11, wherein: After generating a first sample error of the online application processing service request within the i-th preset time period based on the first sample quantity and the second sample quantity, the method further includes: In the case where the first sample error is greater than the preset sample error, an adjustment strategy is generated according to the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter, the adjustment strategy comprising a first adjustment strategy for increasing the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter at the same time according to a preset adjustment amplitude; a second adjustment strategy for decreasing the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter at the same time according to a preset adjustment amplitude; a third adjustment strategy for increasing the first dynamic horizontal smoothing parameter and decreasing the first dynamic trend smoothing parameter according to a preset adjustment amplitude; and a fourth adjustment strategy for decreasing the first dynamic horizontal smoothing parameter and increasing the first dynamic trend smoothing parameter according to a preset adjustment amplitude; According to the adjustment strategy, the first dynamic horizontal smoothing parameter and the first dynamic trend smoothing parameter are adjusted respectively to obtain a second dynamic horizontal smoothing parameter and a second dynamic trend smoothing parameter corresponding to each adjustment strategy; According to the second dynamic level smoothing parameter and the second dynamic trend smoothing parameter corresponding to each adjustment strategy, respectively calculating the number of service requests expected to be processed by the online application within the second preset time period corresponding to each adjustment strategy; Based on the third sample number of service requests expected to be processed by the online application within the second preset time period generated corresponding to each adjustment strategy and the fourth sample number of service requests actually processed by the online application within the second preset time period, a second sample error of service requests processed by the online application within the second preset time period corresponding to each adjustment strategy is generated; based on the second sample error corresponding to each adjustment strategy, a target sample error is screened out, and the error value of the target sample error is less than or equal to the preset error value; The second dynamic horizontal smoothing parameter of the target adjustment strategy corresponding to the target sample error is determined as the target dynamic horizontal smoothing parameter, and the second dynamic trend smoothing parameter of the target adjustment strategy corresponding to the target sample error is determined as the target dynamic trend smoothing parameter.

13. The method according to claim 9, wherein: The adjusting, based on the second quantity, the quantity of the task execution objects in the coroutine resource pool within the third preset time period includes: Calculate a first object expansion quantity of the task execution object in the third preset time period according to the object quantity of the task execution object in the coroutine resource pool in the second preset time period and the preset expansion ratio; When the second number is greater than the first object expansion number, based on the second number, the preset expansion ratio, the number of task execution objects in the coroutine resource pool within the third preset time period, and the preset offset value, the second object expansion number of the newly added task execution objects within the third preset time period is calculated; When a third object expansion number obtained by accumulating the first object expansion number and the second object expansion number is greater than a preset maximum object number, the preset maximum object number is determined as the object number of task execution objects in the coroutine resource pool within the third preset time period.

14. The method according to claim 13, wherein: The adjusting, based on the second quantity, the quantity of the task execution objects in the coroutine resource pool within the third preset time period includes: When the third object expansion number which is the sum of the first object expansion number and the second object expansion number is greater than the preset maximum object number, and the third object expansion number is the same as the preset maximum object number, the object number of task execution objects in the coroutine resource pool within the second preset time period is determined as the object number of task execution objects in the coroutine resource pool within the third preset time period.

15. The method according to claim 9, wherein: The adjusting, based on the second quantity, the quantity of the task execution objects in the coroutine resource pool within the third preset time period includes: Calculate a first reduced number of task execution objects within the third preset time period according to the number of task execution objects in the coroutine resource pool within the second preset time period and a preset reduction ratio; When the second number is less than the first reduced number of objects, based on the number of objects of task execution objects in the coroutine resource pool within the second preset time period, the second number and the preset reduction ratio, a second reduced number of objects for reducing task execution objects within the third preset time period is calculated; When a third object reduction number obtained by accumulating the first reduced object number and the second reduced object number is less than a preset minimum object number, the preset minimum object number is determined as the object number of task execution objects in the coroutine resource pool within the third preset time period.

16. The method according to claim 15, wherein: The adjusting, based on the second quantity, the quantity of the task execution objects in the coroutine resource pool within the third preset time period includes: When the third object reduction number which is the sum of the first reduced object number and the second object reduction number is less than the preset minimum object number, and the third object reduction number is the same as the preset minimum object number, the object number of task execution objects in the coroutine resource pool within the second preset time period is determined as the object number of task execution objects in the coroutine resource pool within the third preset time period.

17. The method according to claim 1, wherein: The step of constructing a task execution object corresponding to the work coroutine according to the work coroutine and the running resource block configured for the work coroutine includes: Add the running resource block configured for the working coroutine to the initial task execution object to which the working coroutine belongs, and the task execution object corresponding to the working coroutine.

18. A data processing device, comprising: The acquisition module is used to obtain the working coroutine in the coroutine resource pool; A construction module, configured to construct a task execution object corresponding to the work coroutine according to the work coroutine and a running resource block configured for the work coroutine, wherein the running resource block is used to store running resources of the online application, and the task execution object is used to process a service request received by the online application based on the running resources; The updating module is used to update the running resources stored in the running resource block in the task execution object based on the running state of the task execution object, wherein the running state includes the state of the task execution object in the process of processing the service request.

19. A computer device, the device comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the data processing method according to any one of claims 1 to 17 is implemented.

20. A storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the data processing method according to any one of claims 1 to 17.

Images