Task processing method and device, computer device, readable storage medium and program product

Abstract

The application relates to a task processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises: obtaining a configuration distribution retry task of network configuration information; performing sharding processing on the configuration distribution retry task based on a preset sharding strategy to obtain a first sharding task of the configuration distribution retry task; performing secondary sharding processing on the first sharding task based on secondary sharding parameters to obtain a secondary sharding task of the configuration distribution retry task; publishing the secondary sharding task to a task processing queue; and performing concurrent processing on the secondary sharding task by subscribing to the task processing queue. The application integrates a secondary decomposition and distribution mechanism of the configuration distribution retry task. According to the set secondary sharding parameters, the system finely divides the task data after the first sharding, and pushes the sharded data to the task processing queue in real time for concurrent processing. Therefore, the processing efficiency of the SDN network configuration distribution retry process is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a task processing method and device, computer equipment, a computer readable storage medium and a computer program product. BACKGROUND

[0002] With the development of computer technology and software technology, the technology of software defined network (SDN) emerges, the core technology of which is to separate the control plane from the data plane of network equipment, thereby realizing flexible control of network traffic, making the network as a pipe more intelligent, and providing a good platform for the innovation of core network and application. In a typical SDN system, a global controller serves as a central hub, responsible for managing all network elements of N resource pools and issuing network configurations.

[0003] Due to sudden reasons such as system failure, network failure, and device failure, network configuration issuing often has a probability of failure. If failure occurs, rollback is often performed, which often causes frequent failure of configuration issuing tasks and affects user experience. The current processing idea expects to record instances of configuration issuing failure and repeatedly retry the tasks multiple times until the problem is solved. At the same time, the operation and maintenance personnel also need to solve various problem points from the perspective of task failure. These two parts are decoupled in processing logic to ensure the simplicity of the system, but the number and frequency of retries need to be increased accordingly to ensure the timeliness of problem solving.

[0004] At present, a distributed task scheduling system can be used to meet the needs of configuration retry batch processing business. However, since these systems only provide task division at the order of executors, the ability of concurrent execution of tasks completely depends on the number of nodes in the executor cluster, and no concurrent framework is provided for a single executor, so the concurrent ability is limited, and there is still a performance bottleneck when processing a large number of network configuration data, affecting the processing efficiency of the configuration reissue process. SUMMARY

[0005] Therefore, it is necessary to provide a task processing method, device, computer equipment, computer readable storage medium and computer program product capable of improving the processing efficiency of the SDN network configuration issuing retry process to solve the above technical problems.

[0006] In a first aspect, the present application provides a task processing method, comprising:

[0007] obtaining a configuration issuing retry task of network configuration information;

[0008] performing sharding processing on the configuration issuing retry task based on a preset sharding strategy to obtain a sharded task of the configuration issuing retry task.

[0009] performing secondary slicing processing on the primary slicing task based on secondary slicing parameters to obtain the secondary slicing task of the configuration distribution retry task;

[0010] publishing the secondary slicing task to a task processing queue;

[0011] performing concurrent processing on the secondary slicing task in the task processing queue by subscribing to the task processing queue.

[0012] In one of the embodiments, the publishing the secondary slicing task to a task processing queue comprises:

[0013] publishing the secondary slicing task to a task processing queue of a message middleware;

[0014] The performing concurrent processing on the secondary slicing task in the task processing queue by subscribing to the task processing queue comprises:

[0015] performing concurrent processing on the secondary slicing task in the task processing queue in a message consumer manner based on a multi-thread processing mechanism of the message middleware by subscribing to the task processing queue.

[0016] In one of the embodiments, the performing secondary slicing processing on the primary slicing task based on secondary slicing parameters to obtain the secondary slicing task of the configuration distribution retry task comprises:

[0017] obtaining a configuration type of the network configuration information;

[0018] in a case where the configuration type is a core configuration, searching for configuration size, network type and network bandwidth information of the configuration distribution retry task;

[0019] determining secondary slicing parameters corresponding to the configuration size, network type and network bandwidth information;

[0020] performing secondary slicing processing on the primary slicing task based on secondary slicing parameters to obtain the secondary slicing task of the configuration distribution retry task.

[0021] In one of the embodiments, the method further comprises:

[0022] in a case where the configuration type is a non-core configuration, determining a task execution priority corresponding to the configuration distribution retry task;

[0023] determining a secondary slicing execution order of the configuration distribution retry task based on the task execution priority.

[0024] In one of the embodiments, the determining of the task execution priority corresponding to the configuration distribution retry task in the case that the configuration type is a non-core configuration comprises:

[0025] In the case that the configuration type is a non-core configuration, the current overall retry quantity of the configuration distribution retry task and system real-time load information are found;

[0026] Based on a preset dynamic secondary slicing strategy, the task execution priority corresponding to the current overall retry quantity and the system real-time load information is determined.

[0027] In one of the embodiments, the method further comprises:

[0028] The execution state and execution result of the secondary slicing task are acquired;

[0029] Based on the execution state and execution result of the secondary slicing task, the execution state and execution result of the configuration distribution retry task are determined;

[0030] The execution state and execution result of the configuration distribution retry task are fed back.

[0031] In a second aspect, the present application further provides a task processing device, comprising:

[0032] A task acquisition module is configured to acquire a configuration distribution retry task of network configuration information;

[0033] A first slicing module is configured to perform slicing processing on the configuration distribution retry task based on a preset slicing strategy, to obtain a primary slicing task of the configuration distribution retry task;

[0034] A second slicing module is configured to perform secondary slicing processing on the primary slicing task based on secondary slicing parameters, to obtain a secondary slicing task of the configuration distribution retry task;

[0035] A task publishing module is configured to publish the secondary slicing task to a task processing queue;

[0036] A task processing module is configured to perform concurrent processing on the secondary slicing task in the task processing queue by subscribing to the task processing queue.

[0037] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0038] A configuration distribution retry task of network configuration information is acquired;

[0039] The configuration distribution retry task is processed by slicing based on a preset slicing strategy, to obtain a first slicing task of the configuration distribution retry task;

[0040] The first slicing task is processed by secondary slicing based on secondary slicing parameters, to obtain a second slicing task of the configuration distribution retry task;

[0041] The second slicing task is published to a task processing queue;

[0042] The second slicing task in the task processing queue is processed concurrently by subscribing to the task processing queue.

[0043] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0044] A configuration distribution retry task of network configuration information is acquired;

[0045] The configuration distribution retry task is processed by slicing based on a preset slicing strategy, to obtain a first slicing task of the configuration distribution retry task;

[0046] The first slicing task is processed by secondary slicing based on secondary slicing parameters, to obtain a second slicing task of the configuration distribution retry task;

[0047] The second slicing task is published to a task processing queue;

[0048] The second slicing task in the task processing queue is processed concurrently by subscribing to the task processing queue.

[0049] In a fifth aspect, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the following steps:

[0050] A configuration distribution retry task of network configuration information is acquired;

[0051] The configuration distribution retry task is processed by slicing based on a preset slicing strategy, to obtain a first slicing task of the configuration distribution retry task;

[0052] The first slicing task is processed by secondary slicing based on secondary slicing parameters, to obtain a second slicing task of the configuration distribution retry task;

[0053] The second slicing task is published to a task processing queue;

[0054] The second slicing task in the task processing queue is processed concurrently by subscribing to the task processing queue.

[0055] The task processing method, device, computer device, computer readable storage medium and computer program product can obtain a configuration distribution retry task by obtaining network configuration information, perform slicing processing on the configuration distribution retry task based on a preset slicing strategy to obtain a first slicing task of the configuration distribution retry task, perform secondary slicing processing on the first slicing task based on secondary slicing parameters to obtain a secondary slicing task of the configuration distribution retry task, publish the secondary slicing task to a task processing queue, and perform concurrent processing on the secondary slicing task in the task processing queue by subscribing to the task processing queue. The secondary decomposition and distribution mechanism of the configuration distribution retry task is integrated. In the task distribution link, the system performs fine division on the task data after the first slicing according to the set secondary slicing parameters, and pushes the sliced data to the task processing queue in real time. Then, the secondary slicing task at the slicing data level is obtained by subscribing to the task processing queue, and the multi-task parallel execution in a single node is realized through the multi-thread processing capability, so that the processing efficiency of the SDN network configuration distribution retry process is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other related drawings can be obtained by those skilled in the art without creative labor.

[0057] Figure 1 The application environment diagram of the task processing method in an embodiment;

[0058] Figure 2 The flowchart of the task processing method in an embodiment;

[0059] Figure 3 The system structure diagram of the task processing system based on the task processing method in an embodiment;

[0060] Figure 4 The structure block diagram of the task processing device in an embodiment;

[0061] Figure 5 The internal structure diagram of the computer device in an embodiment. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0063] The task processing method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 The scheduling center server 102 communicates with the executor cluster server 104 through a network, and the executor cluster server 104 communicates with the message queue server 106 through a network. The data storage system can store data required to be processed by the scheduling center server 102. The data storage system can be integrated on the scheduling center server 102, or placed on a cloud or other network server. The scheduling center server 102 can register a network configuration information configuration distribution retry task and implement scheduling processing of the task. When the scheduling center server 102 receives the network configuration information configuration distribution retry task, the scheduling center server 102 sends the network configuration information configuration distribution retry task to the executor cluster server 104 for corresponding processing. The executor cluster server 104 obtains the network configuration information configuration distribution retry task; performs sharding processing on the configuration distribution retry task based on a preset sharding strategy to obtain a first sharding task of the configuration distribution retry task; performs secondary sharding processing on the first sharding task based on a secondary sharding parameter to obtain a secondary sharding task of the configuration distribution retry task; publishes the secondary sharding task to a task processing queue of the message queue server 106; and then performs concurrent processing on the secondary sharding task in the task processing queue by subscribing to the task processing queue. The processing result is returned to the scheduling center server 102. The scheduling center server 102, the executor cluster server 104 and the message queue server 106 can be independent physical servers, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0064] In an exemplary embodiment, as shown in Figure 2 A task processing method is provided. The method is applied to the executor cluster server 104 in Figure 1 for example, and includes the following steps 201 to 209. Wherein:

[0065] Step 201, a network configuration information configuration distribution retry task is obtained.

[0066] Step 203, the configuration distribution retry task is sharded based on a preset sharding strategy to obtain a first sharding task of the configuration distribution retry task.

[0067] The network configuration information is specifically network configuration information of a software defined network (SDN). In a typical SDN system, a global controller serves as a central hub, is responsible for managing all network elements of N resource pools, and issues network configuration. Due to sudden reasons such as system failure, network failure, and device failure, the configuration issuing often fails with a certain probability. At this time, the generated is a configuration issuing retry task. The preset sharding strategy is an execution strategy for implementing task sharding processing, such as providing a number of shards, sharding based on an average allocation algorithm (similar to paging, providing a total number of pages, and each executor obtaining a page number to be processed), determining an IP ascending / descending order algorithm according to an odd / even number of a hash value of a job name, and rotating a list of executor instances according to the hash value of the job name. A single machine multi-task mode is configured with multiple tasks, each task is specified with different parameters, but uses the same task processing class (jobhanlder). A sharding broadcast strategy is used to schedule a timing task through "sharding broadcast". The broadcast triggers all executors in a corresponding cluster to execute a task once, and the system automatically delivers sharding parameters to the executors. The executors obtain data to be processed according to the sharding parameters. Various different sharding strategies can be selected according to specific business needs and user settings to select the preset sharding strategy.

[0068] Exemplarily, for the network configuration information, if the rollback is performed when the downlink fails, the configuration downlink task is often failed frequently, which affects the user experience. At this time, the configuration downlink processing is realized by configuring the downlink retry task. The configuration downlink failure instance is recorded and the repeated task retry is performed, that is, the configuration downlink retry task, and the repeated task retry is gathered together, which forms a batch processing business scenario. Such batch processing tasks must be started and processed in a timely manner, so the performance requirement of the batch processing system is very high. With the continuous expansion of the business and the continuous increase of the configuration, the order of magnitude of the configuration downlink task that needs to be retried is rising, and the processing capacity of the single machine task system often cannot be borne. At this time, the processing efficiency can be improved by sharding processing and distributed scheduling coordination. Therefore, after the executor cluster server 104 obtains the configuration downlink retry task, the configuration downlink retry task can be sharded based on a preset sharding strategy to obtain a sharded task of the configuration downlink retry task. The task segmentation process of the sharded task mainly depends on the number of executors of the executor cluster server 104. In specific embodiments, the executors in the executor cluster server 104 can be used for task sharding processing. After each node of the executor cluster is started, each executor node registers itself with the scheduling center server 102, provides its own cluster name, and sends a heartbeat signal at regular intervals to inform the survival state. When the task is started in a timely manner, the scheduling center server 102 initiates scheduling to the nodes in the configured executor cluster. Each executor node processes the data according to its own sharding parameters. For example, the total number of shards configured by the scheduling center is 20, there are 5 nodes in the cluster, and the shards are allocated in the manner of “shard number mod node total number = executor number”. The shard numbers allocated to the executor 0 are 0, 5, 10, and 15. If the total amount of data to be processed is 10000, the size of each shard is 500, and the executor 0 can process the 0th-499th, 2000th-2499th, 4500th-4999th, and 7000th-7499th data.

[0069] In step 205, the sharded task is sharded based on the secondary sharding parameters to obtain a secondary sharded task of the configuration downlink retry task.

[0070] The secondary sharded task refers to further splitting the task data after the first sharding into smaller data shards by introducing secondary sharding parameters, so as to control the concurrent processing and retry logic in a more fine-grained manner. The secondary sharding parameters refer to the dependent data of the secondary sharding process, which can specifically include business requirements and data size.

[0071] Exemplarily, the primary slicing mainly implements task division in the order of the number of executors, while the ability of concurrent execution of the task completely depends on the number of nodes in the executor cluster, and no concurrent framework is provided for a single executor, so the concurrent ability is limited, and there is still a performance bottleneck in processing a large number of network configuration data. In this regard, the applicant proposes secondary slicing processing based on the primary slicing task. The task data of the primary slicing task is disassembled into small data slices according to the secondary slicing parameters, so as to more finely control the concurrent processing and retry logic.

[0072] In step 207, the secondary slicing task is published to the task processing queue.

[0073] In step 209, the secondary slicing task in the task processing queue is processed concurrently by subscribing to the task processing queue.

[0074] Exemplarily, after the secondary slicing task is split, the secondary slicing task can be published to the task processing queue, and then more efficient concurrent processing of the secondary slicing task is performed based on the task processing queue. In the scheme of the present application, the executors in the executor cluster server 104 undertake both task disassembly and distribution and task execution. In the task disassembly and distribution category, according to the slicing parameters passed by the system, the data to be processed by the executor is obtained, the data is disassembled into small data slices according to the secondary slicing parameters configured by the scheduling center, and is sent to the slicing queue of the task. Then, the task execution category actively subscribes to the slicing queue of the task as a message consumer, and implements business processing in the consumption logic. Thus, the concurrent real-time processing capability of network configuration distribution retry is further improved.

[0075] The above task processing method obtains a configuration distribution retry task of network configuration information, performs slicing processing on the configuration distribution retry task based on a preset slicing strategy to obtain a primary slicing task of the configuration distribution retry task, performs secondary slicing processing on the primary slicing task based on secondary slicing parameters to obtain a secondary slicing task of the configuration distribution retry task, publishes the secondary slicing task to a task processing queue, and processes the secondary slicing task in the task processing queue concurrently by subscribing to the task processing queue. The present application integrates a secondary disassembly and distribution mechanism of the configuration distribution retry task. In the task distribution link, according to the set secondary slicing parameters, the system finely divides the task data after primary slicing, and pushes the sliced data to the task processing queue in real time. Then, the secondary slicing task at the slicing data level is obtained by subscribing to the task processing queue, and the multi-thread processing capability is used to realize multi-task parallel execution in a single node, thereby effectively improving the processing efficiency of the SDN network configuration distribution retry process.

[0076] In an exemplary embodiment, step 207 comprises: publishing the secondary slicing task to a task processing queue of the message middleware. Step 209 comprises: based on the multi-thread processing mechanism of the message middleware, subscribing to the task processing queue to perform concurrent processing on the secondary slicing task in the task processing queue in a message consumer manner.

[0077] The message oriented middleware (MOM) is a kind of software supporting synchronous or asynchronous sending and receiving of messages between distributed applications. The message oriented middleware expands inter-process communication in a distributed environment by providing a message passing and message queue model, supports multiple communication protocols, languages, applications, hardware and software platforms. The message oriented middleware supports asynchronous communication, and the sender does not need to know the state of the receiver when sending a message, nor does it need to wait for the reply of the receiver. This loosely coupled connection makes the communication non-blocking. Through an efficient and reliable message passing mechanism, the message oriented middleware can realize platform-independent data exchange, shield the characteristics between various platforms and protocols, and realize the cooperation between applications. The message oriented middleware can expand inter-process communication in a distributed system. The scheme of the present application decouples the producer and the consumer of the configuration distribution retry task by introducing the message oriented middleware as a bridge for task distribution and message passing, thereby realizing asynchronous communication and high-concurrency processing.

[0078] In an exemplary embodiment, the scheme of the present application realizes the publishing and execution processing of the secondary slicing task through the message oriented middleware. After the task decomposition class decomposes the configuration distribution retry task into secondary slicing tasks, the task data of the secondary slicing task can be pushed to the message oriented middleware in real time. Then, the message subscription processing can be realized through the task execution class, that is, based on the multi-thread processing mechanism of the message oriented middleware, the task execution class subscribes to the task processing queue to perform concurrent processing on the secondary slicing task in the task processing queue in a message consumer manner. The task execution class obtains the slicing data by subscribing to the message oriented middleware, and realizes the parallel execution of multiple tasks within a single node by virtue of the multi-thread processing capability thereof. In the present embodiment, the message oriented middleware is introduced as a bridge for task distribution and message passing, which can decouple the producer and the consumer of the task, realize asynchronous communication and high-concurrency processing, and thereby improve the efficiency of concurrent processing of the secondary slicing task.

[0079] In an exemplary embodiment, step 205 comprises: obtaining a configuration type of the network configuration information; in the case that the configuration type is a core configuration, searching for configuration size, network type and network bandwidth information of the configuration distribution retry task; determining secondary slicing parameters corresponding to the configuration size, network type and network bandwidth information; and performing secondary slicing processing on the primary slicing task based on the secondary slicing parameters to obtain the secondary slicing task of the configuration distribution retry task.

[0080] Exemplarily, the core configuration is a kind of attribute of the configuration delivery retry task, and whether the configuration delivery retry task belongs to the core configuration can be determined based on the configuration scenario of the configuration delivery retry task. For the core configuration, forced secondary slicing processing is required. Therefore, in the case of the configuration type being the core configuration, the configuration size, network type and network bandwidth information of the configuration delivery retry task are found. Thus, the secondary slicing parameters are obtained by dynamically adjusting the slicing size according to the configuration size, network type, network bandwidth and the like, and finally the secondary slicing processing is performed on the primary slicing task based on the secondary slicing parameters to obtain the secondary slicing task of the configuration delivery retry task. In this embodiment, forced secondary slicing processing is performed on the configuration delivery retry task of the core configuration, and the secondary slicing parameters are determined according to the configuration size, network type and network bandwidth information, so as to finely divide the task data and ensure the efficiency and accuracy of task division.

[0081] Further, the method further comprises: in the case of the configuration type being a non-core configuration, determining the task execution priority corresponding to the configuration delivery retry task; and determining the secondary slicing execution order of the configuration delivery retry task based on the task execution priority.

[0082] Exemplarily, in the case of the task type of the configuration delivery retry task being a non-core configuration, it is explained that these configuration delivery retry tasks do not need to be executed immediately, and at this time, the task execution priority corresponding to the configuration delivery retry task is determined; the secondary slicing execution order of the configuration delivery retry task is determined based on the task execution priority, and then the secondary slicing processing of these tasks is executed based on the secondary slicing execution order. In a specific embodiment, for the identification process of the task execution priority, the current overall retry quantity and system real-time load information of the configuration delivery retry task can be found; based on the preset dynamic secondary slicing strategy, the task execution priority corresponding to the current overall retry quantity and system real-time load information is determined. Thus, according to the total quantity of the current retry task and the system real-time load and other multiple factors, it is intelligently judged whether to perform secondary slicing and the execution priority of secondary slicing, so as to ensure the efficient use of system resources and the flexibility of task processing.

[0083] In an exemplary embodiment, the method further comprises: obtaining the execution state and execution result of the secondary slicing task; determining the execution state and execution result of the configuration delivery retry task based on the execution state and execution result of the secondary slicing task; and feeding back the execution state and execution result of the configuration delivery retry task.

[0084] Exemplarily, after the secondary fragmented tasks in the task processing queue are processed concurrently by subscribing to the task processing queue, the processing results of the concurrent processing are aggregated to realize the collated task state analysis. Therefore, during the concurrent processing of the secondary fragmented tasks in the task processing queue by subscribing to the task processing queue, the system automatically aggregates the execution state and the execution result of the secondary fragmented tasks, and determines the execution state and the execution result of the configuration distribution retry task based on the execution state and the execution result of the secondary fragmented tasks. Finally, the execution state and the execution result of the configuration distribution retry task are fed back to the scheduling center, so that the scheduling center determines the final task execution effect. At this time, the overall execution flowchart of the present scheme can be referred to Figure 3 In the embodiment, the execution state and the execution result of the secondary fragmented tasks are aggregated to complete the feedback of the task execution effect corresponding to the configuration distribution retry task, so that the scheduling center can be fed back the task execution progress in a timely and effective manner.

[0085] In a specific embodiment, the task processing method of the present application comprises: obtaining a configuration distribution retry task of network configuration information; performing fragmentation processing on the configuration distribution retry task based on a preset fragmentation strategy to obtain a primary fragmented task of the configuration distribution retry task; obtaining a configuration type of the network configuration information; in the case that the configuration type is a core configuration, searching for configuration size, network type and network bandwidth information of the configuration distribution retry task; determining secondary fragmentation parameters corresponding to the configuration size, the network type and the network bandwidth information; performing secondary fragmentation processing on the primary fragmented task based on the secondary fragmentation parameters to obtain secondary fragmented tasks of the configuration distribution retry task; in the case that the configuration type is a non-core configuration, determining a task execution priority corresponding to the configuration distribution retry task; determining a secondary fragmentation execution order of the configuration distribution retry task based on the task execution priority; publishing the secondary fragmented tasks to a task processing queue of a message middleware; based on a multi-thread processing mechanism of the message middleware, subscribing to the task processing queue to process the secondary fragmented tasks in the task processing queue in a message consumer manner. Obtaining the execution state and the execution result of the secondary fragmented tasks; determining the execution state and the execution result of the configuration distribution retry task based on the execution state and the execution result of the secondary fragmented tasks; feeding back the execution state and the execution result of the configuration distribution retry task.

[0086] It should be understood that although each step in the flowchart involved in the above embodiments is shown in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of the secondary division of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0087] Based on the same inventive concept, the embodiments of the present application also provide a task processing device for implementing the above-mentioned task processing method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more task processing device embodiments provided below can refer to the limitations of the task processing method in the above text, which will not be repeated here.

[0088] In one exemplary embodiment, as shown in Figure 4 a task processing device is provided, comprising:

[0089] The task acquisition module 401 is configured to acquire a configuration delivery retry task of network configuration information.

[0090] The first division module 403 is configured to perform division processing on the configuration delivery retry task based on a preset division strategy, to obtain a first division task of the configuration delivery retry task.

[0091] The second division module 405 is configured to perform secondary division processing on the first division task based on secondary division parameters, to obtain a secondary division task of the configuration delivery retry task.

[0092] The task publishing module 407 is configured to publish the secondary division task to a task processing queue.

[0093] The task processing module 409 is configured to perform concurrent processing on the secondary division task in the task processing queue by subscribing to the task processing queue.

[0094] In one embodiment, the task publishing module 407 is specifically configured to publish the secondary division task to a task processing queue of a message middleware. The task processing module 409 is specifically configured to perform concurrent processing on the secondary division task in the task processing queue in a message consumer manner by subscribing to the task processing queue based on a multi-thread processing mechanism of the message middleware

[0095] In an embodiment, the second fragmentation module 405 is specifically configured to: acquire a configuration type of the network configuration information; in a case where the configuration type is a core configuration, find configuration size, network type and network bandwidth information of the configuration delivery retry task; determine secondary fragmentation parameters corresponding to the configuration size, the network type and the network bandwidth information; and perform secondary fragmentation processing on the primary fragmentation task based on the secondary fragmentation parameters to obtain the secondary fragmentation task of the configuration delivery retry task.

[0096] In an embodiment, the second fragmentation module 405 is specifically configured to: in a case where the configuration type is a non-core configuration, determine a task execution priority corresponding to the configuration delivery retry task; and determine a secondary fragmentation execution order of the configuration delivery retry task based on the task execution priority.

[0097] In an embodiment, the second fragmentation module 405 is specifically configured to: in a case where the configuration type is a non-core configuration, find current overall retry quantity and system real-time load information of the configuration delivery retry task; and determine a task execution priority corresponding to the current overall retry quantity and the system real-time load information based on a preset dynamic secondary fragmentation strategy.

[0098] In an embodiment, the task state feedback module is further configured to: acquire an execution state and an execution result of the secondary fragmentation task; determine an execution state and an execution result of the configuration delivery retry task based on the execution state and the execution result of the secondary fragmentation task; and feed back the execution state and the execution result of the configuration delivery retry task.

[0099] The above modules can be all or partially implemented by software, hardware and combinations thereof. The above modules can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above modules.

[0100] In an exemplary embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in FIG. 4. Figure 5As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store task processing data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to implement a task processing method.

[0101] Those skilled in the art can understand that, Figure 5 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0102] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in each of the above method embodiments.

[0103] In one embodiment, a computer readable storage medium is provided, storing a computer program, which is executed by a processor to implement the steps in each of the above method embodiments.

[0104] In one embodiment, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in each of the above method embodiments.

[0105] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0106] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0107] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0108] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A task processing method, characterized in that, The method includes: Retrieves network configuration information and performs a configuration retry task. The configuration delivery retry task is segmented based on a preset segmentation strategy to obtain a segmented task of the configuration delivery retry task. Obtain the configuration type of the network configuration information; if the configuration type is core configuration, find the configuration size, network type, and network bandwidth information of the configuration retry task; dynamically adjust the fragment size according to the configuration size, network type, and network bandwidth information to determine the secondary fragmentation parameters; perform secondary fragmentation processing on the primary fragmentation task based on the secondary fragmentation parameters to obtain the secondary fragmentation task of the configuration retry task; The secondary sharding task is published to the task processing queue; By subscribing to the task processing queue, the secondary sharded tasks in the task processing queue are processed concurrently.

2. The method according to claim 1, characterized in that, The step of publishing the secondary sharding task to the task processing queue includes: The secondary sharding task is published to the task processing queue of the message middleware; The step of concurrently processing the secondary-sharded tasks in the task processing queue by subscribing to the task processing queue includes: Based on the multi-threaded processing mechanism of the message middleware, the secondary sharded tasks in the task processing queue are processed concurrently as message consumers by subscribing to the task processing queue.

3. The method according to claim 1, characterized in that, The method further includes: When the configuration type is non-core configuration, determine the task execution priority corresponding to the retry task issued by the configuration. The secondary sharding execution order of the retry task is determined based on the task execution priority.

4. The method according to claim 3, characterized in that, When the configuration type is non-core configuration, determining the task execution priority corresponding to the retry task issued by the configuration includes: If the configuration type is non-core configuration, find the current total number of retries and the real-time system load information for the retry tasks issued under the configuration. Based on a preset dynamic secondary sharding strategy, the current overall number of retries and the task execution priority corresponding to the real-time system load information are determined.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Obtain the execution status and execution result of the secondary sharding task; Based on the execution status and execution result of the secondary sharding task, determine the execution status and execution result of the configuration-issued retry task; The system provides feedback on the execution status and result of the retry task issued under the specified configuration.

6. A task processing device, characterized in that, The device includes: The task acquisition module is used to acquire network configuration information and configure retry tasks. The first sharding module is used to shard the configuration delivery retry task based on a preset sharding strategy to obtain a sharded task of the configuration delivery retry task. The second sharding module is used to obtain the configuration type of the network configuration information; when the configuration type is core configuration, it searches for the configuration size, network type, and network bandwidth information of the configuration retry task; dynamically adjusts the sharding size according to the configuration size, network type, and network bandwidth information to determine the secondary sharding parameters; and performs secondary sharding processing on the primary sharding task based on the secondary sharding parameters to obtain the secondary sharding task of the configuration retry task. The task publishing module is used to publish the secondary sharding task to the task processing queue; The task processing module is used to concurrently process the secondary sharded tasks in the task processing queue by subscribing to the task processing queue.

7. The apparatus according to claim 6, characterized in that, The task publishing module is specifically used to: publish the secondary sharding task to the task processing queue of the message middleware; the concurrent processing of the secondary sharding task in the task processing queue by subscribing to the task processing queue includes: based on the multi-threaded processing mechanism of the message middleware, by subscribing to the task processing queue, the concurrent processing of the secondary sharding task in the task processing queue is performed in the manner of a message consumer.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images