Distributed task processing method, device, system, electronic device and storage medium

Abstract

The application discloses a distributed task processing method, device and system, electronic equipment and a storage medium, to solve the problem that the existing configuration message pushing mode of network equipment cannot guarantee the accurate sequence of message issuing to the network equipment, thereby causing network equipment configuration failure. The method comprises the following steps: a network orchestrator obtains a first RPC task from an RPC task table, wherein the RPC task is created and submitted to the RPC task table by the network orchestrator according to a service request sent by a client when the network orchestrator receives the service request; and if it is determined to execute the first RPC task based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, the first PRC task is sent to a configuration proxy server corresponding to a target network equipment for processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and particularly relates to a distributed task processing method and device, a system, an electronic device and a storage medium. BACKGROUND

[0002] In the process of data center network orchestration, a large number of network devices need to be issued with configuration messages. In order to improve the response speed of network orchestration, in the related technology, a distributed message queue is usually used for data caching and pushing. After generating a configuration message, a message producer pushes the configuration message to a message queue in an asynchronous remote procedure call (RPC) mode. The message queue pushes the configuration message to a network device corresponding to a message consumer according to the principle of first-in first-out.

[0003] However, in the process of pushing configuration messages to a large number of network devices in an asynchronous RPC mode, if multiple message queues simultaneously push messages to the same network device, the problem of concurrent messages will occur, and the accurate order of issuing messages to the network device cannot be guaranteed, thereby causing the network device configuration to fail. SUMMARY

[0004] In order to solve the problem that the existing method of pushing configuration messages to a network device cannot guarantee the accurate order of issuing messages to the network device, thereby causing the network device configuration to fail, the embodiments of the present application provide a distributed task processing method, device, system, electronic device and storage medium.

[0005] In a first aspect, the embodiments of the present application provide a distributed task processing method implemented on a network orchestrator side, comprising:

[0006] The network orchestrator obtains a first RPC task from a remote procedure call (RPC) task table. The RPC task is created and submitted to the RPC task table according to a service request received by the network orchestrator from a client.

[0007] If it is determined to execute the first RPC task based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, the first PRC task is sent to a configuration proxy server bound to a corresponding target network device for processing.

[0008] In a possible implementation, the first PRC task is sent to a configuration proxy server bound to a corresponding target network device for processing, specifically comprising:

[0009] The configuration proxy server corresponding to the target network device corresponding to the first RPC task is determined based on a preset correspondence relationship between a network device identifier and the configuration proxy server.

[0010] push the first RPC task to a message queue corresponding to the configuration proxy server; the message queue is used for the configuration proxy server bound by the target network device to acquire the RPC task therein for processing.

[0011] In a possible implementation, the method further includes:

[0012] If it is determined that the first RPC task execution completion response sent by the configuration proxy server is received, the first RPC task is deleted from the RPC task table.

[0013] In a possible implementation, the network orchestrator is any network orchestrator in a network orchestrator cluster.

[0014] Before the network orchestrator acquires the first RPC task from the remote procedure call (RPC) task table, the method further includes:

[0015] For each network device, the network orchestrator queries whether a record message of binding a master configuration proxy server for the network device exists in a database;

[0016] If it is determined that the record message of binding the master configuration proxy server for the network device does not exist in the database, a master configuration proxy server is bound for the network device, a record message of configuring the master configuration proxy server for the network device is generated, and the record message is stored in the database.

[0017] If it is determined that the record message of binding the master configuration proxy server for the network device exists in the database, a slave configuration proxy server is bound for the network device.

[0018] In a possible implementation, the configuration proxy server acquiring the RPC task from the message queue is the master configuration proxy server bound by the target network device, and the master configuration proxy server bound by the target network device and each slave configuration proxy server respectively listens to a message queue corresponding to the target network device; and

[0019] The method further includes:

[0020] If it is determined that the master configuration proxy server bound by the target network device fails, any slave configuration proxy server bound by the target network device and in an online state is switched to the master configuration proxy server corresponding to the target network device; and

[0021] After the switching is completed, the unprocessed RPC tasks in the message queue corresponding to the failed master configuration proxy server are migrated to the message queue corresponding to the switched master configuration proxy server.

[0022] In a possible implementation, the determining that the master configuration proxy server of the target network device binding is down specifically includes:

[0023] The master configuration proxy server is determined to be down if a heartbeat message sent by the master configuration proxy server of the target network device binding is not received within a set time period, wherein each configuration proxy server sends a heartbeat message to the network orchestrator at a preset time period.

[0024] In a possible implementation, if the master configuration proxy server and the corresponding slave configuration proxy server of each network device binding are different configuration proxy servers of the configuration proxy server cluster, the method further includes:

[0025] If the number of configuration proxy servers serving as master configuration proxy servers is greater than a first preset proportion, the master configuration proxy server and the slave configuration proxy server of a second preset proportion of network device bindings on the configuration proxy server are switched.

[0026] In a possible implementation, the determining to execute the first RPC task based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task list specifically includes:

[0027] If it is determined that there is no other RPC task in the RPC task list that is created earlier than the first RPC task, the first RPC task is determined to be executed.

[0028] If it is determined that there is an other RPC task in the RPC task list that is created earlier than the first RPC task, a next RPC task is obtained from the RPC task list, and the first RPC task is executed after the other RPC task that is created earlier than the first RPC task is executed.

[0029] In a possible implementation, if it is determined that there is no other RPC task in the RPC task list that is created earlier than the first RPC task, the first RPC task is determined to be executed, specifically including:

[0030] If it is determined that the execution identifier of the first RPC task is obtained, the first RPC task is determined to be executed, the execution identifier is added for the first RPC task after a previous RPC task that is created earlier than the first RPC task is executed, and the execution identifier is used to identify that the previous RPC task of the first RPC task has been executed.

[0031] In a possible implementation, the network orchestrator creates an RPC task according to the service request in the following manner:

[0032] The service plug-in in the network orchestrator creates an RPC task according to the service request; and performs the operation of creating the RPC task by the service plug-in and the operation of the service plug-in itself on the database in the same database transaction.

[0033] In a second aspect, the embodiments of the present application provide a distributed task processing apparatus implemented on a network orchestrator side, comprising:

[0034] The obtaining unit is configured to obtain a first RPC task from a remote procedure call (RPC) task table, the RPC task being created and submitted into the RPC task table by the network orchestrator according to a service request sent by a client when the network orchestrator receives the service request.

[0035] The sending unit is configured to send the first RPC task to a configuration proxy server corresponding to a target network device for processing, if it is determined to execute the first RPC task based on a dependency relationship between the first RPC task and other RPC tasks in the RPC task table.

[0036] In a possible implementation, the sending unit is specifically configured to determine a configuration proxy server corresponding to a target network device corresponding to the first RPC task based on a preset correspondence between network device identifiers and configuration proxy servers; and push the first RPC task to a message queue corresponding to the configuration proxy server, the message queue being configured to allow the configuration proxy server corresponding to the target network device to obtain an RPC task therefrom for processing.

[0037] In a possible implementation, the apparatus further comprises:

[0038] The deleting unit is configured to delete the first RPC task from the RPC task table, if it is determined that a first RPC task execution completion response sent by the configuration proxy server is received.

[0039] In a possible implementation, the network orchestrator is any network orchestrator in a network orchestrator cluster.

[0040] The apparatus further comprises:

[0041] The querying unit is configured to, before the network orchestrator obtains a first RPC task from a remote procedure call (RPC) task table, query, for each network device, whether there is a record message of binding a master configuration proxy server for the network device in a database.

[0042] a first binding unit configured to bind a primary configuration agent server to the network device and generate a record message of the network device configured with the primary configuration agent server if it is determined that the record message of the network device bound with the primary configuration agent server does not exist in the database, and store the record message in the database;

[0043] a second binding unit configured to bind a secondary configuration agent server to the network device if it is determined that the record message of the network device bound with the primary configuration agent server exists in the database.

[0044] In a possible implementation, the configuration agent server of the RPC task in the message queue is the primary configuration agent server bound to the target network device, and the primary configuration agent server and each secondary configuration agent server bound to the target network device respectively listen to a corresponding message queue of the target network device.

[0045] The apparatus further includes:

[0046] a switching unit configured to switch any secondary configuration agent server bound to the target network device and in an online state to the corresponding primary configuration agent server of the target network device if it is determined that the primary configuration agent server bound to the target network device fails.

[0047] a migration unit configured to migrate, after the switching is completed, the unprocessed RPC tasks in the message queue corresponding to the failed primary configuration agent server to the message queue corresponding to the switched primary configuration agent server.

[0048] In a possible implementation, the switching unit is specifically configured to determine that the primary configuration agent server fails if a heartbeat message sent by the primary configuration agent server bound to the target network device is not received within a set time length, where each configuration agent server sends a heartbeat message to the network orchestrator according to a preset time period.

[0049] In a possible implementation, if the primary configuration agent server and the corresponding secondary configuration agent server bound to each network device are different configuration agent servers of the configuration agent server cluster, the switching unit is further configured to switch the primary configuration agent server and the secondary configuration agent server bound to a second preset proportion of network devices on the configuration agent server if the number of the same configuration agent server as the primary configuration agent server exceeds a first preset proportion.

[0050] In a possible implementation, the sending unit is specifically configured to: if it is determined that there is no other RPC task with a creation time earlier than that of the first RPC task in the RPC task list, determine to execute the first RPC task; if it is determined that there is other RPC task with a creation time earlier than that of the first RPC task in the RPC task list, obtain a next RPC task from the RPC task list; and after the other RPC task with the creation time earlier than that of the first RPC task is executed, execute the first RPC task.

[0051] In a possible implementation, the sending unit is specifically configured to: if it is determined that the execution identifier of the first RPC task is obtained, determine to execute the first RPC task, the execution identifier being added for the first RPC task after a previous RPC task with a creation time earlier than that of the first RPC task is executed, and the execution identifier being used to identify that the previous RPC task of the first RPC task has been executed.

[0052] In a possible implementation, the apparatus further includes:

[0053] The creating unit is configured to create an RPC task according to the service request by: creating the RPC task according to the service request by using a service plug-in; and executing, in a same database transaction, an operation of creating the RPC task by the service plug-in and an operation of the service plug-in itself on a database.

[0054] In a third aspect, an embodiment of the present application provides a distributed task processing method implemented on a configuration proxy server side, including:

[0055] A configuration proxy server bound to a network device obtains a first remote procedure call (RPC) task, the first RPC task being sent to the configuration proxy server bound to the network device when the configuration proxy server determines to execute the first RPC task based on a dependency relationship between the first RPC task and other RPC tasks in an RPC task list, the RPC task being created and submitted into the RPC task list by a network orchestrator according to a service request received from a client, and the network device being a target network device corresponding to the first PRC task.

[0056] The first RPC task is processed and then delivered to the network device for configuration.

[0057] In a possible implementation, the configuration proxy server bound to the network device obtains the first RPC task, specifically including:

[0058] The configuration proxy server of the network device binding obtains the first RPC task in sequence from its corresponding message queue, which is pushed to the message queue by the network orchestrator after determining to execute the first RPC task.

[0059] In a possible implementation, the configuration proxy server is a master configuration proxy server of the network device binding.

[0060] In a possible implementation, the method further includes:

[0061] receiving a configuration success response message returned by the network device to delete the first RPC task from the RPC task table.

[0062] In a possible implementation, receiving the configuration success response message returned by the network device to delete the first RPC task from the RPC task table specifically includes:

[0063] if it is determined that the configuration success response message returned by the network device is received, deleting the first RPC task from the RPC task table; or

[0064] if it is determined that the configuration success response message returned by the network device is received, sending a first RPC task execution completion response to the network orchestrator to make the network orchestrator delete the first RPC task from the RPC task table.

[0065] In a fourth aspect, an embodiment of the present application provides a distributed task processing apparatus implemented on a configuration proxy server side, including:

[0066] a obtaining unit, configured to obtain a first remote procedure call (RPC) task, the first RPC task being sent to a configuration proxy server of a network device binding after a network orchestrator determines to execute the first RPC task based on a dependency relationship between the first RPC task and other RPC tasks in an RPC task table, the RPC task being created and submitted to the RPC task table by the network orchestrator according to a service request sent by a client when the network orchestrator receives the service request, the network device being a target network device corresponding to the first PRC task;

[0067] a processing unit, configured to process the first RPC task and then deliver the first RPC task to the network device for configuration.

[0068] In a possible implementation, the obtaining unit is specifically configured to sequentially obtain the first RPC task from a message queue corresponding to a configuration proxy server bound to the network device, the first RPC task being pushed to the message queue by the network orchestrator after determining to execute the first RPC task.

[0069] In a possible implementation, the configuration proxy server is a primary configuration proxy server bound to the network device.

[0070] In a possible implementation, the apparatus further includes:

[0071] The deleting unit is configured to receive a configuration success response message returned by the network device to delete the first RPC task from the RPC task table.

[0072] In a possible implementation, the deleting unit is specifically configured to delete the first RPC task from the RPC task table if it is determined that the configuration success response message returned by the network device is received, or send a first RPC task execution completion response to the network orchestrator to make the network orchestrator delete the first RPC task from the RPC task table if it is determined that the configuration success response message returned by the network device is received.

[0073] In a fifth aspect, an embodiment of the present application provides a distributed task processing system, including:

[0074] The network orchestrator is configured to obtain a first RPC task from an RPC task table, the RPC task being created and submitted to the RPC task table by the network orchestrator according to a service request sent by a client when the network orchestrator receives the service request; and send the first PRC task to a configuration proxy server bound to a target network device for processing if it is determined to execute the first RPC task based on a dependency relationship between the first RPC task and other RPC tasks in the RPC task table.

[0075] The configuration proxy server bound to the target network device is configured to process the first RPC task and then issue the first RPC task to the network device for configuration.

[0076] In a sixth aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor implementing the distributed task processing method provided by the present application when executing the program.

[0077] In a seventh aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of the distributed task processing method.

[0078] The beneficial effects of the present application are as follows:

[0079] The task processing scheme provided by the embodiments of the present application, the network orchestrator obtains the first RPC task from the RPC task table, wherein the RPC tasks in the RPC task table are created and submitted to the RPC task table by the network orchestrator according to the service request when the network orchestrator receives the service request sent by the client, and the network orchestrator determines to execute the first RPC task based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, and sends the first RPC task to the corresponding configuration proxy server bound to the target network device for processing. In the present application, the processing of the RPC task is performed by the cooperation between the RPC task table and the configuration proxy server bound to each network device, the RPC task is executed according to the dependency relationship between the RPC task and other RPC tasks, and the corresponding configuration proxy server is bound to different network devices. In this way, not only the message timing sequence of the same network device can be ensured, but also the purpose of concurrent processing of messages between different network devices is achieved, an asynchronous, ordered and high-concurrent communication mode is realized, and the message sending efficiency and stability are improved.

[0080] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structure particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0081] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the specification and illustrate the illustrative embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0082] Figure 1 The architecture diagram of the distributed task processing system provided by the embodiments of the present application is shown in the figure;

[0083] Figure 2 The implementation flowchart of the distributed task processing method provided by the embodiments of the present application is shown in the figure;

[0084] Figure 3 The implementation flowchart of the network orchestrator for binding the configuration proxy server to the network device provided by the embodiments of the present application is shown in the figure;

[0085] Figure 4An implementation flow diagram of a network orchestrator provided by an embodiment of the present application sending a first RPC task to a corresponding target network device bound configuration agent server for processing is shown in FIG. 1.

[0086] Figure 5 An example diagram of an RPC task processing process in a service abnormal situation provided by an embodiment of the present application is shown in FIG. 2.

[0087] Figure 6 An implementation flow diagram of a distributed task processing method implemented on the network orchestrator side provided by an embodiment of the present application is shown in FIG. 3.

[0088] Figure 7 A structure diagram of a distributed task processing device implemented on the network orchestrator side provided by an embodiment of the present application is shown in FIG. 4.

[0089] Figure 8 An implementation flow diagram of a distributed task processing method implemented on the configuration agent server side provided by an embodiment of the present application is shown in FIG. 5.

[0090] Figure 9 A structure diagram of a distributed task processing device implemented on the configuration agent server side provided by an embodiment of the present application is shown in FIG. 6.

[0091] Figure 10 A structure diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0092] To solve the problem that the existing configuration message pushing manner for network devices cannot guarantee the accurate sequence of message issuing to network devices, thereby causing network device configuration failure, an embodiment of the present application provides a distributed task processing method, device, electronic device and storage medium.

[0093] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings of the specification, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0094] As Figure 1As shown, it is an application scenario diagram of the distributed task processing method provided by the embodiment of the application, that is, the architecture diagram of the distributed task processing system provided by the embodiment of the application can include a network orchestrator cluster and a configuration agent server cluster, the network orchestrator cluster includes at least two network orchestrators, and the configuration agent server cluster includes at least two configuration agent servers, in the application, three network orchestrators and three configuration agent servers are taken as an example for description, the network orchestrator binds a respective corresponding configuration agent server for each network device, wherein the configuration agent server is used to run a configuration agent process, the network orchestrator that is bound for each network device, that is, the respective corresponding configuration agent process, the distributed task processing system can further include a message queue cluster, each configuration agent server of each network device respectively listens to a message queue, wherein:

[0095] The network orchestrator is configured to obtain a first RPC task from the remote procedure call (RPC) task table, the RPC task being created and submitted to the RPC task table by the network orchestrator according to a service request received from a client; and if it is determined to execute the first RPC task based on a dependency relationship between the first RPC task and other RPC tasks in the RPC task table, the first PRC task is sent to a configuration agent server bound to a corresponding target network device for processing.

[0096] The network orchestrator is specifically configured to determine a configuration agent server corresponding to a target network device corresponding to the first RPC task based on a preset correspondence between a network device identifier and the configuration agent server; and push the first RPC task to a message queue corresponding to the configuration agent server; the message queue is used for the configuration agent server bound to the target network device to obtain an RPC task therein for processing.

[0097] The configuration agent server bound to the target network device is configured to process the first RPC task and then issue the first RPC task to the network device for configuration.

[0098] In a specific implementation, the configuration agent server bound to the target network device sends the RPC task obtained from the target message queue to a target network device driver, and the target network device driver converts the RPC task into a configuration message and then issues the configuration message to the target network device. Different network device drivers can be arranged on the respective bound configuration agent servers.

[0099] It should be noted that the network orchestrator in the application can be but is not limited to Neutron, and the embodiment of the application does not limit this, and the application only takes Neutron as an example for description.

[0100] The distributed task processing system provided in the application binds a corresponding configuration proxy server for a network device, performs processing of RPC tasks in cooperation between the RPC task table and the bound configuration proxy server of each network device to issue a configuration message to the network device, each configuration proxy server bound to the network device has a corresponding message queue to cache a corresponding RPC task, each network device obtains an RPC task from the corresponding message queue by the bound configuration proxy server, so that the configuration proxy servers bound to each network device do not interfere with each other when performing the corresponding RPC tasks, and the RPC tasks are executed according to the dependency relationship between the RPC tasks and other RPC tasks, so that the message timing sequence between different message queue clusters of the same network device can be ensured, and the purpose of concurrent processing of messages between different network devices is achieved, an asynchronous, ordered and high-concurrency communication mode based on a distributed message queue is realized, and the message sending efficiency and stability are improved.

[0101] Based on the above application scenarios, the following will refer to the accompanying Figures 2 to 9 The exemplary embodiments of the application will be described in more detail below, and it should be noted that the above application scenarios are only shown for the purpose of facilitating understanding of the spirit and principles of the application, and the embodiments of the application are not limited herein. On the contrary, the embodiments of the application can be applied to any applicable scenario.

[0102] As Figure 2 shown, it is an implementation flow diagram of the distributed task processing method provided by the embodiments of the application, the task processing method can be applied to the distributed task processing system described above, and specifically can include the following steps:

[0103] S11, the network orchestrator obtains a first RPC task from the RPC task table.

[0104] Among them, the RPC task is created and submitted to the RPC task table by the network orchestrator according to the business request when the network orchestrator receives the business request sent by the client.

[0105] In implementation, an RPC task table is set in advance for storing and managing the RPC tasks submitted by the network orchestrators. The RPC task table can be stored in a database. In implementation, when any network orchestrator in the network orchestrator cluster receives a service request sent by a client, the network orchestrator creates an RPC task according to the service request by using a service plugin. That is, the service plugin converts the service request into an RPC message based on the RPC protocol. One RPC message is one RPC task. The RPC task is inserted into the RPC task table in the order of generation time. The service request can be a configuration message sent by the client to the corresponding network device. The service request also includes the target network device identifier to which the client needs to send the configuration message. After the RPC task is created, the correspondence between the RPC task and the target network device identifier is submitted to the RPC task table for storage. The network device identifier can be but is not limited to the IP address of the network device. The network device can be but is not limited to a network security device such as a firewall device. The embodiments of the present application do not limit this.

[0106] In implementation, the network orchestrator is any network orchestrator in the network orchestrator cluster. The RPC task thread corresponding to any network orchestrator can obtain the RPC task from the RPC task table in the order of time. The RPC task is recorded as a first RPC task.

[0107] Before this step, the network orchestrator in the network orchestrator cluster also binds a corresponding configuration agent server for each network device. That is, the network orchestrator in the network orchestrator cluster binds a corresponding configuration agent process (rg-agent) for each network device, and stores the correspondence between the network device identifier and the configuration agent server in the database.

[0108] As a preferred implementation, in order to avoid the data transmission interruption of the network device caused by the failure of the configuration agent server, each network orchestrator in the network orchestrator cluster can also bind a corresponding configuration agent server for each network device. One of the configuration agent servers bound by the network orchestrators for the same network device is the master configuration agent server corresponding to the network device. The other configuration agent servers except the master configuration agent server are the slave configuration agent servers corresponding to the network device. That is, the configuration agent servers bound by different network orchestrators for the same network device form a master-slave relationship. The master configuration agent server and the slave configuration agent server of the same network device can be set as different configuration agent servers of the configuration agent server cluster. The correspondence between the network device identifier and the configuration agent server includes the correspondence between the network device identifier and the master configuration agent server, and the correspondence between the network device identifier and the slave configuration agent server.

[0109] In implementation, the network orchestrator can bind the configuration proxy server for each network device according to the flow as shown in Figure 3 The flow for binding the configuration proxy server for each network device includes the following steps:

[0110] S21, for each network device, the network orchestrator queries whether there is a record message of binding the master configuration proxy server for the network device in the database.

[0111] In implementation, for each network device, any network orchestrator queries whether there is a record message of binding the master configuration proxy server for the network device in the database.

[0112] S22, if it is determined that there is no record message of binding the master configuration proxy server for the network device in the database, the master configuration proxy server is bound for the network device, and a record message of binding the master configuration proxy server for the network device is generated, and the record message is stored in the database.

[0113] In implementation, if there is no record message of binding the master configuration proxy server for the network device in the database, it indicates that the network device has not bound the master configuration proxy server, and the current network orchestrator can bind the master configuration proxy server for the network device, generate a record message of binding the master configuration proxy server for the network device, and record the correspondence between the network device identifier and the master configuration proxy server (such as the identifier), and store the record message and the correspondence between the network device identifier and the master configuration proxy server in the database. In this way, when other network orchestrators bind the configuration proxy server for the network device, it can be determined through querying the database that the network device has bound the master configuration proxy server, and the slave configuration proxy server is bound for the network device.

[0114] S23, if it is determined that there is a record message of binding the master configuration proxy server for the network device in the database, the slave configuration proxy server is bound for the network device.

[0115] In implementation, if there is a record message of binding the master configuration proxy server for the network device in the database, it indicates that other network orchestrators have bound the master configuration proxy server for the network device, and the current network orchestrator binds the slave configuration proxy server for the network device, and records the correspondence between the network device identifier and the slave configuration proxy server, and stores the correspondence between the network device identifier and the slave configuration proxy server in the database.

[0116] S12, the network orchestrator determines to execute the first RPC task based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, and sends the first PRC task to the configuration proxy server bound by the corresponding target network device for processing.

[0117] In implementation, when the network orchestrator acquires a first RPC task from the RPC task list by using the RPC task thread, if the following case one occurs, it is determined to execute the first RPC task acquired currently, if the following case two occurs, the first RPC task acquired currently is not executed, and the next task in the RPC task list is acquired:

[0118] Case one: if it is determined that there is no other RPC task in the RPC task list whose creation time is earlier than that of the first RPC task, the first RPC task is determined to be executed.

[0119] In implementation, an RPC task belonging to the same network device has a dependency relationship with a previous RPC task whose creation time is earlier than that of the RPC task, that is, the RPC tasks belonging to the same network device are executed in series according to the order of creation time, and the RPC tasks belonging to different network devices are executed in parallel, and the dependency relationship exists between the RPC tasks belonging to the same network device.

[0120] If the network orchestrator determines to acquire the execution identifier of the first RPC task, the first RPC task is determined to be executed, wherein the execution identifier is added to the first RPC task after the previous RPC task whose creation time is earlier than that of the first RPC task is executed, and the execution identifier is used to identify that the previous RPC task of the first RPC task has been executed. The first RPC task and the previous RPC task belong to the same target network device.

[0121] Specifically, the execution identifier can be implemented by using an optimistic lock, and the optimistic lock is added to any RPC task after the previous RPC task whose creation time is earlier than that of the RPC task is executed. When the network orchestrator acquires the first RPC task by using the RPC task thread, the optimistic lock of the current first RPC task can be acquired, and if the optimistic lock of the current RPC task is acquired, it is determined to execute the current first RPC task.

[0122] Case two: if it is determined that there is another RPC task in the RPC task list whose creation time is earlier than that of the first RPC task, the next RPC task in the RPC task list is acquired, and the first RPC task is executed after the other RPC task whose creation time is earlier than that of the first RPC task is executed.

[0123] In implementation, if the network orchestrator determines not to acquire the execution identifier (i.e., the optimistic lock) of the first RPC task, the execution identifier of the next RPC task in the RPC task list is continuously acquired, and the first RPC task is determined to be executed until the execution identifier of the first RPC task is re-acquired. The first RPC task and the next RPC task belong to the same target network device.

[0124] As another possible implementation, the dependency relationship of one RPC task to other RPC tasks can also indicate a fixed execution order of a group of RPC tasks, that is, the execution of the group of RPC tasks has a logical relationship, for example, RPC task 1, RPC task 2 and RPC task 3 are three business requests issued by the same client, corresponding to the same target network device, and need to be executed in the order of RPC task 1, RPC task 2 and RPC task 3. If the currently acquired is RPC task 1, RPC task 1 is executed, and after the execution of RPC task 1 is completed, the network orchestrator or the master configuration proxy server bound to the target network device deletes RPC task 1 from the RPC task table. If RPC task 1 is being executed and has not been executed, that is, RPC task 1 is still stored in the RPC task table, and the network orchestrator acquires RPC task 2 through the RPC task thread, the network orchestrator does not execute the RPC task 2, and the network orchestrator acquires RPC task 3 through the RPC task thread from the RPC task table. At this time, RPC task 1 and RPC task 2 exist in the RPC task table, and RPC task 1 and RPC task 2 have not been executed. The network orchestrator acquires the next RPC task through the RPC task thread, judges whether there is other RPC task having a dependency relationship with the acquired RPC task, and executes the acquired RPC task if there is no other RPC task having a dependency relationship with the acquired RPC task. After the execution of RPC task 1 is completed, RPC task 2 is executed, and after the execution of RPC task 2 is completed, RPC task 3 is executed. In this way, the timing sequence of the RPC tasks having a dependency relationship can be ensured not to be disturbed.

[0125] When the network orchestrator determines to execute the first RPC task, the first RPC task can be sent to the configuration proxy server bound to the corresponding target network device for processing according to the flow as shown in Figure 4 , including the following steps:

[0126] S31, determining the configuration proxy server corresponding to the target network device corresponding to the first RPC task based on the preset correspondence relationship between the network device identifier and the configuration proxy server.

[0127] In specific implementation, the network orchestrator determines the target network device identifier corresponding to the first RPC task based on the correspondence relationship between the RPC task and the target network device identifier recorded in the RPC task table, and then determines the master configuration proxy server corresponding to the target network device identifier based on the correspondence relationship between the network device identifier and the master configuration proxy server stored in the database, that is, the master configuration proxy server bound to the target network device corresponding to the first RPC task.

[0128] S32, push the first RPC task to a message queue corresponding to the configuration proxy server, and the message queue is used for the target network device to bind the configuration proxy server to acquire the RPC task therein for processing.

[0129] In implementation, each network device binds a master configuration proxy server and each slave configuration proxy server respectively listens to a message queue corresponding to the network device. That is, a network device and a master configuration proxy server and a message queue listened to by the master configuration proxy server have a corresponding relationship, and a network device and a slave configuration proxy server and a message queue listened to by the slave configuration proxy server have a corresponding relationship. The master configuration proxy server and the slave configuration proxy server cannot listen to the same message queue. If the master configuration proxy server and the slave configuration proxy server listen to the same message queue, the master configuration proxy server and the slave configuration proxy server will acquire the RPC task from the message queue. In this way, the consumption order of the RPC message in the message queue cannot be guaranteed. When the master configuration proxy server and the slave configuration proxy server listen to the respective message queues, only the message queue corresponding to the master configuration proxy server has the RPC task, and the message queue corresponding to the slave configuration proxy server is empty. In this way, the slave configuration proxy server cannot acquire the RPC task, and the time sequence of the RPC message can be guaranteed.

[0130] The network orchestrator pushes the first RPC task to the message queue corresponding to the master configuration proxy server bound by the target network device through the RPC task thread, and the message queue corresponding to the master configuration proxy server bound by the target network device is used for the master configuration proxy server bound by the target network device to acquire the RPC task therein for processing. That is, the configuration proxy server acquiring the RPC task from the message queue is the master configuration proxy server bound by the target network device.

[0131] Further, the master configuration proxy server bound by the target network device acquires the RPC task from the message queue corresponding to the target network device and listened to by the master configuration proxy server. The master configuration proxy server executes the RPC task, and the slave configuration proxy server does not execute the RPC task.

[0132] In implementation, the configuration proxy server bound by the target network device acquires the RPC task (i.e., the RPC message) from the corresponding message queue in the order of first in first out, or the message queue pushes the RPC task (i.e., the RPC message) to the corresponding target network device according to the principle of first in first out. The embodiment of the present application does not limit this.

[0133] In an embodiment, the configuration proxy servers (including the master configuration proxy server and the slave configuration proxy servers) bound to each network device send heartbeat messages to the network orchestrator at a preset time period, and the network orchestrator determines whether the configuration proxy servers bound to the network device are online according to whether the heartbeat messages exceed a set time length, wherein the preset time period can be set as required, for example, 1 second, and the set time length can also be set as required, for example, 3 seconds, and the embodiment of the present application does not limit this.

[0134] In a specific implementation, if it is determined that the master configuration proxy server bound to the target network device fails, any slave configuration proxy server bound to the target network device and in an online state is switched to the master configuration proxy server corresponding to the target network device, and after the switching is completed, the unprocessed RPC tasks in the message queue corresponding to the failed master configuration proxy server are migrated to the message queue corresponding to the switched master configuration proxy server.

[0135] Specifically, if the network orchestrator does not receive the heartbeat message sent by the master configuration proxy server bound to the target network device within a set time length, it is determined that the master configuration proxy server bound to the target network device fails. At this time, the network orchestrator can switch any slave configuration proxy server bound to the target network device and in an online state to the master configuration proxy server corresponding to the target network device, and after the switching is completed, the unprocessed RPC messages in the message queue corresponding to the failed master configuration proxy server are migrated to the message queue corresponding to the switched master configuration proxy server, and the switched master configuration proxy server continues to obtain the migrated RPC messages from the message queue it listens to. In this way, the RPC messages in the message queue corresponding to the failed master configuration proxy server can be migrated to the switched master configuration proxy server for listening, so that the data during the failure switching can be guaranteed not to be lost.

[0136] The load balancing of the configuration proxy server cluster mainly aims at the scheduling problem after the failure recovery of the configuration proxy server. For example, the configuration proxy server cluster has three configuration proxy servers, two of which have failed. All the master configuration proxy servers are switched to the configuration proxy server that has not failed. In this way, the RPC messages corresponding to all the network devices will be scheduled to the same configuration proxy server. However, when the two failed configuration proxy servers recover, the RPC messages will not be automatically load balanced to the two recovered configuration proxy servers, and there is a problem of load balancing.

[0137] Based on this, the network orchestrator can determine whether the primary configuration agent server is centralized to a certain configuration agent server based on the primary configuration agent server identifier (such as the host name of the primary configuration agent server) bound by each network device. In an embodiment, if the primary configuration agent server bound by each network device and the corresponding secondary configuration agent server are different configuration agent servers of the configuration agent server cluster, the network orchestrator further includes:

[0138] If the number of the same configuration agent server as the primary configuration agent server exceeds the first preset proportion, the primary configuration agent server and the secondary configuration agent server bound by the second preset proportion of network devices on the configuration agent server are switched. That is, if the number of the primary configuration agent program set on the same configuration agent server exceeds the first preset proportion, the primary configuration agent program and the secondary configuration agent program bound by the second preset proportion of network devices are switched.

[0139] In a specific implementation, if the network orchestration server monitors that the number of the same configuration agent server as the primary configuration agent server of the network device is greater than or equal to the first preset proportion of the total number of all primary configuration agent servers, the network orchestrator switches the primary configuration agent server bound by the second preset proportion of network devices on the configuration agent server to a secondary configuration agent server, and switches any secondary configuration agent server corresponding to each original primary configuration agent server to a primary configuration agent server, thereby ensuring the load balancing of the server cluster where the configuration agent server is located. The first preset proportion and the second preset proportion can be set by the user, and the embodiments of the present application do not limit the same.

[0140] For example, the configuration agent server cluster includes three servers, and there are 10 network devices, each of which is bound to 1 primary configuration agent server and 2 secondary configuration agent servers. The primary configuration agent server bound by 7 network devices is the first server, and the secondary configuration agent server bound by each network device is the second server and the third server, respectively. The primary configuration agent server bound by 1 network device is the second server, and the secondary configuration agent server bound by the network device is the first server and the third server. The primary configuration agent server bound by 2 network devices is the third server, and the secondary configuration agent server bound by each network device is the first server and the second server, respectively. The first preset proportion is assumed to be 50%. Therefore, the proportion of the number of the first server as the primary configuration agent server to the total number of configuration agent servers is 70%, which exceeds 50%. Therefore, the primary configuration agent server bound by 2 network devices can be switched from the first server to the second server, and the primary configuration agent server bound by 1 network device can be switched from the first server to the third server, thereby ensuring the load balancing of the three servers as much as possible.

[0141] S13, the configuration proxy server processes the obtained first RPC task and then sends it to the target network device for configuration.

[0142] In a specific implementation, the main configuration proxy server bound to the target network device sequentially obtains the first RPC task (i.e., the first RPC message) from the message queue corresponding to the target network device, sends the obtained first RPC message to the target network device driver, and then sends the first RPC message to the target network device after the target network device driver converts the first RPC message into a configuration message recognizable by the target network device. The target network device is configured according to the configuration message.

[0143] S14, if the target network device is successfully configured, the target network device returns a configuration success response message to the configuration proxy server, and if the target network device fails to be configured, the target network device returns a configuration failure message to the configuration proxy server.

[0144] In a specific implementation, if the target network device is successfully configured, the target network device returns a configuration success response message to the main configuration proxy server bound to the target network device, and if the target network device fails to be configured, the target network device returns a configuration failure message to the main configuration proxy server bound to the target network device.

[0145] S15, if the configuration proxy server receives the configuration success response message returned by the target network device, the configuration proxy server deletes the first RPC task from the RPC task table, or performs step S16. In a specific implementation, if the main configuration proxy server bound to the target network device receives the configuration success response message returned by the target network device, the main configuration proxy server can directly delete the first RPC task from the RPC task table, or perform step S16.

[0146] S16, if the configuration proxy server receives the configuration success response message returned by the target network device, the configuration proxy server sends a first RPC task execution completion response to the network orchestrator.

[0147] In a specific implementation, if the main configuration proxy server bound to the target network device receives the configuration success response message returned by the target network device, the main configuration proxy server can also send a first task execution completion response to the network orchestrator, and the network orchestrator deletes the first RPC task from the RPC task table.

[0148] S17, if the network orchestrator determines that the first RPC task execution completion response sent by the configuration proxy server is received, the network orchestrator deletes the first RPC task from the RPC task table.

[0149] In a complex business scenario, a single business request involves multiple business function modules, and multiple read and write database and network device configuration messages are required. In general, the network orchestrator pushes the obtained RPC task (i.e., RPC message) to the message queue corresponding to the target network device bound main configuration agent server of the RPC task through the RPC task thread. The target network device bound main configuration agent server obtains the RPC message from the corresponding message queue. However, the business function module may occur abnormal in the process of processing the business request, at this time, the business retry is triggered, and the dirty data (i.e., data that may cause conflict in the network device) generated before the abnormality occurs needs to be rolled back in the business retry process. If the RPC message has been pushed to the main configuration agent server, it cannot be rolled back, thereby causing configuration residual problems on the target network device.

[0150] Based on this, in an embodiment, the network orchestrator can create an RPC task according to a business request in the following manner:

[0151] The business plug-in in the network orchestrator creates an RPC task according to a business request, and executes the operation of creating an RPC task by the business plug-in and the operation of the business plug-in to the database in the same database transaction.

[0152] In specific implementation, when creating an RPC task, the business plug-in in the network orchestrator puts the operation of creating an RPC task by the business plug-in and the operation of the business plug-in to the database into the same database transaction, and uses the atomicity and abnormal rollback mechanism of the database transaction to achieve the purpose of automatically rolling back the RPC task. In this way, when the business process is abnormal, the business request and the RPC task are rolled back, and when the business process is normally executed, the database transaction is committed to the database, at this time, the corresponding RPC task exists in the RPC task table, and the network orchestrator can obtain the RPC task from the RPC task table through the RPC thread, thereby realizing the RPC message rollback in the abnormal case of a complex business chain and ensuring that there is no residual configuration on the target network device.

[0153] In an embodiment, the network orchestrator can create an RPC task according to a business request in the following manner: Figure 5RPC task table, RPC task n is rolled back, RPC task n is created unsuccessfully, the service request corresponding to the RPC task n is rolled back to the corresponding client device, and other tasks RPC task 1~(n-1) in the RPC task table are executed normally. When the RPC task thread obtains an RPC task each time, if it is determined, based on the order of creation time of the current RPC task and other RPC tasks in the RPC task table and the IP address of the target network device of the current RPC task, that the current RPC task has a dependency relationship with other RPC tasks whose creation time is earlier than that of the current RPC task, the current RPC task is executed after the other RPC tasks are executed. According to the IP address of the target network device of the RPC task, a message queue corresponding to a master configuration proxy server to which the target network device is bound is matched, and the RPC task is cached in the message queue. The master configuration proxy server to which the target network device is bound obtains the RPC task (that is, an RPC message) from the message queue, sends the RPC message to a target network device driver, converts the RPC message into a configuration message by the target network device driver, and then delivers the configuration message to the target network device. The target network device is configured according to the configuration message, and the corresponding RPC task is deleted from the RPC task table after the RPC task is executed.

[0154] In the implementation process, if the network device is expanded on a large scale, the existing message queue cluster may not be able to carry a large number of RPC messages of the network device, and the message queue cluster needs to be expanded horizontally. The above-mentioned distributed task processing method provided by the embodiment of the present application can support the scenario of horizontally expanding the message queue cluster, and can ensure the accuracy of the timing sequence of the configuration delivery of the same target network device.

[0155] The task processing method provided in the embodiments of the present application, the network orchestrator obtains a first RPC task from the RPC task table through an RPC task thread, wherein the RPC tasks in the RPC task table are created and submitted to the RPC task table according to service requests received by the network orchestrator from a client, and if it is determined to execute the first RPC task based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, the configuration proxy server corresponding to the target network device corresponding to the first RPC task is determined based on the preset correspondence between the network identifier and the configuration proxy server, the first RPC task is pushed to the message queue corresponding to the configuration proxy server, and the configuration proxy server bound to the target network device obtains the RPC tasks from the corresponding message queue in sequence and then issues the configuration messages to the target network device for configuration. In the present application, the network orchestrator binds the corresponding configuration proxy server to the network device, and the RPC tasks are processed through the cooperation between the RPC task table and the configuration proxy server bound to each network device to issue configuration messages to the network device. Each configuration proxy server bound to each network device has a corresponding message queue to cache the RPC tasks corresponding to each network device, and each network device obtains the RPC tasks from the corresponding message queue through the configuration proxy server bound thereto, so that the configuration proxy servers bound to each network device do not interfere with each other when executing the corresponding RPC tasks, and the RPC tasks are executed according to the dependency relationship between the RPC tasks and other RPC tasks. In this way, the message timing sequence between different message queue clusters of the same network device can be ensured, and the purpose of concurrent processing of messages between different network devices is achieved, realizing an asynchronous, ordered and high-concurrency communication mode based on a distributed message queue, and improving the message sending efficiency and stability.

[0156] Based on the same inventive concept, the embodiments of the present application also provide a network orchestrator-side implemented distributed task processing method. Since the principle of solving problems of the above-mentioned network orchestrator-side implemented distributed task processing method is similar to that of the above-mentioned distributed task processing method, the implementation of the above-mentioned network orchestrator-side implemented distributed task processing method can be referred to the implementation of the above-mentioned method, and the repeated parts will not be described herein.

[0157] As shown in Figure 6 The implementation flowchart of the network orchestrator-side implemented distributed task processing method provided in the embodiments of the present application is shown in the figure, which is applied to a distributed task processing system, the distributed task processing system includes a network orchestrator cluster, each network orchestrator in the network orchestrator cluster binds a corresponding configuration proxy server to each network device, and the method includes:

[0158] S41, the network orchestrator obtains a first RPC task from an RPC task table, the RPC task being created and submitted to the RPC task table by the network orchestrator according to a service request received from a client.

[0159] S42, if it is determined to execute the first RPC task based on a dependency relationship between the first RPC task and other RPC tasks in the RPC task table, the first PRC task is sent to a configuration proxy server corresponding to a target network device for processing.

[0160] In a possible implementation, the first PRC task is sent to the configuration proxy server corresponding to the target network device for processing, specifically including:

[0161] determining the configuration proxy server corresponding to the target network device corresponding to the first RPC task based on a preset correspondence between network device identifiers and configuration proxy servers;

[0162] pushing the first RPC task to a message queue corresponding to the configuration proxy server; the message queue is used for the configuration proxy server corresponding to the target network device to obtain the RPC task therein for processing.

[0163] In a possible implementation, the method further includes:

[0164] if it is determined that the first RPC task execution completion response sent by the configuration proxy server is received, the first RPC task is deleted from the RPC task table.

[0165] In a possible implementation, the network orchestrator is any network orchestrator in a network orchestrator cluster.

[0166] Before the network orchestrator obtains the first RPC task from the remote procedure call (RPC) task table, the method further includes:

[0167] for each network device, the network orchestrator queries whether there is a record message of binding a master configuration proxy server for the network device in a database;

[0168] if it is determined that there is no record message of binding a master proxy server for the network device in the database, a master configuration proxy server is bound for the network device, a record message of configuring the master configuration proxy server for the network device is generated, and the record message is stored in the database.

[0169] if it is determined that there is a record message of binding a master configuration proxy server for the network device in the database, a slave configuration proxy server is bound for the network device.

[0170] In a possible implementation, the configuration proxy server obtaining the RPC task in the message queue is a master configuration proxy server bound to the target network device, and the master configuration proxy server and each slave configuration proxy server listen to a corresponding message queue of the target network device respectively; and

[0171] The method further includes:

[0172] If it is determined that the master configuration proxy server bound to the target network device fails, any slave configuration proxy server in an online state bound to the target network device is switched to the master configuration proxy server corresponding to the target network device; and

[0173] After the switching is completed, the unprocessed RPC tasks in the message queue corresponding to the failed master configuration proxy server are migrated to the message queue corresponding to the switched master configuration proxy server.

[0174] In a possible implementation, the determination that the master configuration proxy server bound to the target network device fails specifically includes:

[0175] If the heartbeat message sent by the master configuration proxy server bound to the target network device is not received within a set time length, it is determined that the master configuration proxy server fails, where each configuration proxy server sends a heartbeat message to the network orchestrator according to a preset time period.

[0176] In a possible implementation, if the master configuration proxy server and the corresponding slave configuration proxy server bound to each network device are different configuration proxy servers in the configuration proxy server cluster, the method further includes:

[0177] If the number of the same configuration proxy server as the master configuration proxy server exceeds a first preset proportion, the master configuration proxy server and the slave configuration proxy server bound to a second preset proportion of network devices on the configuration proxy server are switched.

[0178] In a possible implementation, the determination to execute the first RPC task based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task list specifically includes:

[0179] If it is determined that there is no other RPC task in the RPC task list with a creation time earlier than the creation time of the first RPC task, it is determined to execute the first RPC task.

[0180] If it is determined that there are other RPC tasks in the RPC task list whose creation time is earlier than that of the first RPC task, then the next RPC task is obtained from the RPC task list; and after the other RPC tasks whose creation time is earlier than that of the first RPC task are executed, the first RPC task is executed again.

[0181] In one possible implementation, if it is determined that there are no other RPC tasks in the RPC task list whose creation time is earlier than that of the first RPC task, then it is determined to execute the first RPC task, specifically including:

[0182] If the execution identifier of the first RPC task is obtained, then the first RPC task is executed. The execution identifier is added to the first RPC task after the previous RPC task whose creation time is earlier than the first RPC task has been completed. The execution identifier is used to indicate that the previous RPC task of the first RPC task has been completed.

[0183] In one possible implementation, the network orchestrator creates an RPC task based on the service request in the following manner:

[0184] The business plugin in the network orchestrator creates an RPC task based on the business request; and executes the operation of the business plugin creating the RPC task and the operation of the business plugin itself on the database within the same database transaction.

[0185] Based on the same inventive concept, this application also provides a distributed task processing device implemented on the network orchestrator side. Since the principle of solving the problem by the distributed task processing device implemented on the network orchestrator side is similar to that of the distributed task processing method, the implementation of the distributed task processing device implemented on the network orchestrator side can refer to the implementation of the method, and the repeated parts will not be described again.

[0186] like Figure 7 The diagram shown is a structural schematic of a distributed task processing device implemented on the network orchestrator side according to an embodiment of this application. It is applied to a distributed task processing system, which includes a network orchestrator cluster. Each network orchestrator in the cluster binds its corresponding configuration proxy server to each network device. The device includes:

[0187] The acquisition unit 51 is used to acquire a first RPC task from the Remote Procedure Call (RPC) task table. The RPC task is created and submitted to the RPC task table by the network orchestrator when it receives a service request sent by the client.

[0188] The sending unit 52 is configured to send the first RPC task to a configuration proxy server corresponding to a target network device for processing if it is determined to execute the first RPC task based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table.

[0189] In a possible implementation, the sending unit 52 is specifically configured to determine the configuration proxy server corresponding to the target network device corresponding to the first RPC task based on a preset correspondence between a network device identifier and a configuration proxy server; and push the first RPC task to a message queue corresponding to the configuration proxy server; and the message queue is used for the configuration proxy server corresponding to the target network device to acquire an RPC task in the message queue for processing.

[0190] In a possible implementation, the apparatus further includes:

[0191] The deleting unit is configured to delete the first RPC task from the RPC task table if it is determined that a first RPC task execution completion response sent by the configuration proxy server is received.

[0192] In a possible implementation, the network orchestrator is any network orchestrator in a network orchestrator cluster.

[0193] The apparatus further includes:

[0194] The querying unit is configured to, before the network orchestrator acquires a first RPC task from a remote procedure call (RPC) task table, query, for each network device, whether a record message of binding a master configuration proxy server for the network device exists in a database.

[0195] The first binding unit is configured to, if it is determined that the record message of binding the master configuration proxy server for the network device does not exist in the database, bind a master configuration proxy server for the network device, generate a record message of configuring the master configuration proxy server for the network device, and store the record message in the database.

[0196] The second binding unit is configured to, if it is determined that the record message of binding the master configuration proxy server for the network device exists in the database, bind a slave configuration proxy server for the network device.

[0197] In a possible implementation, the configuration proxy server acquiring the RPC task from the message queue is a master configuration proxy server bound by the target network device, and the master configuration proxy server bound by the target network device and each slave configuration proxy server respectively listen to a message queue corresponding to the target network device.

[0198] The device further comprises:

[0199] The switching unit is configured to switch any slave configuration proxy server in an online state bound to the target network device to a master configuration proxy server corresponding to the target network device if it is determined that the master configuration proxy server bound to the target network device fails.

[0200] The migration unit is configured to migrate, after the switching is completed, an unprocessed RPC task in a message queue corresponding to the failed master configuration proxy server to a message queue corresponding to the switched master configuration proxy server.

[0201] In a possible implementation, the switching unit is specifically configured to determine that the master configuration proxy server fails if a heartbeat message sent by the master configuration proxy server bound to the target network device is not received within a set time length, where each configuration proxy server sends a heartbeat message to the network orchestrator according to a preset time period.

[0202] In a possible implementation, if the master configuration proxy server and the corresponding slave configuration proxy server bound to each network device are different configuration proxy servers of the configuration proxy server cluster, the switching unit is further configured to switch the master configuration proxy server and the slave configuration proxy server bound to a second preset proportion of network devices on the configuration proxy server if a number of configuration proxy servers serving as master configuration proxy servers is greater than a first preset proportion.

[0203] In a possible implementation, the sending unit 52 is specifically configured to determine to execute the first RPC task if it is determined that there is no other RPC task with a creation time earlier than that of the first RPC task in the RPC task list, determine to obtain a next RPC task from the RPC task list if it is determined that there is another RPC task with a creation time earlier than that of the first RPC task in the RPC task list, and execute the first RPC task after the other RPC task with the creation time earlier than that of the first RPC task is executed.

[0204] In a possible implementation, the sending unit 52 is specifically configured to determine to execute the first RPC task if it is determined that an execution identifier of the first RPC task is obtained, the execution identifier being added for the first RPC task after a previous RPC task with a creation time earlier than that of the first RPC task is executed, and the execution identifier being used to identify that the previous RPC task of the first RPC task has been executed.

[0205] The device further comprises:

[0206] The creating unit is configured to create an RPC task according to the service request by: creating the RPC task according to the service request by using a service plug-in; and executing the operation of creating the RPC task by the service plug-in and the operation of the service plug-in itself to the database in the same database transaction.

[0207] Based on the same inventive concept, the embodiment of the present application also provides a distributed task processing method implemented on a configuration proxy server side. Since the principle of solving problems of the distributed task processing method implemented on the configuration proxy server side is similar to that of the distributed task processing method, the implementation of the distributed task processing method implemented on the configuration proxy server side can refer to the implementation of the method, and the repeated parts will not be described here.

[0208] As shown in Figure 8 FIG. 1 is a flowchart of an implementation process of a distributed task processing method implemented on a configuration proxy server side provided by an embodiment of the present application, which is applied to a distributed task processing system including a network orchestrator cluster, each network orchestrator in the network orchestrator cluster binds a respective corresponding configuration proxy server for each network device, and the method includes the following steps.

[0209] S61, the configuration proxy server bound to the network device acquires a first RPC task.

[0210] The first RPC task is determined based on a dependency relationship between the first RPC task and other RPC tasks in an RPC task table when the network orchestrator acquires the first RPC task from the RPC task table, and the first RPC task is sent to the configuration proxy server bound to the network device when the first RPC task is executed, the RPC task is created according to a service request received by the network orchestrator from a client and submitted to the RPC task table, and the network device is a target network device corresponding to the first PRC task.

[0211] S62, the first RPC task is processed and then issued to the network device for configuration.

[0212] In a possible implementation, the configuration proxy server bound to the network device acquires the first RPC task, specifically including:

[0213] The configuration proxy server bound to the network device acquires the first RPC task from a message queue corresponding to the configuration proxy server in order, and the first RPC task is pushed to the message queue by the network orchestrator after determining to execute the first RPC task.

[0214] In a possible implementation, the configuration proxy server is a master configuration proxy server bound to the network device.

[0215] In one possible implementation, the method further includes:

[0216] Receive a configuration success response message returned by the network device to delete the first RPC task from the RPC task table.

[0217] In one possible implementation, receiving a configuration success response message returned by the network device to delete the first RPC task from the RPC task table specifically includes:

[0218] If it is determined that a configuration success response message has been received from the network device, then the first RPC task is deleted from the RPC task table; or,

[0219] If a configuration success response message is received from the network device, a first RPC task completion response is sent to the network orchestrator so that the network orchestrator deletes the first RPC task from the RPC task table.

[0220] Based on the same inventive concept, this application also provides a distributed task processing device implemented on the configuration proxy server side. Since the principle of solving the problem by the distributed task processing device implemented on the configuration proxy server side is similar to that of the distributed task processing method, the implementation of the distributed task processing device implemented on the configuration proxy server side can refer to the implementation of the method, and the repeated parts will not be described again.

[0221] like Figure 9 The diagram shown is a structural schematic of a distributed task processing device implemented on the configuration proxy server side according to an embodiment of this application. It is applied to a distributed task processing system, which includes a network orchestrator cluster. Each network orchestrator in the cluster binds its corresponding configuration proxy server to each network device. The device includes:

[0222] The acquisition unit 71 is used to acquire a first remote procedure call (RPC) task. The first RPC task is acquired by the network orchestrator from the RPC task table and, based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, is sent to the configuration proxy server bound to the network device when executing the first RPC task. The RPC task is created and submitted to the RPC task table by the network orchestrator when it receives a service request sent by the client. The network device is the target network device corresponding to the first RPC task.

[0223] The processing unit 72 is used to process the first RPC task and then send it to the network device for configuration.

[0224] In a possible implementation, the obtaining unit 71 is specifically configured to sequentially obtain the first RPC task from a message queue corresponding to a configuration proxy server bound to the network device, the first RPC task being pushed to the message queue by the network orchestrator after determining to execute the first RPC task.

[0225] In a possible implementation, the configuration proxy server is a primary configuration proxy server bound to the network device.

[0226] In a possible implementation, the apparatus further includes:

[0227] The deleting unit is configured to receive a configuration success response message returned by the network device to delete the first RPC task from the RPC task table.

[0228] In a possible implementation, the deleting unit is specifically configured to delete the first RPC task from the RPC task table if it is determined that the configuration success response message returned by the network device is received, or send a first RPC task execution completion response to the network orchestrator to make the network orchestrator delete the first RPC task from the RPC task table if it is determined that the configuration success response message returned by the network device is received.

[0229] Based on the same technical concept, the embodiment of the present application also provides an electronic device 800, which is described with reference to Figure 10 The electronic device 800 is used to implement the task processing method described in the above method embodiment. The electronic device 800 of the embodiment can include a memory 801, a processor 802, and a computer program stored in the memory and executable on the processor, for example, a task processing program. When the processor executes the computer program, the steps in each of the above task processing method embodiments are implemented.

[0230] The specific connection medium between the memory 801 and the processor 802 is not limited in the embodiment of the present application. In the embodiment of the present application, the memory 801 and the processor 802 are connected through a bus 803, which is represented by a thick line in Figure 10 The connection mode between other components is only schematically described and is not limited. The bus 803 can be divided into an address bus, a data bus, and a control bus. For convenience, only one thick line is used to represent the bus in Figure 10 However, it does not mean that there is only one bus or only one type of bus. Figure 10

[0231] ​The memory 801 can be a volatile memory (volatile memory), such as a random access memory (RAM); the memory 801 can also be a non-volatile memory (non-volatile memory), such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 801 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited to this. The memory 801 can be a combination of the above-mentioned memories.

[0232] The processor 802 is configured to implement the distributed task processing method of various exemplary embodiments of the present application.

[0233] The embodiments of the present application also provide a computer readable storage medium storing computer executable instructions required for the processor to execute.

[0234] In some possible embodiments, various aspects of the distributed task processing method provided by the present application can also be implemented in the form of a program product, which includes program codes for causing an electronic device to perform the steps of the task processing method described above according to various exemplary embodiments of the present application when the program product is run on the electronic device.

[0235] Those skilled in the art should understand that the embodiments of the present application can be provided in the form of a method, device, or computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0236] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (apparatus) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the computer or other programmable data processing device produce a device that implements the functions described in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0237] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart or multiple flows and / or blocks. Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0238] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart or multiple flows and / or blocks. Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0239] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those of skill in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, the appended claims are intended to encompass within their scope all such variations and modifications as being within the scope of the application.

[0240] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover the modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A distributed task processing method, characterized in that, include: The network orchestrator retrieves the first RPC task from the Remote Procedure Call (RPC) task table. The RPC task is created and submitted to the RPC task table by the network orchestrator based on the service request sent by the client when it receives the service request. If the first RPC task is determined to be executed based on its dependency relationship with other RPC tasks in the RPC task table, the first RPC task is sent to the configuration proxy server bound to the corresponding target network device for processing.

2. The method as described in claim 1, characterized in that, The first RPC task is sent to the configuration proxy server bound to the corresponding target network device for processing, specifically including: The configuration proxy server corresponding to the target network device of the first RPC task is determined based on the preset correspondence between network device identifiers and configuration proxy servers. The first RPC task is pushed to the message queue corresponding to the configuration proxy server; the message queue is used for the configuration proxy server bound to the target network device to retrieve the RPC task therein and process it.

3. The method as described in claim 1 or 2, characterized in that, Also includes: If it is determined that the first RPC task has been completed and a response from the configuration proxy server has been received, the first RPC task is deleted from the RPC task table.

4. The method as described in claim 2, characterized in that, The network orchestrator is any network orchestrator in the network orchestrator cluster; Before the network orchestrator retrieves the first RPC task from the Remote Procedure Call (RPC) task table, the following steps are also included: For each network device, the network orchestrator queries the database to see if there is a record message that binds the network device to the primary configuration proxy server; If it is determined that there is no record message in the database for binding the primary configuration proxy server to the network device, then the primary configuration proxy server is bound to the network device, and a record message for configuring the primary configuration proxy server for the network device is generated and stored in the database; If it is determined that there is a record message in the database that binds the network device to a primary configuration proxy server, then a secondary configuration proxy server is bound to the network device.

5. The method as described in claim 4, characterized in that, The configuration proxy server that retrieves the RPC tasks from the message queue is the primary configuration proxy server bound to the target network device. The primary configuration proxy server and each secondary configuration proxy server bound to the target network device each listen to a message queue corresponding to the target network device. as well as The method further includes: If it is determined that the primary configuration proxy server bound to the target network device has failed, then any online secondary configuration proxy server bound to the target network device will be switched to the primary configuration proxy server corresponding to the target network device. and After the switchover is complete, any unprocessed RPC tasks in the message queue corresponding to the faulty primary configuration proxy server will be migrated to the message queue corresponding to the new primary configuration proxy server.

6. The method as described in claim 5, characterized in that, Determining that the primary configuration proxy server bound to the target network device has failed includes: If no heartbeat message is received from the main configuration proxy server bound to the target network device within a set time period, it is determined that the main configuration proxy server has failed. Each configuration proxy server sends a heartbeat message to the network orchestrator according to a preset time period.

7. The method as described in claim 5, characterized in that, If the primary configuration proxy server and the corresponding secondary configuration proxy server bound to each network device are different configuration proxy servers in the configuration proxy server cluster, then the method further includes: If the number of times the same configuration proxy server acts as the primary configuration proxy server exceeds the first preset ratio, then the primary configuration proxy server and the secondary configuration proxy server bound to the network devices on the configuration proxy server at the second preset ratio will be switched.

8. The method as described in claim 1, characterized in that, The execution of the first RPC task is determined based on its dependencies with other RPC tasks in the RPC task table, specifically including: If it is determined that there are no other RPC tasks in the RPC task list whose creation time is earlier than that of the first RPC task, then it is determined to execute the first RPC task; If it is determined that there are other RPC tasks in the RPC task list whose creation time is earlier than that of the first RPC task, then the next RPC task is obtained from the RPC task list; and after the other RPC tasks whose creation time is earlier than that of the first RPC task are executed, the first RPC task is executed again.

9. The method as described in claim 8, characterized in that, If it is determined that there are no other RPC tasks in the RPC task list whose creation time is earlier than that of the first RPC task, then the first RPC task is executed, specifically including: If the execution identifier of the first RPC task is obtained, then the first RPC task is executed. The execution identifier is added to the first RPC task after the previous RPC task whose creation time is earlier than the first RPC task has been completed. The execution identifier is used to indicate that the previous RPC task of the first RPC task has been completed.

10. The method as described in claim 1, characterized in that, The network orchestrator creates an RPC task based on the service request in the following manner: The business plugin in the network orchestrator creates an RPC task based on the business request; and executes the operation of the business plugin creating the RPC task and the operation of the business plugin itself on the database within the same database transaction.

11. A distributed task processing method, characterized in that, include: The configuration proxy server bound to the network device obtains a first remote procedure call (RPC) task. The first RPC task is obtained by the network orchestrator from the RPC task table. Based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, the network orchestrator determines when to send the first RPC task to the configuration proxy server bound to the network device. The RPC task is created and submitted to the RPC task table by the network orchestrator when it receives a service request sent by the client. The network device is the target network device corresponding to the first RPC task. After processing the first RPC task, it is sent to the network device for configuration.

12. The method as described in claim 11, characterized in that, The network device's bound configuration proxy server obtains the first RPC task, which specifically includes: The configuration proxy server bound to the network device retrieves the first RPC task sequentially from its corresponding message queue. The first RPC task is pushed to the message queue by the network orchestrator after determining that the first RPC task will be executed.

13. The method as described in claim 12, characterized in that, The configuration proxy server is the primary configuration proxy server bound to the network device.

14. The method according to any one of claims 11 to 13, characterized in that, Also includes: Receive a configuration success response message returned by the network device to delete the first RPC task from the RPC task table.

15. The method as described in claim 14, characterized in that, Receiving a configuration success response message returned by the network device to delete the first RPC task from the RPC task table specifically includes: If it is determined that a configuration success response message has been received from the network device, then the first RPC task is deleted from the RPC task table; or, If a configuration success response message is received from the network device, a first RPC task completion response is sent to the network orchestrator so that the network orchestrator deletes the first RPC task from the RPC task table.

16. A distributed task processing device, characterized in that, include: The acquisition unit is used to acquire a first RPC task from the Remote Procedure Call (RPC) task table. The RPC task is created and submitted to the RPC task table by the network orchestrator when it receives a service request sent by the client. The sending unit is configured to send the first RPC task to the configuration proxy server bound to the corresponding target network device for processing if the first RPC task is determined to be executed based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table.

17. A distributed task processing device, characterized in that, include: The acquisition unit is used to acquire a first remote procedure call (RPC) task. The first RPC task is acquired by the network orchestrator from the RPC task table. Based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, the configuration proxy server bound to the network device is determined to be sent when executing the first RPC task. The RPC task is created and submitted to the RPC task table by the network orchestrator according to the service request sent by the client when it receives the service request. The network device is the target network device corresponding to the first RPC task. The processing unit is used to process the first RPC task and then send it to the network device for configuration.

18. A distributed task processing system, characterized in that, include: A network orchestrator is used to retrieve a first RPC task from a Remote Procedure Call (RPC) task table. The RPC task is created and submitted to the RPC task table by the network orchestrator based on a service request sent by a client. If the execution of the first RPC task is determined based on the dependency relationship between the first RPC task and other RPC tasks in the RPC task table, the first RPC task is sent to the configuration proxy server bound to the corresponding target network device for processing. The configuration proxy server bound to the target network device is used to process the first RPC task and then send it to the network device for configuration.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the task processing method as described in any one of claims 1 to 15.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the task processing method as described in any one of claims 1 to 15.

Images