A task processing method, apparatus, electronic device, and storage medium

Abstract

The present application discloses a task processing method, apparatus, electronic device and storage medium, relating to the technical field of cloud computing. The specific implementation solution is as follows: determining a plurality of to-be-processed subtasks corresponding to the to-be-processed task, and obtaining the queries per second of the service providing server corresponding to the external service required when executing the to-be-processed subtasks; allocating the to-be-processed subtasks to each computing node server based on the queries per second; wherein the total number of the to-be-processed subtasks allocated to each computing node server does not exceed the queries per second; sending task allocation messages to each computing node server respectively according to the to-be-processed subtasks allocated to each computing node server, so that each computing node server executes the corresponding to-be-processed subtasks by accessing the service providing server according to the received task allocation messages. The technical solution of the present application reduces the number of task processing failures and the time cost invested in repeated task allocation, thereby improving the task processing efficiency.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular to the technical field of cloud computing. Background Art

[0002] A cloud service system is a system platform built using cloud computing technologies and concepts, used to provide services such as data computing and data storage.

[0003] When using a cloud service system to provide services, the cloud service system will receive tasks to be processed and allocate the tasks to the corresponding service-providing servers. Since the service-providing servers have limited task-processing capabilities at the same time, task-processing failures often occur.

[0004] For example, the Intelligent Logistics System (ILS) is an intelligent logistics cloud service system based on the Baidu Map Engine and geographic big data, with a high-performance self-developed scheduling algorithm at its core, providing solutions for dispatching and routing in various transportation scenarios.

[0005] When the server in the intelligent scheduling system receives the destination network point input by the user, it will perform path planning based on the pre-established road network matrix and output the planned free delivery path. However, when performing the task of building the road network matrix, there is usually a situation where the task of calling the road network matrix building server fails, and the task needs to be reissued and processed, reducing the task-processing efficiency and bringing a poor user experience to users.

[0006] Application Content

[0007] This application provides a task processing method, device, electronic device, and storage medium to reduce task processing time and improve task processing efficiency.

[0008] In a first aspect, an embodiment of this application provides a task processing method, applied to a master node server, including:

[0009] Determine multiple subtasks to be processed corresponding to the task to be processed, and obtain the queries per second rate of the service-providing servers corresponding to the external services required when executing the subtasks to be processed;

[0010] Allocate the subtasks to be processed to each computing node server corresponding to the master node server based on the queries per second rate; where the total number of subtasks to be processed allocated to each computing node server does not exceed the queries per second rate;

[0011] Send task assignment messages to the corresponding computing node servers respectively according to the sub-tasks to be processed assigned to each computing node server, so that each computing node server executes the corresponding sub-tasks to be processed by accessing the service providing server according to the received task assignment messages.

[0012] One embodiment of the above application has the following beneficial effects: It can reduce the task processing time and improve the task processing efficiency. By obtaining the queries per second rate of the service providing server and performing task assignment for multiple sub-tasks to be processed corresponding to the task to be processed based on the queries per second rate, the service providing server can effectively respond to the access operations corresponding to the computing node servers when executing the sub-tasks to be processed, which not only ensures the processing efficiency of the sub-tasks to be processed but also takes into account the processing capacity of the service providing server. Therefore, it overcomes the technical problem that the service providing server fails to process tasks in time, resulting in task processing failures, thereby reducing the number of task processing failures, reducing the time cost invested in repeated task assignment, and further improving the task processing efficiency.

[0013] Optionally, the task to be processed is a road network task to be processed, and the road network task to be processed contains multiple network points;

[0014] Correspondingly, determining multiple sub-tasks to be processed corresponding to the task to be processed includes:

[0015] Determine pairs of network points that need to obtain road network information according to the multiple network points, and use each determined pair of network points as a sub-task to be processed respectively;

[0016] Correspondingly, the external service that needs to be requested includes a service for providing road network information between the pairs of network points.

[0017] The above optional method can allocate multiple pairs of network points corresponding to the road network task to be processed and send task assignment messages based on the queries per second rate of the service providing server, so that the service providing server can provide road network information between the pairs of network points according to the received task assignment information, thereby reducing the time consumed when providing the road network information service and improving the service providing efficiency.

[0018] Optionally, after sending task assignment messages to the corresponding computing node servers respectively according to the sub-tasks to be processed assigned to each computing node server, it further includes:

[0019] If there are unassigned sub-tasks to be processed, return to execute the sub-task to be processed assignment operation until all sub-tasks to be processed are assigned.

[0020] The above optional method can continue to trigger the sub-task to be processed assignment operation when the sub-tasks to be processed are not fully assigned, ensuring the integrity of the assignment of the task to be processed.

[0021] Optionally, after all the to-be-processed subtasks are assigned, it further includes:

[0022] Query the database for storing the execution results of the subtasks to determine whether each to-be-processed subtask is successfully executed, and store the determination result;

[0023] Query the determination result, and when there are to-be-processed subtasks that are not successfully executed in the determination result, return to perform the to-be-processed subtask assignment operation until each to-be-processed subtask in the determination result is successfully executed or the task assignment stop condition is met.

[0024] The above optional method determines by querying the execution results of each to-be-processed subtask during the execution of the to-be-processed subtasks, so as to re-perform the assignment operation of the to-be-processed subtask when there are to-be-processed subtasks that are not successfully executed, ensuring the effectiveness of the processing process of each to-be-processed subtask in the to-be-processed task.

[0025] Optionally, based on the query rate per second, assign to-be-processed subtasks to each computing node server corresponding to the node server, including:

[0026] Determine the number of the computing node servers, and determine the subtask assignment quantity corresponding to each computing node server according to the ratio of the query rate per second to the determined quantity;

[0027] Select the to-be-processed subtasks with the subtask assignment quantity from the multiple to-be-processed subtasks for each computing node server respectively for task assignment.

[0028] The above optional method determines the subtask assignment quantity corresponding to each computing node server by calculating the number of the computing node servers, and then selects and assigns the to-be-processed subtasks for each computing node server according to the subtask assignment quantity, realizing the task balancing of each computing node server corresponding to the master node server, avoiding the impact caused by the downtime of a certain computing node server due to the unbalanced tasks of the computing node servers, and thus indirectly improving the task processing efficiency.

[0029] Optionally, select the to-be-processed subtasks with the subtask assignment quantity from the multiple to-be-processed subtasks for each computing node server respectively, including:

[0030] If the number of the to-be-processed tasks is multiple, determine the to-be-processed tasks corresponding to selecting the to-be-processed subtasks for each computing node server according to the polling mechanism or the task priority.

[0031] The above optional method selects the tasks to be processed through a polling mechanism or task priority, improving the distribution mechanism of the subtasks to be processed when there are multiple tasks to be processed, and ensuring the orderly distribution of the tasks to be processed.

[0032] Optionally, task assignment messages are sent to the corresponding computing node servers respectively according to the subtasks to be processed assigned to each computing node server, including:

[0033] According to the subtasks to be processed assigned to each computing node server, task assignment messages for each computing node server are sent to the central node in the message queue MQ cluster, so that each computing node server reads its own task assignment message from the ActiveMQ in the central node; or,

[0034] According to the subtasks to be processed assigned to each computing node server, task assignment messages are sent to the corresponding computing node servers respectively by using the zeroMQ mechanism.

[0035] The above optional method sends task assignment messages by setting up a message queue, improving the system performance through asynchronous processing, peak shaving, and reducing the coupling of the system composed of the master node server and the computing node servers.

[0036] Optionally, task assignment messages are sent to the corresponding computing node servers respectively according to the subtasks to be processed assigned to each computing node server, including:

[0037] According to the subtasks to be processed assigned to each computing node server, task assignment messages are sent to the corresponding computing node servers respectively at a set rate.

[0038] The above optional method sets the sending rate when sending task assignment messages to the master node server, avoiding the occurrence of message blocking caused by too fast sending of task assignment messages, thereby improving the task assignment efficiency.

[0039] In a second aspect, an embodiment of the present application further provides a task processing method, which is applied to a computing node server and includes:

[0040] Obtain the task assignment message sent by the master node server according to the subtasks to be processed assigned to the computing node server; wherein, the subtasks to be processed come from the same task to be processed, and the sum of the number of the subtasks to be processed and the number of the subtasks to be processed assigned by the master node server to other computing node servers does not exceed the queries per second of the service provider server corresponding to the external service required when executing the subtasks to be processed;

[0041] Access the service providing server according to the task assignment message to execute the to-be-processed subtask corresponding to the task assignment message.

[0042] Optionally, the to-be-processed task is a to-be-processed road network task, which includes a plurality of network points, and each to-be-processed subtask is a pair of network points that needs to obtain road network information.

[0043] Optionally, before accessing the service providing server according to the task assignment message, it further includes:

[0044] If the number of to-be-processed subtasks corresponding to the task assignment message received per second is greater than the query rate per second, discard at least one to-be-processed subtask according to a preset rule.

[0045] Optionally, obtaining the task assignment message sent by the master node server according to the to-be-processed subtasks assigned to the computing node server includes:

[0046] Read the task assignment message corresponding to the computing node server from ActiveMQ in the central node of the message queue MQ cluster; wherein the task assignment message is sent by the master node server according to the to-be-processed subtasks assigned to the computing node server; or,

[0047] Receive the task assignment message sent by the master node server according to the to-be-processed subtasks assigned to the computing node server and using the zeroMQ mechanism.

[0048] In a third aspect, an embodiment of the present application further provides a task processing device, configured in the master node server, including:

[0049] A query rate acquisition module, configured to determine a plurality of to-be-processed subtasks corresponding to a to-be-processed task, and obtain the query rate per second of the service providing server corresponding to the external service required when executing the to-be-processed subtask;

[0050] A subtask assignment module, configured to assign to-be-processed subtasks to each computing node server corresponding to the master node server based on the query rate per second; wherein, the total number of to-be-processed subtasks assigned to each computing node server does not exceed the query rate per second;

[0051] An assignment message sending module, configured to send task assignment messages to the corresponding computing node servers respectively according to the to-be-processed subtasks assigned to each computing node server, so that each computing node server executes the corresponding to-be-processed subtask by accessing the service providing server according to the received task assignment message.

[0052] Fourthly, an embodiment of the present application further provides a task processing device configured in a computing node server, including:

[0053] An allocation message acquisition module, configured to acquire a task allocation message sent by the master node server according to the to-be-processed subtasks allocated to the computing node server; wherein, the to-be-processed subtasks come from the same to-be-processed task, and the sum of the number of the to-be-processed subtasks and the number of the to-be-processed subtasks allocated by the master node server to other computing node servers does not exceed the queries per second of the service provider server corresponding to the external service required when executing the to-be-processed subtasks.

[0054] A subtask processing module, configured to access the service provider server according to the task allocation message to execute the to-be-processed subtasks corresponding to the task allocation message.

[0055] Fifthly, an embodiment of the present application further provides an electronic device, including:

[0056] At least one processor; and

[0057] A memory communicatively connected to the at least one processor; wherein,

[0058] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute a task processing method provided in the embodiment of the first aspect.

[0059] Sixthly, an embodiment of the present application further provides an electronic device, including:

[0060] At least one processor; and

[0061] A memory communicatively connected to the at least one processor; wherein,

[0062] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute a task processing method provided in the embodiment of the second aspect.

[0063] Seventhly, an embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause a computer to execute a task processing method provided in the embodiment of the first aspect.

[0064] Eighthly, an embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause a computer to execute a task processing method provided in the embodiment of the second aspect.

[0065] Other effects of the above optional methods will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The drawings are used to better understand the present solution and do not limit the present application. Among them:

[0067] Figure 1A is a schematic structural diagram of a task processing system provided by an embodiment of the present application;

[0068] Figure 1B is a flowchart of a task processing method in Embodiment 1 of the present application;

[0069] Figure 2 is a flowchart of a task processing method in Embodiment 2 of the present application;

[0070] Figure 3 is a flowchart of a task processing method in Embodiment 3 of the present application;

[0071] Figure 4A is a flowchart of a task processing method in Embodiment 4 of the present application;

[0072] Figure 4B is a schematic structural diagram of a task processing system in Embodiment 4 of the present application;

[0073] Figure 5 is a structural diagram of a task processing device in Embodiment 5 of the present application;

[0074] Figure 6 is a structural diagram of a task processing device in Embodiment 6 of the present application;

[0075] Figure 7 is a block diagram of a server for implementing the task processing method of the embodiment of the present application in Embodiment 7 of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0076] The following describes exemplary embodiments of the present application with reference to the accompanying drawings. Various details of the embodiments of the present application are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted below.

[0077] To clearly introduce the technical solutions of the embodiments of the present application, a system architecture involved in the present application will be described in detail first.

[0078] See Figure 1ASchematic diagram of the structure of a system architecture shown. This system framework adopts the Master-Slave device mode. This system includes a master node server 10, a computing node server 20, and a service providing server 30.

[0079] Among them, the master node server 10 corresponds to the master device in the Master-Slave device, and the computing node server 20 corresponds to the slave device in the Master-Slave device.

[0080] The master node server 10 turns an original task into different subtasks, distributes the subtasks to the computing node server, and integrates the execution results of the subtasks fed back by the computing node server to calculate the final result.

[0081] The computing node server 20 processes the subtasks by accessing the service providing server and feeds back the execution results of each subtask to the master node server.

[0082] The service providing server 30 provides the required services when each computing node server executes the subtasks.

[0083] Embodiment 1

[0084] Figure 1B It is a flowchart of a task processing method in Embodiment 1 of this application. This embodiment of this application is applicable to the situation where the master node server in the system architecture shown receives a task to be processed such as waiting for road network information acquisition, and distributes the task to be processed. This method is executed by a task processing device, which is implemented by software and / or hardware and is specifically configured on the master node server in the server cluster. Figure 1A Shown. A task processing method, applied to the master node server, includes:

[0085] Such as Figure 1B Shown. A task processing method, applied to the master node server, includes:

[0086] S101. Determine a plurality of subtasks to be processed corresponding to the task to be processed, and obtain the queries per second rate of the service providing server corresponding to the external service required when executing the subtasks to be processed.

[0087] Among them, the task to be processed can be pre-stored locally on the master node server and the task to be processed and the determination of its subtasks to be processed are obtained when the task to be processed needs to be processed; it can also be pre-stored in other electronic devices associated with the master node server or in the cloud, such as in a message queue, and the task to be processed and the determination of its subtasks to be processed are obtained when the task to be processed needs to be processed; or it can also be directly communicatively connected to the user's terminal device, and when the user triggers a set area in the terminal device, the task to be processed is sent to the master node server in real time, and the corresponding master node server receives the task to be processed and determines the subtasks to be processed corresponding to the task to be processed.

[0088] Among them, the queries per second are used to measure the traffic processed by the server within a specified time and are associated with the data processing capacity of the server.

[0089] Exemplarily, to obtain the queries per second of the service providing server corresponding to the external service required when executing the to-be-processed subtask, it can be that the master node server directly looks up the queries per second corresponding to the service providing server required to process the to-be-processed subtask from the locally pre-stored server performance list; it can also obtain the queries per second corresponding to the service providing server from each computing node server communicatively connected to the service providing server; it can also directly obtain the queries per second from the service providing server; or an additional database can be set up in the system to store the performance metrics of each server in the system, and obtain the queries per second of the service providing server in the database when needed.

[0090] In an alternative implementation manner of the embodiment of the present application, the to-be-processed task can be a to-be-processed road network task, and the to-be-processed road network task contains multiple network points; correspondingly, determining the multiple to-be-processed subtasks corresponding to the to-be-processed task can be to determine the network point pairs that need to obtain road network information based on the multiple network points, and each determined network point pair is respectively used as a to-be-processed subtask. Correspondingly, the external service that needs to be requested includes the service for providing the road network information between the network point pairs.

[0091] Among them, the to-be-processed road network task can be understood as a road network matrix construction task that needs to be pre-performed during intelligent logistics distribution, so that after the intelligent distribution system receives the distribution destination network point input by the user, it can perform path planning based on the pre-constructed road network matrix and output the planned optimal distribution path. Among them, each element in the road network matrix includes the distribution distance, distribution time, and distribution route between the corresponding distribution start network point and the distribution target network point. Correspondingly, the road network information of the network point pair can be understood as at least one of the distance information, time information, and route information between the two network points of the network point pair.

[0092] S102. Allocate the to-be-processed subtasks to each computing node server corresponding to the master node server based on the queries per second; among them, the total number of to-be-processed subtasks allocated to each computing node server does not exceed the queries per second.

[0093] Optionally, the sub-tasks to be processed can be allocated to each computing node server corresponding to the master node server based on the queries per second rate. This can be done by determining the number of computing node servers and determining the sub-task allocation quantity corresponding to each computing node server according to the ratio of the queries per second rate to the determined quantity. Then, for each computing node server, select the sub-tasks to be processed with the sub-task allocation quantity from the multiple sub-tasks to be processed for task allocation.

[0094] Alternatively, optionally, the sub-tasks to be processed can be allocated to each computing node server corresponding to the master node server based on the queries per second rate. It can also be done by determining the allocable task quantity of each computing node server; determining the sub-task allocation quantity corresponding to each computing node server according to the queries per second rate and the allocable task quantity of each computing node server; and for each computing node server, select the sub-tasks to be processed with the sub-task allocation quantity from the multiple sub-tasks to be processed for task allocation.

[0095] Exemplarily, to determine the allocable task quantity of each computing node server, it can be directly obtaining the allocable task quantity corresponding to each computing node server at the current moment; or obtaining the maximum task allocation quantity and the currently allocated task quantity of each computing node server, and calculating the difference between the maximum task allocation quantity and the currently allocated task quantity to obtain the allocable task quantity corresponding to each computing node server.

[0096] Exemplarily, to determine the sub-task allocation quantity corresponding to each computing node server according to the queries per second rate and the allocable task quantity of each computing node server, it can be determining the total sum of the allocable task quantity of each computing node server; determining the task allocation ratio corresponding to each computing node server according to the ratio of the allocable task quantity corresponding to the computing node server to the total sum of the allocable task quantity; and determining the allocable task quantity corresponding to each computing node server according to the product of the queries per second rate and the task allocation ratio, and using the allocable task quantity as the sub-task allocation quantity.

[0097] It can be understood that by determining the sub-task allocation quantity corresponding to the computing node server and selecting and allocating the sub-tasks to be processed for each computing node server according to the sub-task allocation quantity, the task balance of each computing node server corresponding to the master node server is achieved, avoiding the impact caused by the downtime of a certain computing node server due to the unbalanced tasks of the computing node servers, thereby indirectly improving the task processing efficiency.

[0098] In an alternative implementation manner of the embodiment of the present application, when there are multiple tasks to be processed, in order to ensure the preferred and orderly allocation of each task to be processed, and thus ensure the effective processing of each task to be processed, each computing node server is respectively selected from the multiple sub-tasks to be processed the sub-tasks to be processed with the number of sub-tasks allocated. For task allocation, the task to be processed corresponding to each computing node server when selecting the sub-tasks to be processed can be determined according to the polling mechanism or task priority, and the sub-tasks to be processed with the number of sub-tasks allocated are selected from the sub-tasks to be processed corresponding to the determined task to be processed for task allocation.

[0099] S103. Task allocation messages are respectively sent to the corresponding computing node servers according to the sub-tasks to be processed allocated to each computing node server, so that each computing node server executes the corresponding sub-tasks to be processed by accessing the service providing server according to the received task allocation messages.

[0100] Exemplarily, the task allocation messages can be directly sent to the corresponding computing node servers according to the sub-tasks to be processed allocated to each node server. It can be understood that, in order to improve system performance and smooth the peak, and reduce the coupling between the master node server and each computing node server, a message queue can also be set between the master node server and each computing node server, and the task allocation messages are sent to each computing node server through the message queue.

[0101] Correspondingly, the computing node server receives the task allocation message, and executes the corresponding sub-tasks to be processed by accessing the service providing server according to the received task allocation message.

[0102] In an alternative implementation manner of the embodiment of the present application, sending the task allocation messages to the corresponding computing node servers respectively according to the sub-tasks to be processed allocated to each computing node server may be sending the task allocation messages for each computing node server to the central node in the Message Queue (MQ) cluster according to the sub-tasks to be processed allocated to each computing node server, so that each computing node server reads its own task allocation message from the ActiveMQ in the central node.

[0103] Specifically, the master node server, as the message producer, generates the task allocation messages for each computing node server according to the sub-tasks to be processed allocated to each computing node server, and transmits the task allocation messages to the central node; each computing node server, as the message consumer, reads the task allocation message corresponding to itself from the ActiveMQ in the central node.

[0104] In another alternative embodiment of the embodiment of the present application, sending task allocation messages to corresponding computing node servers respectively according to the to-be-processed subtasks allocated to each computing node server may be to send task allocation messages to corresponding computing node servers respectively by using the zeroMQ mechanism according to the to-be-processed subtasks allocated to each computing node server.

[0105] It can be understood that, in order to avoid the occurrence of message blocking caused by too fast sending of task allocation messages, thereby improving the task allocation efficiency, sending task allocation messages to corresponding computing node servers respectively according to the to-be-processed subtasks allocated to each computing node server may also be to send task allocation messages to corresponding computing node servers respectively at a set rate according to the to-be-processed subtasks allocated to each computing node server. By setting the sending rate of task allocation messages, the number and magnitude of message sending by the master node server are limited, so as to avoid the occurrence of message blocking phenomenon from the source.

[0106] The technical solution of the embodiment of the present application obtains the queries per second rate of the service providing server, and performs task allocation of multiple to-be-processed subtasks corresponding to the to-be-processed tasks based on the queries per second rate, so that the service providing server can effectively respond to the access operations corresponding to the computing node servers when executing the to-be-processed subtasks, which not only ensures the processing efficiency of the to-be-processed subtasks, but also takes into account the processing capacity of the service providing server. Therefore, it overcomes the technical problem that the service providing server fails to process tasks in time, thereby reducing the number of task processing failures, reducing the time cost invested in repeated task allocation, and further improving the task processing efficiency.

[0107] Embodiment 2

[0108] Figure 2 It is a flowchart of a task processing method in Embodiment 2 of the present application. The embodiment of the present application is optimized and improved on the basis of the technical solutions of the above embodiments.

[0109] Further, after the operation of "sending task allocation messages to corresponding computing node servers respectively according to the to-be-processed subtasks allocated to each computing node server", append "if there are to-be-processed subtasks that have not been allocated, then return to execute the to-be-processed subtask allocation operation until all to-be-processed subtasks are allocated", to improve the task processing mechanism.

[0110] Such as Figure 2 shown, a task processing method, applied to a master node server, includes:

[0111] S201. Determine multiple to-be-processed subtasks corresponding to the to-be-processed task, and obtain the queries per second rate of the service providing server corresponding to the external service that needs to be requested when executing the to-be-processed subtask.

[0112] S202. Allocate the to-be-processed subtasks to each computing node server corresponding to the master node server based on the queries per second; wherein, the total number of the to-be-processed subtasks allocated to each computing node server does not exceed the queries per second.

[0113] S203. Send task allocation messages to the corresponding computing node servers respectively according to the to-be-processed subtasks allocated to each computing node server, so that each computing node server executes the corresponding to-be-processed subtasks by accessing the service providing server according to the received task allocation messages.

[0114] S204. If there are unallocated to-be-processed subtasks, return to execute the to-be-processed subtask allocation operation until all to-be-processed subtasks are allocated.

[0115] Since there is at least one to-be-processed task and one to-be-processed task corresponds to multiple to-be-processed subtasks, it is possible that not all to-be-processed subtasks can be allocated in only one task allocation. Therefore, it is necessary to query the unallocated to-be-processed subtasks. When there are still unallocated to-be-processed subtasks, continue the task allocation operation of the to-be-processed subtasks until all to-be-processed subtasks are allocated.

[0116] Optionally, when allocating the to-be-processed subtasks, the to-be-processed subtasks can be marked. When there are still unmarked to-be-processed subtasks in the area where the to-be-processed subtasks are stored (such as the local of the master node server, other storage devices associated with the master node server, or the cloud), it indicates that the to-be-processed subtasks have not been fully allocated.

[0117] It can be understood that, in order to save storage resources, optionally, when allocating the to-be-processed subtasks, the allocated to-be-processed subtasks can also be directly deleted. When there are still to-be-processed subtasks in the area where the to-be-processed subtasks are stored (such as the local of the master node server, other storage devices associated with the master node server, or the cloud), it indicates that the to-be-processed subtasks have not been fully allocated.

[0118] It should be noted that after all the to-be-processed subtasks are allocated, there may still be a situation where due to interference from the computing environment or other factors, the to-be-processed subtasks fail during execution, affecting the execution result of the entire to-be-processed task. To avoid the above situation, in an alternative implementation manner of the embodiments of the present application, a database can be preset in advance to store the execution results of each to-be-processed subtask; correspondingly, the master node server determines whether each to-be-processed subtask is successfully executed by querying the database for storing the execution results of the subtasks. To facilitate the query of the execution results, the master node server can also store the determination results.

[0119] Exemplarily, to facilitate subsequent query of the determination results, data structures such as bitmaps can be used to store the determination results.

[0120] To ensure the smooth execution of the subtasks that fail to execute, during the execution of the to-be-processed task, the master node server queries the determination results in real-time or at regular intervals, and when there are to-be-processed subtasks that are not successfully executed in the determination results, it returns to perform the to-be-processed subtask allocation operation until all the to-be-processed subtasks in the determination results are successfully executed or meet the task allocation stop condition.

[0121] Among them, the task allocation stop condition can be that the to-be-processed subtasks that fail to execute are still not successful after being repeatedly allocated and executed a set number of times, or a task termination request sent by the task initiator is received, etc.

[0122] In the embodiments of the present application, after sending task allocation messages to the corresponding computing node servers according to the to-be-processed subtasks allocated to each computing node server, an operation is added that if there are to-be-processed subtasks that are not allocated, then return to perform the to-be-processed subtask allocation operation until all the to-be-processed subtasks are allocated, enabling the automatic implementation of the next to-be-processed subtask allocation operation until all the to-be-processed subtasks are allocated, ensuring the integrity of the allocation of the to-be-processed tasks and laying a foundation for the effective execution of each to-be-processed task.

[0123] Embodiment III

[0124] Figure 3 is a flowchart of a task processing method in Embodiment III of the present application. The embodiments of the present application are applicable to Figure 1A the situation where the computing node server shown receives a to-be-processed task such as a road network matrix construction and processes the to-be-processed task. This method is executed by a task processing device, which is implemented by software and / or hardware and is specifically configured on the computing node server in the server cluster.

[0125] As Figure 3 shown, a task processing method applied to a computing node server includes:

[0126] S301. Obtain a task allocation message sent by the master node server according to the to-be-processed subtasks allocated to the computing node server.

[0127] Wherein, the to-be-processed subtasks come from the same to-be-processed task, and the sum of the number of the to-be-processed subtasks and the number of the to-be-processed subtasks allocated by the master node server to other computing node servers does not exceed the queries per second of the service provider server corresponding to the external service required when executing the to-be-processed subtasks.

[0128] Wherein, the to-be-processed task can be pre-stored locally in the master node server, and the to-be-processed task is obtained and its to-be-processed subtasks are determined when the to-be-processed task needs to be processed; it can also be pre-stored in other electronic devices associated with the master node server or in the cloud, such as in a message queue, and the to-be-processed task is obtained and its to-be-processed subtasks are determined when the to-be-processed task needs to be processed; or it can also be directly communicatively connected to the user's terminal device, and when the user triggers a set area in the terminal device, the to-be-processed task is sent to the master node server in real time, and the corresponding master node server receives the to-be-processed task and determines the to-be-processed subtasks corresponding to the to-be-processed task.

[0129] Wherein, the queries per second is used to measure the traffic processed by the server within a specified time and is associated with the data processing capacity of the server.

[0130] Exemplarily, the master node server determines multiple to-be-processed subtasks corresponding to the to-be-processed task, and obtains the queries per second of the service provider server corresponding to the external service required when executing the to-be-processed subtasks; allocates the to-be-processed subtasks to each computing node server corresponding to the master node server based on the queries per second; wherein, the sum of the number of the to-be-processed subtasks allocated to each computing node server does not exceed the queries per second; and sends task allocation messages to the corresponding computing node servers according to the to-be-processed subtasks allocated to each computing node server.

[0131] Correspondingly, the computing node server obtains the task allocation message sent by the master node server according to the to-be-processed subtasks allocated to the computing node server.

[0132] It can be understood that since the sum of the number of the to-be-processed subtasks received by the computing node server and the number of the to-be-processed subtasks allocated by the master node server to other computing node servers does not exceed the queries per second of the service provider server, the service provider server can effectively respond to the access operation of the computing node server to the service provider server when executing the to-be-processed subtasks, which not only ensures the processing efficiency of the to-be-processed subtasks but also takes into account the processing capacity of the service provider server.

[0133] In an alternative implementation manner of the embodiment of the present application, the task to be processed may be a road network task to be processed, and the road network task to be processed includes a plurality of network points; correspondingly, each sub-task to be processed is a pair of network points that needs to obtain road network information.

[0134] Among them, the road network task to be processed can be understood as a road network matrix construction task that needs to be performed in advance during intelligent logistics distribution, so that after the intelligent distribution system receives the distribution destination network point input by the user, it can perform path planning based on the pre-constructed road network matrix and output the planned optimal distribution path. Among them, each element in the road network matrix includes the distribution distance, distribution time, and distribution route between the corresponding distribution start network point and the distribution target network point. Correspondingly, the road network information of the network point pair can be understood as at least one of the distance information, time information, and route information between the two network points of the network point pair.

[0135] In an alternative implementation manner of the embodiment of the present application, obtaining the task assignment message sent by the master node server according to the sub-task to be processed assigned to the computing node server may be to read the task assignment message corresponding to the computing node server from the ActiveMQ in the central node of the message queue MQ cluster; where the task assignment message is sent by the master node server according to the sub-task to be processed assigned to the computing node server.

[0136] Specifically, the master node server, as a message producer, generates task assignment messages for each computing node server according to the sub-tasks to be processed assigned to each computing node server, and transmits the task assignment messages to the central node. Correspondingly, the computing node server, as a message consumer, reads the task assignment message corresponding to itself from the ActiveMQ in the central node.

[0137] In another alternative implementation manner of the embodiment of the present application, obtaining the task assignment message sent by the master node server according to the sub-task to be processed assigned to the computing node server may be to receive the task assignment message sent by the master node server according to the sub-task to be processed assigned to the computing node server and using the zeroMQ mechanism.

[0138] S302. According to the task assignment message, access the service providing server to execute the sub-task to be processed corresponding to the task assignment message.

[0139] Specifically, when the computing node server executes the sub-task to be processed, or according to the task assignment message, it accesses the service providing server corresponding to the external service that needs to be requested when executing the sub-task to be processed, and the service providing server provides the corresponding service to assist the computing node server in executing the sub-task to be processed.

[0140] The technical solution of the embodiment of the present application limits the number of to-be-processed subtasks allocated to the computing node server through the queries per second of the service providing server, so that the sum of the number of to-be-processed subtasks received by the computing node server and the number of to-be-processed subtasks sent by the master node server to other computing node servers does not exceed the queries per second of the service providing server, enabling the service providing server to effectively respond to the corresponding access operations when the computing node server executes the to-be-processed subtasks. This not only ensures the processing efficiency of the to-be-processed subtasks but also takes into account the processing capacity of the service providing server. Therefore, it overcomes the technical problem that the service providing server fails to process tasks in a timely manner, resulting in task processing failures, thereby reducing the number of task processing failures, reducing the time cost of repeated task allocation, and further improving the task processing efficiency.

[0141] Based on the technical solutions of the above embodiments, when the computing node server obtains too many task allocation messages at a single time, there is a situation where it is unable to effectively process the to-be-processed subtasks corresponding to the task allocation messages, which will further affect the task execution results of the to-be-processed tasks. To avoid the occurrence of the above situation, before accessing the service providing server according to the task allocation message, the computing node server can also determine the number of task allocation messages received per second by itself; if the number of to-be-processed subtasks corresponding to the task allocation messages received per second is greater than the queries per second, at least one to-be-processed subtask is discarded according to a preset rule.

[0142] Embodiment 4

[0143] Figure 4A is a flowchart of a task processing method in Embodiment 4 of the present application. The embodiment of the present application provides a preferred implementation manner based on the technical solutions of the above embodiments.

[0144] Combined with Figure 4B the architecture diagram of a road network task processing system shown, the task processing method provided by the embodiment of the present application is described in detail.

[0145] Among them, Figure 4B the road network task processing system shown adopts a master-slave device mode, including a master node server 40 as the master device and computing node servers as slave devices. The road network task processing system further includes a message queue 60, a map server 70, and a database 80.

[0146] As Figure 4A shown, a task processing method includes:

[0147] S401. The master node server initializes a road network snapshot corresponding to the road network task.

[0148] Among them, the road network task includes multiple network points. Any two network points can form paired data. The road network task is the task of obtaining the road network information between each pair of network points. The road network information includes delivery distance, delivery time, and delivery route.

[0149] Among them, the road network information can be obtained by requesting the map server. Therefore, the number of paired data in the road network task is equal to the number of times the map server needs to be requested. Since the paired data needs to be assembled into task packages for distribution, the number of task packages distributed is also equal to the number of paired data.

[0150] When the road network task includes 6 network points, there are 6*(6 - 1) = 30 paired data corresponding to the road network task. Correspondingly, 30 task packages can be formed, and the number of times of requesting the map server is also 30 times.

[0151] The road network snapshot refers to the execution status information of the paired data stored through a series of bitmaps.

[0152] S402. Search for the paired data that has not been successfully executed in the road network snapshot.

[0153] The master node server knows the total number of task packages (i.e., paired data) that need to be distributed. When the master node senses that the task packages of the current road network task have been sent, it will wait for a period of time (8 times the time for the task package to be executed). After the waiting is completed, it will check the execution status information of each paired data in the database, and then store it locally using a bitmap (each byte can store 8 statuses, which can greatly reduce the memory usage).

[0154] S403. Assemble the paired data that has not been successfully executed into task packages.

[0155] Based on the execution status information of the paired data recorded in the road network snapshot, the master node server can sense which paired data has not been successfully executed. Then, it returns the paired data that has not been successfully executed to the process of assembling task packages, continues to search for the task packages that have not been successfully executed due to response timeout or map interface error, etc., resends the searched task packages, and searches for the paired data that was not executed last time in the database after the sending is completed.

[0156] S404. Obtain the query rate per second opened by the map server for the road network task processing system.

[0157] S405. Determine the number of task packages allocated to each computing node server based on the ratio of the query rate per second to the computing node server.

[0158] S406. For the determined number of task packages, send the corresponding number of task packages to each computing node server respectively through a message queue or self-communication.

[0159] For example, the ActiveMQ of the central node in the message queue cluster can be used to issue task messages; or, zeroMQ can be used for end-to-end message issuance.

[0160] Since the issuance of task packages is implemented through a message queue, the number and magnitude of task processing messages sent by the master node server can be controlled by dynamically adjusting or fixedly setting the number of messages issued per second by the message queue. For example, the rate of activeMQ or zeroMQ can be restricted, so as to control the production capacity of the producer (master node server) at the source of the task processing system.

[0161] Since the consumer is the carrier for executing task packages, in order to prevent a large number of message accumulations in the computing node server (consumer), the maximum number of task packages received by each computing node server per second can also be set to the query rate per second of the map server. Of course, when there are multiple computing node servers in the cluster, even if the maximum value is set, because the production rate of the producer is fixed, the entire cluster will eventually return to the time period of sharing the entire query rate per second, without causing waste of map resources.

[0162] When the point pair magnitude of the messages received by the computing node per second is greater than the query rate per second, some task packages will be discarded. The discarded task packages will be re-issued and executed during the next task snapshot collection.

[0163] S407. After receiving the task message package, the computing node server unpacks it.

[0164] S408. After unpacking, the task is executed in slices by accessing the map server.

[0165] S409. Write the point pair data into the database.

[0166] S410. After sending the task packages, the master node server incrementally queries the execution status of the point pairs in the database.

[0167] S411. After the query is completed, mark the execution status corresponding to the point pair data with values in the road network snapshot.

[0168] For example, for the pair data "A - B" formed by network point A and network point B, after the task package corresponding to the pair data "A - B" is successfully executed, the executed status of "A - B" can be modified to successfully executed in the database. When the database uses bitMap to store the executed status, if the bit index corresponding to the successfully executed pair data is 23, then set the value at the 23rd position in this bitMap to "true", indicating that the task package corresponding to this pair data is successfully executed.

[0169] S412. The master node server periodically searches the database until all the road network information corresponding to all pair data is obtained.

[0170] It can be understood that a repeated execution count needs to be set in advance to limit the pair data that has not been successfully executed. When the maximum number of attempts is reached but it is still not successfully executed, a preset fallback calculation rule is used to process this task package.

[0171] Embodiment Five

[0172] Figure 5 is a structural diagram of a task processing device in Embodiment Five of the present application. The embodiments of the present application are applicable to the situation where, after the master node server in the system architecture shown receives a task such as a road network matrix construction waiting to be processed, it distributes and processes the task to be processed. This device is implemented by software and / or hardware and is specifically configured on the master node server in the server cluster. Figure 1A As shown, a task processing device 500 includes: a query rate acquisition module 501, a subtask allocation module 502, and an allocation message sending module 503.

[0173] Such as Figure 5 As shown, a task processing device 500 includes: a query rate acquisition module 501, a subtask allocation module 502, and an allocation message sending module 503.

[0174] The query rate acquisition module 501 is configured to determine a plurality of subtasks to be processed corresponding to the task to be processed, and acquire the query rate per second of the service - providing server corresponding to the external service required when executing the subtasks to be processed.

[0175] The subtask allocation module 502 is configured to allocate the subtasks to be processed to each computing node server corresponding to the master node server based on the query rate per second; wherein, the total number of subtasks to be processed allocated to each computing node server does not exceed the query rate per second.

[0176] The allocation message sending module 503 is configured to send task allocation messages to the corresponding computing node servers respectively according to the subtasks to be processed allocated to each computing node server, so that each computing node server executes the corresponding subtasks to be processed by accessing the service - providing server according to the received task allocation messages.

[0177] The technical solution of the embodiment of the present application obtains the query rate per second of the service - providing server through the query rate acquisition module, and tasks are assigned to multiple to - be - processed subtasks corresponding to the to - be - processed task based on the query rate per second through the subtask assignment module and the assignment message sending module, enabling the service - providing server to effectively respond to the access operations corresponding to the to - be - processed subtasks when the computing node server executes them. This not only ensures the processing efficiency of the to - be - processed subtasks but also takes into account the processing capacity of the service - providing server. Therefore, it overcomes the technical problem that the service - providing server fails to process tasks in a timely manner, resulting in task - processing failures, thereby reducing the number of task - processing failures, reducing the time cost invested in repeated task assignment, and further improving the task - processing efficiency.

[0178] Further, the to - be - processed task is a to - be - processed road network task, and the to - be - processed road network task contains multiple network points.

[0179] Correspondingly, when the query rate acquisition module 501 executes to determine multiple to - be - processed subtasks corresponding to the to - be - processed task, it is specifically used for:

[0180] Determine pairs of network points that need to obtain road network information according to the multiple network points, and use each determined pair of network points as a to - be - processed subtask respectively.

[0181] Correspondingly, the external service that needs to be requested includes a service for providing road network information between the pairs of network points.

[0182] Further, the device further includes a subtask assignment loop module, which is used for:

[0183] After sending task assignment messages to the corresponding computing node servers according to the to - be - processed subtasks assigned to each computing node server, if there are unassigned to - be - processed subtasks, return to execute the to - be - processed subtask assignment operation until all to - be - processed subtasks are assigned.

[0184] Further, the device further includes an execution result determination module, which is used for:

[0185] After all to - be - processed subtasks are assigned, query the database for storing the execution results of the subtasks to determine whether each to - be - processed subtask is successfully executed, and store the determination results.

[0186] Correspondingly, the subtask assignment loop module is further used for:

[0187] Query the determination results, and when there are to - be - processed subtasks that are not successfully executed in the determination results, return to execute the to - be - processed subtask assignment operation until each to - be - processed subtask in the determination results is successfully executed or meets the task assignment stop condition.

[0188] Further, the subtask allocation module 502 includes:

[0189] An allocation quantity determination unit, configured to determine the number of the computing node servers, and determine the subtask allocation quantity corresponding to each computing node server according to the ratio of the queries per second to the determined number;

[0190] A task allocation subunit, configured to select the subtask allocation quantity of the to-be-processed subtasks from the multiple to-be-processed subtasks for each computing node server respectively, and perform task allocation.

[0191] Further, when the task allocation subunit executes the operation of selecting the subtask allocation quantity of the to-be-processed subtasks from the multiple to-be-processed subtasks for each computing node server respectively, it is specifically configured to:

[0192] If the number of the to-be-processed tasks is multiple, determine the to-be-processed tasks corresponding to the selection of the to-be-processed subtasks for each computing node server according to the polling mechanism or the task priority.

[0193] Further, the allocation message sending module 503 is specifically configured to:

[0194] According to the to-be-processed subtasks allocated to each computing node server, send a task allocation message for each computing node server to the central node in the message queue MQ cluster, so that each computing node server reads its own task allocation message from the ActiveMQ in the central node; or,

[0195] According to the to-be-processed subtasks allocated to each computing node server, send task allocation messages to the corresponding computing node servers respectively by using the zeroMQ mechanism.

[0196] Further, the allocation message sending module 503 is specifically configured to:

[0197] According to the to-be-processed subtasks allocated to each computing node server, send task allocation messages to the corresponding computing node servers respectively at a set rate.

[0198] The above task processing device can execute the task processing method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing the task processing method.

[0199] Embodiment Six

[0200] Figure 6 It is a structural diagram of a task processing device in Embodiment Six of the present application. The embodiments of the present application are applicable to Figure 1AThe computing node server shown receives a task to be processed such as a road network matrix construction waiting to be processed, and processes the task to be processed. The device is implemented by software and / or hardware and is specifically configured on the computing node server in the server cluster.

[0201] As Figure 6 shown, a task processing device 600 includes: a distribution message acquisition module 601 and a subtask processing module 602.

[0202] The distribution message acquisition module 601 is used to acquire a task distribution message sent by the master node server according to the subtasks to be processed assigned to the computing node server; wherein, the subtasks to be processed come from the same task to be processed, and the sum of the number of the subtasks to be processed and the number of the subtasks to be processed assigned by the master node server to other computing node servers does not exceed the query per second rate of the service providing server corresponding to the external service required when executing the subtasks to be processed.

[0203] The subtask processing module 602 is used to access the service providing server according to the task distribution message to execute the subtasks to be processed corresponding to the task distribution message.

[0204] The technical solution of the embodiment of the present application limits the number of subtasks to be processed assigned to the computing node server by the distribution message acquisition module based on the query per second rate of the service providing server, so that the sum of the number of subtasks to be processed received by the computing node and the number of subtasks to be processed sent by the master node server to other computing node servers does not exceed the query per second rate of the service providing server, enabling the service providing server to effectively respond to the access operations corresponding to the computing node server when executing the subtasks to be processed, ensuring both the processing efficiency of the subtasks to be processed and taking into account the processing capacity of the service providing server. Therefore, it overcomes the technical problem that the service providing server fails to process tasks in time, reduces the number of task processing failures, reduces the time cost invested in repeated task allocation, and thus improves the task processing efficiency.

[0205] Further, the task to be processed is a road network task to be processed, and the road network task to be processed contains multiple network points, and each subtask to be processed is a pair of network points that need to obtain road network information.

[0206] Further, the device further includes a task discarding module, which is used for:

[0207] Before accessing the service providing server according to the task distribution message, if the number of subtasks to be processed corresponding to the task distribution messages received per second is greater than the query per second rate, at least one subtask to be processed is discarded according to a preset rule.

[0208] Further, the allocation message acquisition module 601 is specifically configured to:

[0209] Read the task allocation message corresponding to the computing node server from ActiveMQ in the central node of the message queue MQ cluster; wherein the task allocation message is sent by the master node server according to the to-be-processed subtasks allocated to the computing node server; or,

[0210] Receive the task allocation message sent by the master node server according to the to-be-processed subtasks allocated to the computing node server and using the zeroMQ mechanism.

[0211] The above task processing device can execute the task processing method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing the task processing method.

[0212] Embodiment VII

[0213] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0214] As Figure 7 shown, it is a block diagram of an electronic device for the task processing method according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0215] As Figure 7 shown, the electronic device includes: one or more processors 701, a memory 702, and an interface for connecting each component, including a high-speed interface and a low-speed interface. Each component is interconnected using different buses and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed within the electronic device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (such as, as a server array, a group of blade servers, or a multi-processor system). Figure 7 Here, one processor 701 is taken as an example.

[0216] The memory 702 is the non-transitory computer-readable storage medium provided by this application. Among them, the memory stores instructions executable by at least one processor, so that the at least one processor executes the task processing method provided by this application. The non-transitory computer-readable storage medium of this application stores computer instructions, and these computer instructions are used to make a computer execute the task processing method provided by this application.

[0217] As a non-transitory computer-readable storage medium, the memory 702 can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the task processing method in the embodiments of this application (for example, Figure 5 the task processing device 500 shown in the figure includes a query rate acquisition module 501, a subtask allocation module 502, and an allocation message sending module 503; or Figure 6 the task processing device 600 shown in the figure includes an allocation message acquisition module 601 and a subtask processing module 602). By running the non-transitory software programs, instructions, and modules stored in the memory 702, the processor 701 executes various functional applications and data processing of the server, that is, implements the task processing method in the above method embodiments.

[0218] The memory 702 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the electronic device that executes the task processing method, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 702 may optionally include a memory remotely set relative to the processor 701, and these remote memories can be connected to the electronic device that executes the task processing method through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0219] The electronic device that executes the task processing method may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703, and the output device 704 can be connected through a bus or other means, Figure 7 taking the connection through the bus as an example.

[0220] The input device 703 can receive input digital or character information, and generate key signal inputs related to user settings and function controls of the electronic device that executes the task processing method, such as input devices like touchscreens, keypads, mice, trackpads, touchpads, pointing sticks, one or more mouse buttons, trackballs, joysticks, etc. The output device 704 can include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), etc. The display device can include, but is not limited to, liquid crystal displays (LCDs), light emitting diode (LED) displays, and plasma displays. In some embodiments, the display device can be a touchscreen.

[0221] The various embodiments of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuit systems, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0222] These computing programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor, and can implement these computing programs using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disks, optical disks, memories, programmable logic devices (PLDs)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal for providing machine instructions and / or data to a programmable processor.

[0223] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0224] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0225] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship between the client and the server is created by computer programs running on the respective computers and having a client-server relationship with each other.

[0226] According to the technical solution of the embodiment of the present application, during the process of the server executing the task processing method, it can obtain the queries per second rate of the service providing server and perform task allocation for multiple pending subtasks corresponding to the pending task based on the queries per second rate, so that the service providing server can effectively respond to the access operations corresponding to the execution of the pending subtasks by the computing node server, which not only ensures the processing efficiency of the pending subtasks but also takes into account the processing capacity of the service providing server. Therefore, it overcomes the technical problem that the service providing server fails to process tasks in time, resulting in task processing failures, thereby reducing the number of task processing failures, reducing the time cost of repeated task allocation, and further improving the task processing efficiency.

[0227] It should be understood that the various forms of processes shown above can be used, with steps reordered, added or deleted. For example, the steps recited in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and no limitations are imposed herein.

[0228] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. A method for task processing, applied to a master node server, characterized in that, Including: Based on multiple network nodes in the to-be-processed road network task, determine the node pairs that need to obtain road network information, use each determined node pair as a to-be-processed subtask respectively, and obtain the queries per second rate of the service providing server corresponding to the external service required when executing multiple said to-be-processed subtasks; wherein, the to-be-processed road network task is to perform route planning based on a pre-constructed road network matrix during intelligent logistics distribution and output the optimal distribution route; the external service that needs to be requested includes the service for providing the road network information between the node pairs; the to-be-processed road network task is a to-be-processed task; Determine the allocable task quantity of each computing node server and the total of the allocable task quantities; Based on the ratio of the allocable task quantity corresponding to each computing node server to the total of the allocable task quantities, determine the task allocation ratio corresponding to each computing node server respectively; Based on the product of the queries per second rate and the task allocation ratio, determine the allocable task quantity corresponding to each computing node server respectively, and use the allocable task quantity as the subtask allocation quantity; Select the to-be-processed subtasks with the subtask allocation quantity from the multiple to-be-processed subtasks for each computing node server respectively for task allocation; wherein, the total quantity of the to-be-processed subtasks allocated to each computing node server does not exceed the queries per second rate; Send task allocation messages to the corresponding computing node servers respectively according to the to-be-processed subtasks allocated to each computing node server, so that each computing node server executes the corresponding to-be-processed subtask by accessing the service providing server according to the received task allocation message; Integrate the subtask execution results fed back by each computing node server and calculate the final result.

2. The method according to claim 1, characterized in that, After sending task allocation messages to the corresponding computing node servers respectively according to the to-be-processed subtasks allocated to each computing node server, it further includes: If there are unallocated to-be-processed subtasks, return to execute the to-be-processed subtask allocation operation until all to-be-processed subtasks are allocated.

3. The method according to claim 2, wherein After all to-be-processed subtasks are allocated, it further includes: Query the database for storing the subtask execution results to determine whether each to-be-processed subtask is successfully executed and store the determination result; Query the determination result, and when there are to-be-processed subtasks that are not successfully executed in the determination result, return to execute the to-be-processed subtask allocation operation until each to-be-processed subtask in the determination result is successfully executed or meets the task allocation stop condition.

4. The method according to claim 1, wherein The step of selecting the to-be-processed subtasks with the subtask allocation quantity from the multiple to-be-processed subtasks for each computing node server respectively includes: If the quantity of the to-be-processed tasks is multiple, determine the to-be-processed tasks corresponding to selecting to-be-processed subtasks for each computing node server according to the polling mechanism or task priority.

5. The method according to claim 1, wherein The step of sending task allocation messages to the corresponding computing node servers respectively according to the to-be-processed subtasks allocated to each computing node server includes: According to the sub-tasks to be processed assigned to each computing node server, send task assignment messages for each computing node server to the central node in the message queue MQ cluster, so that each computing node server reads its own task assignment message from ActiveMQ in the central node; or, According to the sub-tasks to be processed assigned to each computing node server, use the zeroMQ mechanism to send task assignment messages to the corresponding computing node servers respectively.

6. The method according to any one of claims 1-5, characterized in that, The step of respectively sending task assignment messages to the corresponding computing node servers according to the sub-tasks to be processed assigned to each computing node server includes: According to the sub-tasks to be processed assigned to each computing node server, send task assignment messages to the corresponding computing node servers at a set rate respectively.

7. A task processing method, characterized in that, Applied to a computing node server, it includes: Obtain the task assignment message sent by the master node server according to the sub-tasks to be processed assigned to the computing node server; wherein, the sub-tasks to be processed come from the same to-be-processed task, and the sum of the number of sub-tasks to be processed assigned to the computing node server and the number of sub-tasks to be processed assigned by the master node server to other computing node servers does not exceed the queries per second rate of the service provider server corresponding to the external service required when executing the sub-tasks to be processed; wherein, the sub-tasks to be processed assigned to the computing node server include: determining the allocable task quantity of each computing node server and the total sum of the allocable task quantities; according to the ratio of the allocable task quantity corresponding to the computing node server to the total sum of the allocable task quantities, respectively determining the task assignment ratio corresponding to each computing node server; according to the product of the queries per second rate and the task assignment ratio, respectively determining the allocable task quantity corresponding to each computing node server, and taking the allocable task quantity as the sub-task assignment quantity; respectively selecting the sub-task assignment quantity of sub-tasks to be processed from a plurality of the sub-tasks to be processed for each computing node server for task assignment; According to the task assignment message, access the service provider server to execute the sub-tasks to be processed corresponding to the task assignment message, and feedback the execution results of each sub-task to the master node server for the master node server to integrate the sub-task execution results fed back by each computing node server and calculate the final result; Wherein, the to-be-processed task is a to-be-processed road network task, the to-be-processed road network task contains multiple network points, and each sub-task to be processed is a pair of network points that need to obtain road network information; the to-be-processed road network task is to perform path planning based on a pre-constructed road network matrix during intelligent logistics distribution and output the planned optimal distribution path; correspondingly, the external service that needs to be requested includes the service for providing the road network information between the pair of network points.

8. The method according to claim 7, wherein Before accessing the service provider server according to the task assignment message, it further includes: If the number of sub-tasks to be processed corresponding to the task assignment messages received per second is greater than the queries per second rate, discard at least one sub-task to be processed according to a preset rule.

9. The method according to any one of claims 7-8, characterized in that, The task assignment message sent by the acquisition master node server according to the to-be-processed subtasks assigned to the computing node server includes: Reading, from ActiveMQ in the central node of the message queue MQ cluster, the task assignment message corresponding to the computing node server; wherein the task assignment message is sent by the master node server according to the to-be-processed subtasks assigned to the computing node server; or, Receiving the task assignment message sent by the master node server according to the to-be-processed subtasks assigned to the computing node server and using the zeroMQ mechanism.

10. A device for task processing, configured in a master node server, characterized in that, It includes: A query rate acquisition module, configured to determine, according to multiple network points in the to-be-processed road network task, the network point pairs that need to acquire road network information, use each determined network point pair as a to-be-processed subtask respectively, and acquire the query rate per second of the service providing server corresponding to the external service that needs to be requested when executing multiple said to-be-processed subtasks; wherein, the to-be-processed road network task is for path planning based on a pre-constructed road network matrix during intelligent logistics distribution and outputting the optimal distribution path planned; the external service that needs to be requested includes the service for providing the road network information between the network point pairs; the to-be-processed road network task is a to-be-processed task; A subtask assignment module, configured to determine the assignable task quantity of each computing node server and the total sum of the assignable task quantities; determine the task assignment ratio corresponding to each computing node server respectively according to the ratio of the assignable task quantity corresponding to the computing node server to the total sum of the assignable task quantities; determine the assignable task quantity corresponding to each computing node server respectively according to the product of the query rate per second and the task assignment ratio, and use the assignable task quantity as the subtask assignment quantity; select the subtask assignment quantity of to-be-processed subtasks from the multiple to-be-processed subtasks for each computing node server respectively for task assignment; wherein, the total sum of the to-be-processed subtasks assigned to each computing node server does not exceed the query rate per second; An assignment message sending module, configured to send task assignment messages to the corresponding computing node servers respectively according to the to-be-processed subtasks assigned to each computing node server, so that each computing node server executes the corresponding to-be-processed subtasks by accessing the service providing server according to the received task assignment message; An execution result integration module, configured to integrate the subtask execution results fed back by each computing node server and calculate the final result.

11. A task processing device, characterized in that, Configured in the computing node server, it includes: An allocation message acquisition module, configured to acquire a task allocation message sent by a master node server according to the to-be-processed subtasks allocated to the computing node server; wherein, the to-be-processed subtasks come from the same to-be-processed task, and the sum of the number of to-be-processed subtasks allocated by the master node server to other computing node servers does not exceed the queries per second of the service provider server corresponding to the external service required when executing the to-be-processed subtasks; wherein, the to-be-processed subtasks allocated to the computing node server include: determining the allocable task quantity and the total of allocable task quantities of each computing node server; respectively determining the task allocation ratio corresponding to each computing node server according to the ratio of the allocable task quantity corresponding to the computing node server to the total of allocable task quantities; respectively determining the allocable task quantity corresponding to each computing node server according to the product of the queries per second and the task allocation ratio, and using the allocable task quantity as the subtask allocation quantity; respectively selecting, for each computing node server, the to-be-processed subtasks with the subtask allocation quantity from the multiple to-be-processed subtasks for task allocation; A subtask processing module, configured to access the service provider server according to the task allocation message to execute the to-be-processed subtasks corresponding to the task allocation message, and feedback the execution results of each subtask to the master node server for the master node server to integrate the subtask execution results fed back by each computing node server and calculate the final result; Wherein, the to-be-processed task is a to-be-processed road network task, the to-be-processed road network task contains multiple network points, and each to-be-processed subtask is a pair of network points that need to obtain road network information; the to-be-processed road network task is to perform path planning based on a pre-constructed road network matrix during intelligent logistics distribution and output the planned optimal distribution path; correspondingly, the required external service includes the service for providing the road network information between the pair of network points.

12. An electronic device, characterized in that, Comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute a task processing method according to any one of claims 1-6.

13. An electronic device, characterized in that, Comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute a task processing method according to any one of claims 7-9.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute a task processing method according to any one of claims 1-6.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute a task processing method according to any one of claims 7-9.

16. A task processing method, characterized in that, Comprising: Determining a plurality of to-be-processed subtasks corresponding to the to-be-processed task; Send at least one task assignment message to the service providing server according to the queries per second rate of the service providing server that needs to be accessed when executing the to-be-processed subtask, so that the service providing server executes the to-be-processed subtasks corresponding to each of the task assignment messages; Wherein, the to-be-processed task is a to-be-processed road network task, the to-be-processed road network task contains multiple network points, and each to-be-processed subtask is a pair of network points that needs to obtain road network information; the to-be-processed road network task is to perform path planning based on a pre-constructed road network matrix during intelligent logistics distribution and output the optimal distribution path planned; correspondingly, the service providing server is used to provide the service of the road network information between the pair of network points; the to-be-processed subtask is a to-be-processed subtask assigned by the master node server to the computing node server; the to-be-processed subtask assigned to the computing node server includes: determining the allocable task quantity of each computing node server and the total sum of the allocable task quantities; respectively determining the task assignment ratio corresponding to each computing node server according to the ratio of the allocable task quantity corresponding to the computing node server to the total sum of the allocable task quantities; respectively determining the allocable task quantity corresponding to each computing node server according to the product of the queries per second rate and the task assignment ratio, and using the allocable task quantity as the subtask assignment quantity; respectively selecting the subtask assignment quantity of to-be-processed subtasks from the multiple to-be-processed subtasks for each computing node server for task assignment.

Images