Task processing method and device, storage medium and electronic equipment

Abstract

The invention provides a task processing method and device, a storage medium and electronic equipment. The method comprises the steps that a state switching instruction is obtained, the state switching instruction is used for switching the object state of a current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in the distributed data processing system; in response to the state switching instruction, switching the current processing object from the first object state to a reference object state; under the condition that a target task forwarded by a proxy object is received, the target task is cached, and the proxy object is an object used for task forwarding in the distributed data processing system; and under the condition that the current processing object meets the state switching condition, switching the object state of the current processing object from the reference object state to a second object state, and processing the cached target task. The technical problem that node read-write conflicts can be caused by an existing capacity expansion and contraction scheme is solved.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a task processing method and device, a storage medium and an electronic device. Background Art

[0002] In a distributed data processing system, different data processing tasks can be forwarded to different task processing nodes through proxy nodes for processing, thereby improving the concurrent processing efficiency of the system through multiple task processing nodes.

[0003] In the above-mentioned distributed data processing system, the task processing nodes in the system are often expanded or reduced due to the adjustment of the business scale. In the process of the above-mentioned expansion or reduction operation, since the routing tables stored in multiple proxy nodes are difficult to update synchronously, two versions of the routing tables, old and new, may be stored in different proxy nodes, which may lead to read and write conflicts between the task processing nodes affected by the reduction or expansion operation and other task processing nodes. In other words, the existing expansion and reduction operation process will lead to technical problems of node read and write conflicts.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a task processing method and device, a storage medium and an electronic device to at least solve the technical problem of node read and write conflicts caused by the existing expansion and contraction operation process.

[0006] According to one aspect of an embodiment of the present invention, a task processing method is provided, comprising: obtaining a state switching instruction, wherein the state switching instruction is used to switch the object state of a current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in a distributed data processing system; in response to the state switching instruction, switching the current processing object from the first object state to a reference object state; in the case of receiving a target task forwarded by a proxy object, caching the target task, wherein the proxy object is an object used for task forwarding in the distributed data processing system; in the case of the current processing object satisfying a state switching condition, switching the object state of the current processing object from the reference object state to the second object state, and processing the cached target task.

[0007] According to another aspect of an embodiment of the present invention, a task processing device is provided, comprising: an acquisition unit, configured to acquire a state switching instruction, wherein the state switching instruction is configured to switch the object state of a currently processed object from a first object state to a second object state, wherein the currently processed object is an object used for task processing in a distributed data processing system; a first switching unit, configured to switch the currently processed object from the first object state to a reference object state in response to the state switching instruction; a cache unit, configured to cache the target task upon receiving a target task forwarded by a proxy object, wherein the proxy object is an object used for task forwarding in the distributed data processing system; and a second switching unit, configured to switch the object state of the currently processed object from the reference object state to the second object state, and to process the cached target task, when the currently processed object satisfies a state switching condition.

[0008] Optionally, the second switching unit includes: a first switching module, used to switch the object state of the current processing object from a pre-exit state to an exit state when the state switching instruction is a shrinking instruction, wherein the shrinking instruction is used to switch the object state of the current processing object from an online state to an exit state, and the reference object state includes the pre-exit state; and a forwarding module, used to send the cached target task to the proxy object.

[0009] Optionally, the above-mentioned first switching module is also used to: obtain a reference task, wherein the above-mentioned reference task is a data processing task received by the above-mentioned current processing object in the above-mentioned online state; when the above-mentioned reference task is in an unfinished state, the above-mentioned reference task is processed by the above-mentioned current processing object in the above-mentioned pre-exit state.

[0010] Optionally, the first switching module is also used for at least one of the following: determining that the current processing object satisfies the state switching condition when the duration of the current processing object being in the pre-exit state is greater than or equal to a first threshold; determining that the current processing object satisfies the state switching condition when the reference task associated with the current processing object is in a completed state, wherein the reference task is a data processing task received by the current processing object in the online state.

[0011] Optionally, the above-mentioned first switching module is also used for at least one of the following: sending a first state switching prompt information to the above-mentioned proxy object, wherein the above-mentioned first state switching prompt information is used to update a mapping relationship list stored in the above-mentioned proxy object, and the above-mentioned mapping relationship list is used to indicate the correspondence between the data processing tasks to be processed and the task processing objects; when receiving the pending tasks forwarded by the above-mentioned proxy object, the above-mentioned pending tasks are sent to the above-mentioned proxy object.

[0012] Optionally, the above-mentioned first switching module is also used to: update the return parameter in the task return prompt information carrying the above-mentioned pending task, wherein the above-mentioned return parameter is used to indicate the cumulative number of times that the above-mentioned current processing object sends the above-mentioned pending task to the above-mentioned proxy object, and the above-mentioned proxy object is used to stop forwarding the above-mentioned pending task to the above-mentioned current processing object when the above-mentioned return parameter meets the target condition.

[0013] Optionally, the second switching unit includes: a second switching module, used to switch the object state of the current processing object from a pre-online state to an online state when the state switching instruction is an expansion instruction; a processing module, used to process the cached target task through the current processing object in the online state, wherein the expansion instruction is used to switch the object state of the current processing object from an exit state to an online state, and the reference object state includes the pre-online state.

[0014] Optionally, the second switching module is further used to determine that the current processing object satisfies the state switching condition when the current processing object is in the pre-online state and the duration of receiving the first target task is greater than or equal to a second threshold.

[0015] Optionally, the above-mentioned second switching module is also used to: send a second state switching prompt information to the above-mentioned proxy object, wherein the above-mentioned second state switching prompt information is used to update the mapping relationship list saved in the above-mentioned proxy object, and the above-mentioned mapping relationship list is used to indicate the correspondence between the data processing tasks to be processed and the task processing objects.

[0016] Optionally, the second switching module is further configured to: upon acquiring a target task forwarded by the proxy object, process the target task through the current processing object in the online state.

[0017] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned task processing method when running.

[0018] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above task processing method.

[0019] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the task processing method through the computer program.

[0020] In an embodiment of the present invention, a state switching instruction is obtained, wherein the state switching instruction is used to switch the object state of a current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in a distributed data processing system; in response to the state switching instruction, the current processing object is switched from the first object state to a reference object state; in the case of receiving a target task forwarded by a proxy object, the target task is cached, wherein the proxy object is an object used for task forwarding in a distributed data processing system; in the case of the current processing object satisfying a state switching condition, the object state of the current processing object is switched from the reference object state to the second object state, and the cached target task is processed, thereby achieving accurate control of the task processing process of the processing node during the expansion and contraction process of the distributed data processing system.

[0021] Through the above-mentioned implementation mode of the present application, by caching and controlling the processing progress of the related processing tasks received by the current processing object in the process of switching the object state according to the system expansion and contraction operation, task processing time is reserved for other processing objects in the system used to process related tasks, thereby avoiding task processing conflicts caused by untimely updating of routing tables in different proxy objects during the system expansion and contraction operation, solving the technical problem of data reading and writing conflicts generated during the expansion and contraction operation of the existing distributed data processing system, and improving the operating efficiency of the distributed data processing system and the reliability of task processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 is a schematic diagram of a hardware environment of an optional task processing method according to an embodiment of the present invention;

[0024] Figure 2 is a flowchart of an optional task processing method according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of an optional task processing method according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of another optional task processing method according to an embodiment of the present invention;

[0027] Figure 5 is a schematic diagram of another optional task processing method according to an embodiment of the present invention;

[0028] Figure 6 is a timing diagram of an optional task processing method according to an embodiment of the present invention;

[0029] Figure 7 is a timing diagram of another optional task processing method according to an embodiment of the present invention;

[0030] Figure 8 is a schematic diagram of another optional task processing method according to an embodiment of the present invention;

[0031] Fig. 9 is a schematic diagram of another optional task processing method according to an embodiment of the present invention;

[0032] Fig.10 is a schematic diagram of another optional task processing method according to an embodiment of the present invention;

[0033] Fig.11 A schematic diagram of the structure of an optional task processing device according to an embodiment of the present invention;

[0034] Fig.12 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] According to one aspect of the embodiment of the present application, a task processing method is provided. As an optional implementation, the task processing method can be but is not limited to being applied to Figure 1 The task processing hardware system shown in FIG. 1 is composed of a service node 102, a network 104, a task node 106, and a task node 108. Figure 1 As shown, the service node 102 is connected to the task node 106 and the task node 108 through the network 104. The above network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that realize wireless communication. The above network 104 includes a plurality of proxy nodes for distributing data processing tasks, and the corresponding task nodes assigned to different tasks are determined by the stored routing table in the proxy node. The above service node 102 can be used to generate corresponding data processing tasks according to business needs. The above data processing tasks may include but are not limited to data integration processing, reading processing, writing processing, data calculation processing and other specific data processing processes. After the above service node 102 generates the data processing tasks to be processed, the relevant tasks can be forwarded to the corresponding task nodes through the proxy nodes included in the network 104.

[0038] exist Figure 1 The task processing hardware system shown may include multiple task processing nodes including task node 106 and task node 108, and the above-mentioned task processing nodes may be specifically different server instances. Task node 106 and task node 108 may include processing engine 110 and processing engine 112 respectively, and the above-mentioned processing engine 110 and processing engine 112 may be used to perform the above-mentioned data processing tasks. Optionally, the process of performing task processing in the above-mentioned task node may be implemented by the task process running in the above-mentioned task node, and after the corresponding data processing task is completed by the above-mentioned task, the processing result may be forwarded to the corresponding data storage server through network 110.

[0039] exist Figure 1 In the hardware system shown, the above task processing method may specifically include the following steps:

[0040] As shown in step S102, the task node 106 obtains a state switching instruction, wherein the state switching instruction is used to switch the object state of the current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in a distributed data processing system; it should be noted that, in the case where a single system process for processing data tasks is running in the task node 106, the above-mentioned current processing object may specifically be the above-mentioned task node 106; in the case where multiple system processes for processing the above-mentioned data processing tasks are running in the above-mentioned task node 106, the above-mentioned current processing object may be a system process among the above-mentioned multiple system processes that needs to adjust the process state according to the system expansion and contraction operation.

[0041] In step S104, the task node 106 switches the current processing object from the first object state to the reference object state in response to the state switching instruction;

[0042] Then, in step S106, the service node 106 sends the target task to be processed through the proxy object in the network 104, wherein the proxy object stores a mapping relationship list indicating the mapping relationship between the task identifier and the processing object identifier, and the proxy object is used to forward the target task to be processed to the node 106;

[0043] Next, node 106 executes S108-S110, and caches the target task when receiving the target task forwarded by the proxy object, wherein the proxy object is an object used for task forwarding in the distributed data processing system; when the current processing object meets the state switching condition, the object state of the current processing object is switched from the reference object state to the second object state, and the cached target task is processed.

[0044] Specifically, in the case of the state switching instruction scaling down instruction, the above step S110 may be specifically S110-2 and S110-3, the task node 106 returns the cached target task to the proxy object in the network 104, and the proxy object in the network 104 forwards the target task to the task node 108 according to the updated mapping relationship list;

[0045] In the case where the state switching instruction is a capacity expansion instruction, the step S110 may be specifically S110 - 1 , where the task node 106 processes the cached target task locally.

[0046] In an embodiment of the present invention, a state switching instruction is obtained, wherein the state switching instruction is used to switch the object state of a current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in a distributed data processing system; in response to the state switching instruction, the current processing object is switched from the first object state to a reference object state; in the case of receiving a target task forwarded by a proxy object, the target task is cached, wherein the proxy object is an object used for task forwarding in a distributed data processing system; in the case of the current processing object satisfying a state switching condition, the object state of the current processing object is switched from the reference object state to the second object state, and the cached target task is processed, thereby achieving accurate control of the task processing process of the processing node during the expansion and contraction process of the distributed data processing system.

[0047] Through the above-mentioned implementation mode of the present application, by caching and controlling the processing progress of the related processing tasks received by the current processing object in the process of switching the object state according to the system expansion and contraction operation, task processing time is reserved for other processing objects in the system used to process related tasks, thereby avoiding task processing conflicts caused by untimely updating of routing tables in different proxy objects during the system expansion and contraction operation, solving the technical problem of data reading and writing conflicts generated during the expansion and contraction operation of the existing distributed data processing system, and improving the operating efficiency of the distributed data processing system and the reliability of task processing.

[0048] As an optional implementation, Figure 2 As shown, the above task processing method includes the following steps:

[0049] S202, obtaining a state switching instruction, wherein the state switching instruction is used to switch the object state of a current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in a distributed data processing system;

[0050] S204, in response to the state switching instruction, switching the current processing object from the first object state to the reference object state;

[0051] S206, caching the target task when receiving the target task forwarded by the proxy object, wherein the proxy object is an object used for task forwarding in the distributed data processing system;

[0052] S208 , when the current processing object meets the state switching condition, the object state of the current processing object is switched from the reference object state to the second object state, and the cached target task is processed.

[0053] It should be noted that the above-mentioned implementation of the present application can be implemented by a task processing object in a distributed data processing system. The above-mentioned distributed data processing system may include multiple business logic nodes, proxy nodes and task processing nodes. The above-mentioned business logic nodes can be used to generate tasks to be processed for implementing specific Internet services. The above-mentioned proxy nodes may include routing tables to forward different tasks to be processed to their corresponding task processing nodes.

[0054] It can be understood that in the above-mentioned embodiments of the present application, the above-mentioned task processing node can be used for task processing. Further, the above-mentioned task processing node can execute the above-mentioned data processing task through at least one task process running therein.

[0055] In the above-mentioned implementation mode of the present application, the above-mentioned state switching instruction may be a control instruction generated in response to an expansion operation or a reduction operation performed on the above-mentioned distributed data processing system, and is generated for a processing object used to process a specific data processing task. Optionally, the above-mentioned expansion operation and reduction operation may be for either a task processing node in the above-mentioned distributed data processing system or a task processing process running in the above-mentioned task processing node. Correspondingly, in the case where the above-mentioned expansion operation and reduction operation are for the above-mentioned task processing node, the current processing object in the above-mentioned steps S202 to S208 may be a task processing node in the distributed data processing system; in the case where the above-mentioned expansion operation and reduction operation are for the above-mentioned task processing process, the current processing object in the above-mentioned steps S202 to S208 may be a task processing process running in the above-mentioned task processing node.

[0056] It should be noted that the above-mentioned capacity expansion operation can be used to indicate that when the specific business volume increases, which in turn leads to an increase in data processing tasks, more task processing nodes are started in the above-mentioned distributed data processing system to process data processing tasks, and more task processing processes are started in the above-mentioned distributed data processing system to process data processing tasks; the above-mentioned capacity expansion operation can be used to indicate that when the specific business volume decreases, which in turn leads to a reduction in data processing tasks, some task processing nodes are reduced in the above-mentioned distributed data processing system, and some task processing processes are terminated in the above-mentioned distributed data processing system to save system resources.

[0057] It can be understood that in the process of performing expansion or reduction operations on the above-mentioned distributed data processing system, as the task nodes or task processes are started or shut down, the routing tables in each proxy node need to be adjusted accordingly to forward the data processing tasks to the correct node or process.

[0058] like Figure 3As shown, when the routing tables in proxy node 301 (i.e., the proxy object) and proxy node 302 in the distributed data processing system are the same, for tasks with the same identifier, the consistent hashing algorithm can determine the identifier of the same service node, and then accurately forward the data processing task. Figure 3 In the figure, for the data processing task with task identifier Key=1, by performing identifier mapping via proxy node 301, it can be determined that the service node used to process the task is service node 303 (i.e., the current processing object mentioned above); by performing identifier mapping via proxy node 302 that stores the same routing table, it can be determined that the service node used to process the task is also service node 303, and then read and write operations can be performed on the same task through service node 303.

[0059] The following combination Figure 4 A possible conflict situation caused by the expansion operation of a distributed data processing system is described. Before the expansion operation of the distributed data processing system, the data processing tasks with the task identifier Key=1 are all forwarded to the service node 403 for processing. After the expansion operation of the distributed data processing system, the data processing tasks with the task identifier Key=1 need to be forwarded to the service node 404 for processing. During the expansion process, the update process of the routing table in each proxy node may not be unified. For example, at the target time, Figure 4 The routing table stored in the proxy node 401 is the old routing table, which indicates that the data processing task with the task identifier Key=1 is forwarded to the service node 403 for processing; at the target time, Figure 4 The proxy node 401 in the example has been updated with a new routing table, which indicates that the data processing task with the task identifier Key=1 is forwarded to the service node 404 for processing. Figure 4 In the illustrated case, when proxy node 401 and proxy node 402 receive a data processing task with task identifier Key=1, proxy node 401 will forward the task to service node 403 for processing, and proxy node 402 will forward the task to service node 404 for processing. Since the storage location of the relevant data of the same task may be unique, data write conflicts may occur when service node 403 and service node 404 process the task separately;

[0060] The following combination Figure 5Another possible conflict situation caused by scaling down the distributed data processing system is described below. Before scaling down the distributed data processing system, all data processing tasks with task identifier Key=1 are forwarded to service node 503 for processing. After scaling down the distributed data processing system, the data processing tasks with task identifier Key=1 need to be forwarded to service node 504 for processing. During the scaling down process, the update process of the routing table in each proxy node may differ from the actual operating state of service node 503. For example, at the target time, Figure 5 The routing tables stored in the proxy nodes 501 and 502 have been updated to obtain new routing tables, which indicate that the data processing task with task identifier Key=1 is forwarded to the service node 404 for processing; and at the reference time before the target time (i.e., the historical time before the downscaling operation), the proxy node 501 has forwarded the data processing task with task identifier Key=1 to the service node 403 for processing according to the old routing table, and when the service node 503 has not completed the processing of the task at the target time, the proxy node 502 does not know the node status of the service node 503, and then directly forwards the data processing task with Key=1 to the service node 504 for processing according to the new routing table, which leads to a data write conflict between the service node 503 and the service node 504 for the same task.

[0061] Therefore, in the process of expanding or reducing the capacity of a distributed data processing system, it is easy for the task processing request to be migrated due to inconsistent updates of the routing tables between proxy nodes and inconsistencies between the status of the service node and the node status recorded in the proxy node, which in turn causes the same task to be executed by two nodes at the same time, resulting in a write conflict.

[0062] In the above-mentioned implementation mode of the present application, when the current processing object receives the above-mentioned state switching instruction, as in steps S204 and S206, the current processing object can be first switched to the reference object state, and the received new target task can be cached in this state, and then the state inconsistency and routing table inconsistency that may exist in the system expansion and contraction process can be synchronized through the cache operation, and then the object state of the current processing object can be switched through step S208, and the cached target task can be processed accordingly, thereby avoiding the problem of read-write conflicts that may be caused during the system expansion and contraction process.

[0063] Optionally, the method of caching the received new target task in the reference object state can be determined according to the object type of the current processing object. When the current processing object is a server node, a cache process can be created to cache the received new target task; when the current processing object is a data processing process, a cache queue can be created to cache the task identifier of the received new target task.

[0064] Through the above-mentioned implementation mode of the present application, by caching and controlling the processing progress of the related processing tasks received by the current processing object in the process of switching the object state according to the system expansion and contraction operation, task processing time is reserved for other processing objects in the system used to process related tasks, thereby avoiding task processing conflicts caused by untimely updating of routing tables in different proxy objects during the system expansion and contraction operation, solving the technical problem of data reading and writing conflicts generated during the expansion and contraction operation of the existing distributed data processing system, and improving the operating efficiency of the distributed data processing system and the reliability of task processing.

[0065] In the case where the state switching instruction is a shrinking instruction, when the current processing object meets the state switching condition, switching the object state of the current processing object from the reference object state to the second object state, and processing the cached target task includes:

[0066] S1, when the state switching instruction is a shrinking instruction, switching the object state of the current processing object from a pre-exit state to an exit state, wherein the shrinking instruction is used to switch the object state of the current processing object from an online state to an exit state, and the reference object state includes a pre-exit state;

[0067] S2, sends the cached target task to the proxy object.

[0068] It can be understood that, when the state switching instruction is a shrinking instruction, the shrinking instruction can be used to control the current processing object to switch from an online state to an exit state. Correspondingly, the reference object state can be specifically a pre-exit state.

[0069] In this embodiment, when the current processing object receives a scaling-down instruction, the current processing object can first be switched from an online state to a pre-exit state, and then the target task received in the pre-exit state can be cached. Then, when the current processing object meets the switching conditions, the current processing object can be switched from the pre-exit state to the exit state, and the cached target task received in the pre-exit state can be returned and sent to the proxy object.

[0070] It can be understood that in this embodiment, when the state switching instruction is a shrinking instruction, the object state of the current processing object is switched from the pre-exit state to the exit state; the cached target task is sent to the proxy object, so that the current processing object in the pre-exit state receives the pending task forwarded by the proxy object whose routing table has not been updated, and when the state switching condition is met, the received pending target task is returned to the proxy object, so that the proxy object correctly forwards the pending target task based on the updated routing table, thereby solving the technical problem of read-write conflicts caused by different objects processing the same task during system shrinkage.

[0071] In an optional implementation, after the current processing object is switched from the first object state to the reference object state in response to the state switching instruction, the method further includes:

[0072] S1, obtaining a reference task, wherein the reference task is a data processing task received by the current processing object in an online state;

[0073] S2, when the reference task is in an unfinished state, the reference task is processed by the current processing object in a pre-exit state.

[0074] It can be understood that in this embodiment, when the current processing object is in the pre-exit state, the unfinished reference task received in the online state can continue to be executed, thereby ensuring the read-write consistency during the execution of the same task.

[0075] In an optional implementation manner, before switching the object state of the currently processed object from the pre-exit state to the exit state, at least one of the following is further included:

[0076] Method 1: when the duration of the current processing object being in the pre-exit state is greater than or equal to the first threshold, determining that the current processing object meets the state switching condition;

[0077] Method 2: when the reference task associated with the current processing object is in the completed state, determine whether the current processing object meets the state switching condition, wherein the reference task is a data processing task received by the current processing object in the online state.

[0078] It can be understood that in the above-mentioned implementation manner of the present application, whether the current processing object meets the state switching condition can be determined in two ways.

[0079] The first method is to determine whether the current processing object meets the state switching condition according to the duration of the current processing object being in the pre-exit state. In this embodiment, the current processing object can be used to process the unfinished reference tasks received in the online state during the pre-exit state, and then ensure that the current processing object completes the processing of the unfinished tasks by setting the duration; in addition, the above duration can also be determined according to the duration required for the update of the routing table of each proxy object in the system. For example, according to statistics, it takes up to 500ms for the routing tables of all proxy nodes in the system to be updated, and then the above first threshold can be set to a value greater than the above 500ms;

[0080] In the second method mentioned above, whether the current processing object meets the state switching condition can be determined based on the processing status of the unfinished reference task received in the online state, that is, when the above-mentioned reference tasks have been processed, the current processing object is switched to the exit state, thereby ensuring the consistency of reading and writing of the same task.

[0081] In another optional implementation, it is also possible to determine whether the current processing object meets the state switching condition according to the task amount of the cached target task; that is, when the task amount of the cached target task is greater than or equal to the target amount, it is determined that the current processing object meets the state switching condition;

[0082] In another optional implementation, whether the current processing object meets the state switching condition can also be determined based on the data volume of the cached target task; that is, when the data volume of the cached target task is greater than or equal to the target data volume, it is determined that the current processing object meets the state switching condition.

[0083] It can be understood that the above cache conditions can be used to determine whether the current processing object meets the state switching conditions alone, and can also be used in combination to determine whether the current processing object meets the state switching conditions. In this embodiment, the specific judgment method is not limited.

[0084] Through the above implementation of the present application, through the combination of the above pre-exit state and the state switching condition, the read-write consistency of processing the same task during the system scaling down process is ensured, and the read-write conflict is avoided.

[0085] In an optional implementation manner, after the object state of the currently processed object is switched from the pre-exit state to the exit state, at least one of the following is further included:

[0086] Method 1: sending a first state switching prompt message to the proxy object, wherein the first state switching prompt message is used to update a mapping relationship list stored in the proxy object, and the mapping relationship list is used to indicate a corresponding relationship between a data processing task to be processed and a task processing object;

[0087] Method 2: When receiving a pending task forwarded by a proxy object, the pending task is sent to the proxy object.

[0088] It can be understood that in this embodiment, after the current processing object is switched to the exit state, the first state switching prompt information can be sent to the proxy object to indicate to the proxy object that the current processing object is in the exit state, thereby notifying the proxy object to update the object state of the current processing object in the routing table stored therein;

[0089] In addition, when the current processing object is in the exit state, in addition to the need to return the cached target task to the proxy object to instruct the proxy object to forward the target task to be processed to other processing objects according to the updated routing table, since there are task messages that are still in the transmission process, new tasks to be processed may also be received, and the newly received tasks to be processed need to be returned directly to the proxy object so that the proxy object can forward the target task to be processed to other processing objects for task processing based on the new routing table.

[0090] Through the above-mentioned implementation mode of the present application, a first state switching prompt message is sent to the proxy object; when a pending task forwarded by the proxy object is received, the pending task is sent to the proxy object, thereby ensuring that the routing table stored in the proxy object is updated in a timely manner, and the pending task transmitted in transit based on the old routing table is directly returned, and is transmitted to the correct processing object through the proxy object in time for task processing.

[0091] In an optional embodiment, the above-mentioned process of, when receiving a pending task forwarded by a proxy object, before sending the pending task to the proxy object, further includes: updating a return parameter in the task return prompt information carrying the pending task, wherein the return parameter is used to indicate the cumulative number of times the current processing object sends the pending task to the proxy object, and the proxy object is used to stop forwarding the pending task to the current processing object when the return parameter meets the target condition.

[0092] It is understandable that when the current processing object is in the exit state, the received new pending tasks can be returned to the proxy object, but the routing table in the proxy object may not be updated in time, and the proxy object resends the pending tasks to the current processing object based on the old routing table. In addition, it is also possible that during the scaling process, the routing table is updated multiple times, and when the routing table saved in the proxy object is not the latest routing table, the task may be returned after being forwarded multiple times.

[0093] In order to solve the above problem, in this embodiment, a return parameter can be added to the prompt information of the pending task with the return to indicate the number of times the current pending message has been returned. Then the proxy object can determine whether to continue forwarding the current task according to the number of returns indicated by the return parameter.

[0094] The following combination Figure 6 The task processing method in the scaling process is described in detail. Figure 6 As shown, in this embodiment, the above-mentioned proxy object can be specifically a proxy node, the current processing object can be the shrinking process S1, and the service process S2 can be other service processes in the above-mentioned distributed data processing system.

[0095] Specifically, before the distributed data processing system performs a scaling-down operation, that is, when each process is in a stable stage, the proxy node executes S602 to send a Key=1 task; the scaling-down process S1 executes S604 to execute the task;

[0096] It is understandable that, in each proxy node, tasks with the same identifier can be processed by consistent hash routing to the scaling-down process S1.

[0097] During the process of the distributed data processing system performing the scaling-down operation, it is determined that the process that needs to be scaled down is the scaling-down process S1, and the scaling-down process executes step S606 to send a first state identifier to the proxy node; when the proxy node executes S608 to send a Key=1 task to the scaling-down process S1, the scaling-down process S1 executes S610 to execute the old task and cache the newly received task;

[0098] It can be understood that in the above steps S606 to S610, the system can instruct the shrinking process S1 to prepare to perform the shrinking operation according to the predisable command (pre-exit command); and then in response to the above command, the shrinking process S1 switches to the pre-exit state and then enters the cache request stage. However, it is still in the normal state (i.e., the online state) to the outside. Therefore, the Key=1 task can continue to reference the shrinking process S1, that is, the proxy node continues to forward the Key=1 task to the above-mentioned shrinking process S1. When the shrinking process S1 is in the pre-exit state, S1 continues to process the current task and caches the newly received requests.

[0099] When the shrinking process S1 is not in the task execution state, the shrinking process S1 executes S612 and switches to the exit state; at the same time, S614 is executed to send a second state flag (disable flag, used to indicate that the shrinking process has exited) to the proxy node; when the proxy node executes S616 and sends the Key=1 task; the shrinking process S1 executes S618 to return the newly received task and the cached task; the proxy node executes S620 according to the updated routing table and forwards the Key=1 task;

[0100] It is understandable that when the scaling-down process S1 is not in the task execution state, it can immediately switch to the non-exit state and then enter the forwarding request stage. It should be noted that when the scaling-down process S1 enters the forwarding request stage, the disable state (exit state) is first sent to the proxy node to notify the proxy node to update the routing table. In addition, the newly received request information is bounced to the proxy node. The proxy node forwards it to other instances through the new routing table, thereby achieving the purpose of forwarding requests even when the routing information is not saved in the scaling-down process S1.

[0101] In addition, when the scaling-down process S1 enters the forwarding request phase, cached requests or tasks need to be forwarded, and newly received requests also need to be forwarded. Because there is still a time difference at this time, the requests or tasks in transit may still be to S1. In the above process, the scaling-down process S1 is only responsible for forwarding requests at this time, and does not execute them. Therefore, the consistency principle is still maintained.

[0102] Finally, when the scaling down operation is completed, the proxy node executes S622 according to the updated routing table and sends the Key=1 task to the service process S2. It can be understood that when the request or task is forwarded, it enters the stable stage. S1 can exit. At this time, the consistency principle is also maintained.

[0103] Through the above implementation of the present application, the cached tasks or requests are saved in the scaled-down instance, thereby avoiding the use of other system resources to save the above cached content, reducing the impact on the distributed system, and improving the operational efficiency during the scale-down process. In addition, during the scale-down process, the process used to execute the task does not need to save the routing table, and the routing table only needs to be saved in the proxy node, thereby saving the resources of the task process; and for the process or node object that does not need to perform the scale-down operation, there is no need to perform unnecessary operations, which improves the system operation efficiency; for the proxy node, there is no need to perform unnecessary operations on the routing table in the proxy node, and the above control operation is also imperceptible to the proxy node, and it is still processing the heartbeat packet normally, thereby avoiding the adjustment and control of the proxy node and improving the system control efficiency.

[0104] The following describes the task processing method during the expansion process.

[0105] In the case where the state switching instruction is an expansion instruction, when the current processing object meets the state switching condition, switching the object state of the current processing object from the reference object state to the second object state, and processing the cached target task includes:

[0106] S1, when the state switching instruction is an expansion instruction, switching the object state of the current processing object from the pre-online state to the online state, wherein the expansion instruction is used to switch the object state of the current processing object from the exit state to the online state, and the reference object state includes the pre-online state;

[0107] S2, processing the cached target task through the current processing object in the online state.

[0108] It is understandable that in the above-mentioned implementation mode of the present application, when the system is expanded, it is difficult to ensure that the routing tables stored in each proxy node are updated to be consistent at the same time, so there will be a situation where the task routing with the same identifier is distributed to two different task processing nodes or task processing processes. In the above-mentioned implementation mode of the present application, the simultaneous execution operation mode is not adopted, but the received processing request is cached by controlling the newly expanded task processing node, so as to achieve the delayed execution of the received processing request, thereby achieving the purpose of only one node executing the task with the same identifier at the same time.

[0109] In an optional embodiment, before switching the object state of the current processing object from the reference object state to the second object state when the current processing object satisfies the state switching condition, it also includes: when the current processing object is in a pre-online state and the duration of the received first target task is greater than or equal to a second threshold, determining that the current processing object satisfies the state switching condition.

[0110] In an optional embodiment, after switching the object state of the current processing object from the pre-online state to the online state, it also includes: sending a second state switching prompt information to the proxy object, wherein the second state switching prompt information is used to update the mapping relationship list saved in the proxy object, and the mapping relationship list is used to indicate the correspondence between the data processing tasks to be processed and the task processing objects.

[0111] In an optional implementation, after switching the object state of the current processing object from the pre-online state to the online state, the method further includes: when obtaining the target task forwarded by the proxy object, processing the target task by the current processing object in the online state.

[0112] The following combination Figure 7 The task processing method in the expansion process is described in detail. Figure 7 As shown, in this embodiment, the above-mentioned proxy object can be specifically a proxy node, the current processing object can be the expansion process S1, and the service process S2 is the service process in the above-mentioned distributed data processing system used to process the Key=1 task before the expansion operation.

[0113] Specifically, before the distributed data processing system performs the expansion operation, that is, when each process is in a stable stage, the proxy node executes S702 to send a Key=1 task to the service process S2; the service process S2 executes S704 to execute the task;

[0114] It is understandable that in the above steps, when each task process is in a stable stage, the service process S2 can be selected with the same task identifier, and the related tasks are normally executed in the service process S2, maintaining the consistency principle.

[0115] Next, in the process of the distributed data processing system performing the capacity expansion operation, the newly added capacity expansion process S1 corresponding to the Key=1 task can execute S706 to send the second state identifier to the proxy node; then, for the proxy node whose routing table has been updated, S708 can be executed to send the Key=1 task to the capacity expansion process S1, and the capacity expansion process S1 executes S710 to cache the received task; for the proxy node whose routing table has not been updated, S712 can be executed to send the Key=1 task to the service process S2 according to the old routing table, and the service process S2 executes S714 to execute the task;

[0116] It can be understood that in the above steps, during the expansion operation of the distributed data processing system, after the expansion process S1 is started, the Preenable command can be accepted to enter the pre-online state, and then enter the cache request stage; for the proxy node, after receiving the second state identifier (i.e., the state identifier indicating the online state), it can be considered that the expansion process S1 has been started, and then there will be Key=1 requests that will reach S1, and at this time S1 will cache the request and delay execution. For the proxy node whose routing table has not been updated, it will continue to forward the request to S2 according to the old routing table for continued execution. At this time, it is normal execution to maintain the consistency principle. In other words, at this time, only the service process S2 is processing the Key=1 request or task, thereby ensuring that there is no conflict in the read and write process.

[0117] Then, after reaching the preset cache duration, the expansion process S1 switches to the cache request processing stage, as in step S716, the proxy node sends the Key=1 task to the expansion process S1; S718, the expansion process S1 processes the received task and the cached task;

[0118] It is understandable that by setting the preset cache duration, it can be ensured that the task corresponding to Key=1 in S1 has been processed. Therefore, at this time, only the expansion process S1 is processing the request. This also maintains the principle of consistency.

[0119] Then the stable phase is executed normally. That is, as in steps S720 and S722, the proxy node forwards the Key=1 task to the expansion process S1; the expansion process S1 processes the received task.

[0120] The following is an explanation of the setting method of the above-mentioned preset cache time. In a specific method, the preset cache time can be determined as the maximum execution time of the task and the time of the maximum flight package. The time of the above-mentioned maximum flight package can be the maximum time consumed for forwarding the task through the proxy object. Specifically, when the current processing object is in the pre-online state, the timing can be started after receiving the first packet, and the cache request processing stage can be entered after 2s to process the request. Finally, it enters the stable stage.

[0121] In the above implementation, the cached tasks or requests are saved in the expansion instance, thereby avoiding the use of other system resources to save the above cached content, reducing the impact on the distributed system, and improving the operational efficiency during the expansion process. In addition, during the expansion process, the process used to execute the task does not need to save the routing table, and the routing table only needs to be saved in the proxy node, thereby saving the resources of the task process; and for the process or node object that does not need to perform the expansion operation, there is no need to perform unnecessary operations, which improves the system operation efficiency; for the proxy node, there is no need to perform unnecessary operations on the routing table in the proxy node, and the above control operation is also imperceptible to the proxy node, and it is still processing the heartbeat packet normally, thereby avoiding the adjustment and control of the proxy node and improving the system control efficiency.

[0122] The following combination Figure 8 The various object states shown illustrate an implementation of the above-mentioned expansion and contraction process.

[0123] like Figure 8 As shown, the task processing process may include four states.

[0124] Among them, during the expansion process, the newly added task processing process can be switched to the PreEnable pre-online state. When the task processing process is in the PreEnable pre-online state, it can receive task processing requests and cache the received requests. Then, it can start timing when the first packet is received. If the maximum tolerance time is exceeded, the task processing process will be switched to the Enable state (online state);

[0125] When the task processing process is in the Enable state, that is, the normal state, the received tasks or cached tasks are processed locally;

[0126] During the scaling-down process, for the task processing process that needs to be scaled down (i.e. offline), it can be switched to the PreDisable state (pre-exit state). When the task processing process is in the PreDisable state, the current task continues to be processed and the newly received requests are cached; when there are no historically received requests to be processed, the task processing process is switched to the Disable state (exit state);

[0127] When the task processing process is in the Disable state, requests for the cache are forwarded to the proxy node, and newly received new requests are forwarded to the proxy node.

[0128] It is understandable that there are two states in the process of scaling in or out. Through the two-phase commit state change, the message is cached at the receiving end and the execution is delayed. The process does not need to understand the routing table, and makes routing decisions based on the state, ultimately achieving lossless migration of requests.

[0129] The following combination Fig. 9 The task processing process of batch scaling down is described.

[0130] like Fig. 9 As shown, it is assumed that the current distributed data processing nodes include proxy nodes 901 and 902, and the processes used to process tasks include S1, S2, S3, S4, and S5. It is assumed that a scaling-down operation takes processes S3, S4, and S5 offline (exit), and the updates of the routing tables in proxy nodes 901 and 902 are inconsistent, such as Fig. 9 In which, after the routing table in the proxy node 901 is updated, it is indicated that S5 is offline, and after the routing table in the proxy node 902 is updated, it is indicated that S3, S4, and S5 are all offline.

[0131] When receiving the task identifier key of the task to be processed, the proxy node 901 can process the key through the hash algorithm. The process identifier obtained for the first time is S5. Since it has been recorded that S5 is offline, the proxy node 901 processes the key for the second time with the hash algorithm. Assuming that the obtained process identifier is S3, since the routing table is not updated in time, the task is forwarded to the process S3. After receiving the above task, the offline process S3 will directly return the task to the above proxy node 901, and then through the further forwarding operation of the proxy node 901, it is forwarded to the correct task processing process that is not offline.

[0132] When receiving the task identifier key of the task to be processed, the proxy node 902 can process the key through the hash algorithm. The process identifier obtained for the first time is S5. Since it has been recorded that S5 has gone offline, the proxy node 902 performs a second hash algorithm on the key. Assuming that the process identifier obtained is S3 and the routing table has been updated, the key is directly processed for the third time using the hash algorithm. Assuming that the process identifier obtained is S1, the task is forwarded to process S1 for processing.

[0133] It can be seen that in the above implementation, for a system that shrinks in batches, regardless of whether the routing table status in each proxy node is consistent, a task "bounce" will occur due to forwarding the task to the wrong node that has gone offline, and further routing operations are performed to find the correct node to execute the above task.

[0134] The following combination Fig.10 The task processing process of batch expansion is described.

[0135] like Fig.10 As shown, it is assumed that the current distributed data processing nodes include proxy nodes 1001 and 1002, and the processes used to process tasks include S1 and S2. Assume that a capacity expansion operation is to add processes S3, S4, and S5, and the updates of the routing tables in proxy nodes 1001 and 1002 are inconsistent, such as Fig.10 In which, the routing table in the proxy node 1001 is not updated, and only includes the identifiers S1 and S2 of the old process; while the routing table in the proxy node 1002 is updated, and only includes the identifiers S1 and S2 of the old process and the identifiers S3, S4, and S5 of the newly added process.

[0136] During the expansion process, processes S3, S4, and S5 are first started and set to the pre-online state (PreEnable);

[0137] For the above-mentioned proxy node 1001, since the routing table has not been updated, the relevant tasks are forwarded to the old processes S1 and S2 for processing, and processes S1 and S2 can execute directly after receiving the tasks; for the above-mentioned proxy node 1002, since the routing table has been updated, the relevant tasks can be forwarded to the new processes S3, S4, and S5. The new processes S3, S4, and S5 can cache the relevant requests after receiving the first task to be processed, and start executing the cached requests and tasks when the cache time is greater than the preset time length.

[0138] Through the above implementation methods of the present application, whether it is a capacity expansion process or a capacity reduction process, for the task processing node, the processing flow is unified, and it only needs to be judged according to the status; and the new state migration of the node only affects the process that needs to be operated, but does not affect other processes, and the impact range is small, ensuring the stability of the system; for the proxy node, the routing logic in the proxy node is not changed, only the general framework of the service node is modified, and the impact and influence on the distributed system are very small; in addition, the above implementation methods improve the disaster recovery capability, and the request migration process is unaware of the business end; through the above implementation methods, the operation and maintenance costs are reduced, and the operation and maintenance can take the machine off the shelf at any time, which improves the flexibility of the distributed data processing system.

[0139] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0140] According to another aspect of an embodiment of the present invention, a task processing device for implementing the above task processing method is also provided. Fig.11 As shown, the device comprises:

[0141] An acquiring unit 1102 is used to acquire a state switching instruction, wherein the state switching instruction is used to switch the object state of a current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in a distributed data processing system;

[0142] A first switching unit 1104, configured to switch the current processing object from the first object state to the reference object state in response to a state switching instruction;

[0143] A cache unit 1106, configured to cache the target task upon receiving the target task forwarded by the proxy object, wherein the proxy object is an object used for task forwarding in the distributed data processing system;

[0144] The second switching unit 1108 is used to switch the object state of the current processing object from the reference object state to the second object state when the current processing object meets the state switching condition, and process the cached target task.

[0145] Optionally, the second switching unit 1108 includes: a first switching module, used to switch the object state of the current processing object from a pre-exit state to an exit state when the state switching instruction is a shrinking instruction, wherein the shrinking instruction is used to switch the object state of the current processing object from an online state to an exit state, and the reference object state includes the pre-exit state; and a forwarding module, used to send the cached target task to the proxy object.

[0146] Optionally, the above-mentioned first switching module is also used to: obtain a reference task, wherein the above-mentioned reference task is a data processing task received by the above-mentioned current processing object in the above-mentioned online state; when the above-mentioned reference task is in an unfinished state, the above-mentioned reference task is processed by the above-mentioned current processing object in the above-mentioned pre-exit state.

[0147] Optionally, the first switching module is also used for at least one of the following: determining that the current processing object satisfies the state switching condition when the duration of the current processing object being in the pre-exit state is greater than or equal to a first threshold; determining that the current processing object satisfies the state switching condition when the reference task associated with the current processing object is in a completed state, wherein the reference task is a data processing task received by the current processing object in the online state.

[0148] Optionally, the above-mentioned first switching module is also used for at least one of the following: sending a first state switching prompt information to the above-mentioned proxy object, wherein the above-mentioned first state switching prompt information is used to update a mapping relationship list stored in the above-mentioned proxy object, and the above-mentioned mapping relationship list is used to indicate the correspondence between the data processing tasks to be processed and the task processing objects; when receiving the pending tasks forwarded by the above-mentioned proxy object, the above-mentioned pending tasks are sent to the above-mentioned proxy object.

[0149] Optionally, the above-mentioned first switching module is also used to: update the return parameter in the task return prompt information carrying the above-mentioned pending task, wherein the above-mentioned return parameter is used to indicate the cumulative number of times that the above-mentioned current processing object sends the above-mentioned pending task to the above-mentioned proxy object, and the above-mentioned proxy object is used to stop forwarding the above-mentioned pending task to the above-mentioned current processing object when the above-mentioned return parameter meets the target condition.

[0150] Optionally, the second switching unit 1108 includes: a second switching module, used to switch the object state of the current processing object from a pre-online state to an online state when the state switching instruction is an expansion instruction; a processing module, used to process the cached target task through the current processing object in the online state, wherein the expansion instruction is used to switch the object state of the current processing object from an exit state to an online state, and the reference object state includes the pre-online state.

[0151] Optionally, the second switching module is further used to determine that the current processing object satisfies the state switching condition when the current processing object is in the pre-online state and the duration of receiving the first target task is greater than or equal to a second threshold.

[0152] Optionally, the above-mentioned second switching module is also used to: send a second state switching prompt information to the above-mentioned proxy object, wherein the above-mentioned second state switching prompt information is used to update the mapping relationship list saved in the above-mentioned proxy object, and the above-mentioned mapping relationship list is used to indicate the correspondence between the data processing tasks to be processed and the task processing objects.

[0153] Optionally, the second switching module is further configured to: upon acquiring a target task forwarded by the proxy object, process the target task through the current processing object in the online state.

[0154] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned task processing method when running.

[0155] Optionally, in this embodiment, the embodiments to be implemented by the above-mentioned various unit modules can refer to the above-mentioned various method embodiments, which will not be repeated here.

[0156] According to another aspect of the embodiment of the present invention, an electronic device for implementing the above task processing method is also provided. The electronic device may be Fig.12 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a terminal device as an example. Fig.12 As shown, the electronic device includes a memory 1202 and a processor 1204. The memory 1202 stores a computer program, and the processor 1204 is configured to execute the steps in any of the above method embodiments through the computer program.

[0157] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0158] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0159] S1, obtaining a state switching instruction, wherein the state switching instruction is used to switch the object state of a current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in a distributed data processing system;

[0160] S2, in response to the state switching instruction, switching the current processing object from the first object state to the reference object state;

[0161] S3, caching the target task when receiving the target task forwarded by the proxy object, wherein the proxy object is an object used for task forwarding in the distributed data processing system;

[0162] S4, when the current processing object meets the state switching condition, the object state of the current processing object is switched from the reference object state to the second object state, and the cached target task is processed.

[0163] Alternatively, a person skilled in the art may understand that: Fig.12 The structure shown is for illustration only, and the electronic device may also be a vehicle-mounted terminal, a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Fig.12 The structure of the electronic device is not limited. Fig.12More or fewer components (such as network interfaces, etc.) as shown in, or with Fig.12 Different configurations are shown.

[0164] Among them, the memory 1202 can be used to store software programs and modules, such as the program instructions / modules corresponding to the task processing method and device in the embodiment of the present invention. The processor 1204 executes various functional applications and task processing by running the software programs and modules stored in the memory 1202, that is, realizing the above-mentioned task processing method. The memory 1202 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1202 may further include a memory remotely arranged relative to the processor 1204, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1202 can be specifically used, but not limited to, for storing file information such as observation data, motion state prediction results, etc. As an example, such as Fig.12 As shown, the memory 1202 may include but is not limited to the acquisition unit 1102, the first switching unit 1104, the cache unit 1106 and the second switching unit 1208 in the task processing device. In addition, it may also include but is not limited to other module units in the task processing device, which will not be repeated in this example.

[0165] Optionally, the transmission device 1206 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 1206 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1206 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0166] In addition, the electronic device further includes: a display 1208 for displaying the observation data and motion state prediction results; and a connection bus 1210 for connecting the various module components in the electronic device.

[0167] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0168] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the above methods of various embodiments of the present invention.

[0169] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0170] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program having a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0171] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0172] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0173] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0174] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A task processing method, It is characterized in that include: Acquire a state switching instruction, wherein the state switching instruction is used to switch the object state of a current processing object from a first object state to a second object state, and the current processing object is an object used for task processing in a distributed data processing system; In response to the state switching instruction, switching the currently processed object from the first object state to a reference object state; In case of receiving a target task forwarded by a proxy object, caching the target task, wherein the proxy object is an object used for task forwarding in the distributed data processing system; When the current processing object satisfies the state switching condition, the object state of the current processing object is switched from the reference object state to the second object state, and the cached target task is processed.

2. The method according to claim 1, It is characterized in that When the current processing object satisfies the state switching condition, switching the object state of the current processing object from the reference object state to the second object state, and processing the cached target task includes: In the case where the state switching instruction is a shrinking instruction, switching the object state of the current processing object from a pre-exit state to an exit state, wherein the shrinking instruction is used to switch the object state of the current processing object from an online state to an exit state, and the reference object state includes the pre-exit state; The cached target task is sent to the proxy object.

3. The method according to claim 2, It is characterized in that After the current processing object is switched from the first object state to the reference object state in response to the state switching instruction, the method further includes: Acquire a reference task, wherein the reference task is a data processing task received by the current processing object in the online state; In a case where the reference task is in an unfinished state, the reference task is processed by the current processing object in the pre-exit state.

4. The method according to claim 2, It is characterized in that Before switching the object state of the currently processed object from the pre-exit state to the exit state, at least one of the following is further included: In a case where the duration of the current processing object being in the pre-exit state is greater than or equal to a first threshold, determining that the current processing object satisfies the state switching condition; In a case where a reference task associated with the current processing object is in a completed state, it is determined that the current processing object meets a state switching condition, wherein the reference task is a data processing task received by the current processing object in the online state.

5. The method according to claim 2, It is characterized in that After the object state of the currently processed object is switched from the pre-exit state to the exit state, at least one of the following is further included: Sending first state switching prompt information to the proxy object, wherein the first state switching prompt information is used to update a mapping relationship list stored in the proxy object, and the mapping relationship list is used to indicate a corresponding relationship between a data processing task to be processed and a task processing object; In case of receiving the pending task forwarded by the proxy object, the pending task is sent to the proxy object.

6. The method according to claim 5, It is characterized in that In the case of receiving a pending task forwarded by the proxy object, before sending the pending task to the proxy object, the method further includes: In the task return prompt information carrying the pending task, the return parameter is updated, wherein the return parameter is used to indicate the cumulative number of times the current processing object sends the pending task to the proxy object, and the proxy object is used to stop forwarding the pending task to the current processing object when the return parameter meets the target condition.

7. The method according to claim 1, It is characterized in that When the current processing object satisfies the state switching condition, switching the object state of the current processing object from the reference object state to the second object state, and processing the cached target task includes: In the case where the state switching instruction is an expansion instruction, the object state of the currently processed object is switched from a pre-online state to an online state, wherein the expansion instruction is used to switch the object state of the currently processed object from an exit state to an online state, and the reference object state includes the pre-online state; The cached target task is processed by the current processing object in the online state.

8. The method according to claim 7, It is characterized in that In the case where the currently processed object meets the state switching condition, before switching the object state of the currently processed object from the reference object state to the second object state, the method further includes: When the current processing object is in the pre-online state and the duration of the first received target task is greater than or equal to a second threshold, it is determined that the current processing object meets the state switching condition.

9. The method according to claim 7, It is characterized in that After the object state of the currently processed object is switched from the pre-online state to the online state, the method further includes: Sending second state switching prompt information to the proxy object, wherein the second state switching prompt information is used to update a mapping relationship list stored in the proxy object, and the mapping relationship list is used to indicate the correspondence between the data processing tasks to be processed and the task processing objects.

10. The method according to claim 7, It is characterized in that After the object state of the currently processed object is switched from the pre-online state to the online state, the method further includes: When the target task forwarded by the proxy object is obtained, the target task is processed by the current processing object in the online state.

11. A task processing device, It is characterized in that include: an acquisition unit, configured to acquire a state switching instruction, wherein the state switching instruction is used to switch an object state of a current processing object from a first object state to a second object state, wherein the current processing object is an object used for task processing in a distributed data processing system; A first switching unit, configured to switch the current processing object from the first object state to a reference object state in response to the state switching instruction; a cache unit, configured to cache the target task upon receiving the target task forwarded by the proxy object, wherein the proxy object is an object used for task forwarding in the distributed data processing system; The second switching unit is used to switch the object state of the current processing object from the reference object state to the second object state when the current processing object meets the state switching condition, and process the cached target task.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 10 when executed.

13. A computer program product comprising a computer program / instructions, It is characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. An electronic device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 10 through the computer program.

Images