Distributed data processing method and related device

Abstract

The embodiment of the present application discloses a distributed data processing method and related devices, wherein the method includes: obtaining a consumption message and a unique identifier corresponding to the consumption message from a first server; determining that the consumption message is a consumption message to be executed that meets the execution conditions; obtaining a distributed lock corresponding to the consumption message to be executed from a second server, wherein the unique identifier corresponding to the consumption message is not recorded by the second server; executing the business process corresponding to the consumption message to be executed according to the obtained distributed lock; and notifying the first server to modify the execution status of the consumption message to be executed according to the execution result of the business process. The embodiment of the present application determines that the consumption message meets the execution conditions, obtains the distributed lock corresponding to the consumption message according to the unique identifier of the consumption message, and completes the execution of the business process. Congestion and duplication of the distributed lock acquisition process are avoided, the execution efficiency of the business process is improved, repeated consumption of messages is prevented, and the accuracy of platform data is improved.

Description

Technical Field

[0001] The present application relates to the field of distributed processing technology, and in particular to a distributed data processing method and related devices. Background Art

[0002] As a key component for achieving the scalability and extensibility of distributed systems, distributed messaging systems need to have the characteristics of high throughput and high availability. Distributed locks are a way to control the synchronous access to shared resources between distributed systems. In a distributed messaging system, multiple messages will be generated due to multiple executions, multiple business executions, or message interactions. When consuming messages, repeated consumption of messages will also occur due to network jitter, cyclic consumption, or other reasons. Repeated consumption of messages may cause dirty data in the system and even cause loss of time and money for the business party. Distributed locks can be used to achieve mutually exclusive access to different messages, but how to allocate distributed locks in a reasonable way and use distributed locks to prevent repeated consumption of messages is a problem worth exploring. Summary of the invention

[0003] The embodiment of the present application provides a distributed data processing method and related devices, so that when it is determined that the consumption message meets the execution conditions, the distributed lock corresponding to the consumption message is obtained according to the unique identifier of the consumption message and the execution of the business process is completed. Congestion and duplication of the distributed lock acquisition process are avoided, the execution efficiency of the business process is improved, repeated consumption of messages is prevented, and the accuracy of platform data is improved.

[0004] In a first aspect, an embodiment of the present application provides a distributed data processing method, the method comprising:

[0005] Acquire a consumption message and a unique identifier corresponding to the consumption message from the first server;

[0006] Determining that the consumption message is a to-be-executed consumption message that meets an execution condition;

[0007] Acquire, from the second server, a distributed lock corresponding to the to-be-executed consumption message, wherein the unique identifier corresponding to the to-be-executed consumption message is not recorded by the second server;

[0008] Execute the business process corresponding to the to-be-executed consumption message according to the obtained distributed lock;

[0009] The first server is notified to modify the execution status of the to-be-executed consumption message according to the execution result of the business process.

[0010] In an optional example, determining that the consumption message is a to-be-executed consumption message that satisfies an execution condition includes:

[0011] Obtaining the execution time corresponding to the consumption message, wherein the execution time includes determining according to a timer or determining according to the duration of generating the consumption message;

[0012] When the time from the execution time is less than a first preset time threshold, determining that the consumption message is a consumption message that reaches the execution time;

[0013] Determine the consumption message that reaches the execution time as the to-be-executed consumption message that meets the execution condition.

[0014] In an optional example, after determining that the consumption message is a consumption message that reaches the execution time, the method further includes:

[0015] Determine the number of consumption messages that arrive at the execution time;

[0016] When the number of consumption messages arriving at the execution time is greater than a preset number, obtaining multiple waiting times of the multiple consumption messages arriving at the execution time;

[0017] Sort the messages according to the waiting time, and determine the consumption message corresponding to the waiting time at the top of the sort as the highest priority;

[0018] Get the consumption message with the highest priority as the consumption message to be executed.

[0019] In an optional example, before obtaining the distributed lock corresponding to the to-be-executed consumption message from the second server, the method further includes:

[0020] Obtaining a service ID corresponding to the unique identifier;

[0021] Traversing the simulation execution table in the simulation execution system according to the business ID;

[0022] If it is determined that the simulation execution table does not include the service ID, obtaining a consumption message corresponding to the service ID, and simulating the execution of the service process corresponding to the consumption message;

[0023] If the execution result of the business process is execution failure, the execution time corresponding to the consumption message is postponed, and a business ID corresponding to the unique identifier of the consumption message is regenerated;

[0024] Diagnose and modify the business logic corresponding to the consumption message, and repeat the simulation execution process.

[0025] In an optional example, the method further includes:

[0026] The service ID corresponding to the consumption message of the successful simulation execution is obtained, and a simulation execution table is generated according to the service ID.

[0027] In an optional example, notifying the first server to modify the execution state of the consumption message according to the execution result of the business process includes:

[0028] If the execution result of the business process is execution failure, sending a first prompt message to the first server to prompt the first server to generate a new unique identifier corresponding to the consumption message;

[0029] If the execution result of the business process is successful, a second prompt message is sent to the first server to prompt the first server to mark the execution status of the consumption message as executed.

[0030] In an optional example, obtaining the consumption message and the unique identifier corresponding to the consumption message from the first server includes:

[0031] A consumption message with an execution status of unexecuted and a unique identifier corresponding to the consumption message are obtained from the first server.

[0032] In a second aspect, the present application provides a service execution device, the device comprising:

[0033] A first acquisition unit, configured to acquire a consumption message and a unique identifier corresponding to the consumption message from a first server, and determine that the consumption message is a to-be-executed consumption message that meets an execution condition;

[0034] A second acquisition unit acquires the distributed lock corresponding to the to-be-executed message from a second server, wherein the unique identifier corresponding to the to-be-executed consumption message is not recorded by the second server;

[0035] An execution unit, used to execute the business process corresponding to the to-be-executed consumption message according to the acquired distributed lock;

[0036] A notification unit is used to notify the first server to modify the execution status of the to-be-executed consumption message according to the execution result of the business process.

[0037] In an optional example, the first acquiring unit is specifically configured to:

[0038] Obtaining the execution time corresponding to the consumption message, wherein the execution time includes determining according to a timer or determining according to the duration of generating the consumption message;

[0039] When the time from the execution time is less than a first preset time threshold, determining that the consumption message is a consumption message that reaches the execution time;

[0040] Determine the consumption message that reaches the execution time as the to-be-executed consumption message that meets the execution condition.

[0041] In an optional example, the first acquiring unit is further configured to:

[0042] Determine the number of consumption messages that arrive at the execution time;

[0043] When the number of consumption messages arriving at the execution time is greater than a preset number, obtaining multiple waiting times of the multiple consumption messages arriving at the execution time;

[0044] The message consumption corresponding to the maximum waiting time is obtained as the consumption message to be executed.

[0045] In an optional example, the first acquiring unit is further configured to:

[0046] Obtaining a service ID corresponding to the unique identifier;

[0047] Traversing the simulation execution table in the simulation execution system according to the business ID;

[0048] If it is determined that the simulation execution table does not include the service ID, obtaining a consumption message corresponding to the service ID, and simulating the execution of the service process corresponding to the consumption message;

[0049] If the execution result of the business process is execution failure, the execution time corresponding to the consumption message is postponed, and a business ID corresponding to the unique identifier of the consumption message is regenerated;

[0050] Diagnose and modify the business logic corresponding to the consumption message, and repeat the simulation execution process.

[0051] In an optional example, the first acquiring unit is further configured to:

[0052] The service ID corresponding to the consumption message of the successful simulation execution is obtained, and a simulation execution table is generated according to the service ID.

[0053] In an optional example, the notification unit is specifically configured to:

[0054] If the execution result of the business process is execution failure, sending a first prompt message to the first server to prompt the first server to generate a new unique identifier corresponding to the consumption message;

[0055] If the execution result of the business process is successful, a second prompt message is sent to the first server to prompt the first server to mark the execution status of the consumption message as executed.

[0056] In an optional example, the first acquiring unit is specifically configured to:

[0057] A consumption message with an execution status of unexecuted and a unique identifier corresponding to the consumption message are obtained from the first server.

[0058] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program comprises instructions for executing the steps of any method of the first aspect.

[0059] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute instructions of the steps described in any method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The following is a brief introduction to the drawings involved in the embodiments of the present application.

[0061] Figure 1A A schematic diagram of a distributed message consumption scenario provided in an embodiment of the present application;

[0062] Figure 1B A schematic diagram of a distributed data processing method provided in an embodiment of the present application;

[0063] Figure 1C A schematic diagram of a lottery platform architecture provided in an embodiment of the present application;

[0064] Figure 1D A schematic diagram of a simulated execution business process provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of another distributed data processing method provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of another distributed data processing method provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of another distributed data processing method provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0069] Figure 6 This is a block diagram of the functional units of the service execution device involved in the embodiments of the present application. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0071] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0072] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0073] In order to more conveniently understand the embodiment of the present solution, the implementation scenario of the present solution is first described.

[0074] See also Figure 1A , Figure 1A A schematic diagram of a distributed message consumption scenario provided in an embodiment of the present application is shown in FIG. Figure 1A As shown, the consumer end 10 is connected to the first server 20 through a network, and is connected to the second server 30 through a network or structure, wherein the consumer end refers to the terminal used by the user, and each terminal includes an application server for exposing the business logic corresponding to the application. The application at the consumer end needs to obtain the consumption message from the first server 20 for consumption or execution. The first server 20 is a distributed message system or a load balancing server in a distributed message system. The distributed message system includes multiple terminals or servers, and the terminal includes an agent program to form a proxy server. After the application at the consumer end obtains the consumption message from the first server 20, it needs to obtain a distributed lock from the second server 30 in order to obtain the access resource corresponding to the consumption message. The second server 30 is a distributed lock server, or a distributed database redis.

[0075] The embodiments of the present application are described in detail below.

[0076] See also Figure 1B , Figure 1B is a schematic diagram of a distributed data processing method provided in an embodiment of the present application, such as Figure 1B As shown, the method comprises the following steps:

[0077] 101. The consumer terminal obtains a consumption message and a unique identifier corresponding to the consumption message from a first server.

[0078] The consumer end obtains the consumption message from the first server in order to consume the consumption message, that is, to complete the execution of the business process corresponding to the consumption message. The consumption message may be a message issued by any proxy server in the distributed system. When obtaining the consumption message, the unique identifier corresponding to the consumption message must also be obtained. The unique identifier is used to mark the uniqueness of the consumption message. In order to avoid repeated consumption of the consumption message due to network jitter, cyclic consumption or other reasons, the distributed lock server only issues a distributed lock once for each consumption message. The consumption message that obtains the distributed lock executes the business process. If the execution fails, the distributed lock will not be obtained again.

[0079] For example, suppose Figure 1A The application in the consumer end 10 is a lottery platform, please refer to Figure 1C , Figure 1C A schematic diagram of a lottery platform architecture provided in an embodiment of the present application is shown in FIG. Figure 1C As shown, the business processes that the lottery platform can carry out include user registration, lottery activities, information sharing, community communication, consultation browsing and lottery reservation. For these business processes, the data layer includes user information data, lottery data, community data and user browsing logs. The lottery platform sends a request message to the distributed messaging system to obtain consumption messages. Here, the request message can be a lottery request. Then the consumption message that the distributed messaging system can respond to the lottery request includes a successful lottery and a reward, or a lottery failure. The unique identifiers corresponding to these two consumption messages are different, which are used to uniquely mark the consumption message.

[0080] Optionally, the consumer terminal sends a request message to the first server to obtain the consumption message and the unique identifier corresponding to the consumption message, and the unique identifier can be determined according to the consumer terminal hardware parameters and the request message related parameters. The hardware parameters include the name, model, shape, color, function, physical address (MAC Address), central processing unit (CPU, Central Processing Unit) model, etc. of the device hardware. The request message related parameters include the time when the request message is generated, the service type or service type number, etc. For example, in the lottery platform, the six services of user registration, lottery activities, information sharing, community communication, consultation and browsing, and lottery reservation are numbered from 1 to 6. The timestamp corresponding to the time when the message request for the lottery activity occurs is 1574930871, then the unique identifier generated by the service type number + timestamp is 2-1574930871, and the consumer end MAC address is B0-25-AA-21-75-5E. The unique identifier generated according to the service type number + timestamp + MAC address is: 2-1574930871-(B0-25-AA-21-75-5E), so that the consumption messages corresponding to different services generated by different consumer ends at different times can be distinguished by the unique identifier. The above hardware parameters and service types or service type numbers can also have other variations, which will not be described in detail here.

[0081] 102. The consumer determines that the consumption message is a to-be-executed consumption message that meets an execution condition.

[0082] Because there may be multiple consumer ends that need to obtain distributed locks, or there may be multiple business processes on the same consumer end that need to execute consumer messages, in order to fairly distribute the opportunities for consumers to obtain distributed locks, only consumer messages that meet the execution conditions can obtain distributed locks. Meeting the execution conditions can include reaching the execution time, meeting the execution priority, having an expedited execution flag, etc.

[0083] Optionally, determining that a consumption message is a consumption message to be executed that meets the execution conditions includes: obtaining the execution time corresponding to the consumption message, the execution time includes determining according to a timer or determining according to the duration of generating the consumption message; when the difference between the execution time and the current time is less than a first preset time threshold, determining that the consumption message is a consumption message that has reached the execution time; determining that the consumption message that has reached the execution time is a consumption message to be executed that meets the execution conditions.

[0084] When the first server generates a consumption message, or after the consumption message is obtained by the consumer end, a timer can be generated for the consumption message to count down, and the execution time is reached when the countdown ends; or the generation time of the consumption message is marked with a timestamp, and the execution time is determined to have arrived when the generation time reaches a certain length of time; the consumption message that has arrived at the execution time can be determined as a consumption message to be executed that meets the execution condition. Alternatively, before the execution time is reached, that is, when the execution time is less than the first preset time threshold, the consumption message can be determined as a consumption message to be executed that meets the execution condition.

[0085] When acquiring a distributed lock on the consumer side, there may be a situation where the consumption messages of multiple consumer sides meet the execution time. Under this premise, the allocation strategy of the distributed lock can be determined according to the priority of the consumption message. The priority of the consumption message can be determined based on factors such as whether the consumption message includes a priority identifier, the business logic sorting of the consumption message, and the waiting time of the consumption message.

[0086] Optionally, after determining that the consumption message is a consumption message that has arrived at the execution time, the method also includes: determining the number of consumption messages that have arrived at the execution time; when the number of consumption messages that have arrived at the execution time is greater than a preset number, obtaining multiple waiting times for multiple consumption messages that have arrived at the execution time; and obtaining the message consumption corresponding to the maximum waiting time as the consumption message to be executed.

[0087] Specifically, first detect the number of consumer messages that have reached the execution time. If it is greater than the preset number, you need to set a priority for them. Specifically, get the waiting time corresponding to each consumer message, and sort them in order of the length of the waiting time. The consumer message with the longest waiting time is sorted first, and its corresponding priority is the highest priority; determine the consumer message with the highest priority as the consumer message to be executed. Or, if there is only one consumer message waiting, it may be that the business process corresponding to this consumer message has not yet opened access to resources, such as Figure 1C In the corresponding lottery reservation business process, if the lottery time has not arrived, the access resources corresponding to the lottery activity need to wait. When the first waiting time of the consumption message is greater than the preset threshold, the consumption message can be preferentially allocated a distributed lock to execute the subsequent business process.

[0088] It can be seen that in the embodiment of the present application, by determining that the consumption message reaches the execution time and the consumption message has the highest priority, it is determined that the consumption message meets the execution condition, so that multiple consumption messages that reach the execution time can further clarify the order of acquiring distributed locks according to the priority order, thereby improving the efficiency of acquiring distributed locks.

[0089] Optionally, before obtaining the distributed lock corresponding to the consumption message to be executed from the second server, the method also includes: obtaining the business ID corresponding to the unique identifier; traversing the simulation execution table in the simulation execution system according to the business ID; if it is determined that the simulation execution table does not include the business ID, obtaining the consumption message corresponding to the business ID, and simulating the execution of the business process corresponding to the consumption message; if the execution of the business process fails, postponing the execution time corresponding to the consumption message, and regenerating the business ID corresponding to the unique identifier of the consumption message; diagnosing and modifying the business logic corresponding to the consumption message, and repeating the simulation execution process.

[0090] Because each consumer message corresponding to a unique identifier will only generate a distributed lock once, that is, it will only allocate access resources once. In order to increase the probability of the smooth execution of the business process corresponding to the consumer message, you can simulate the execution of the business process before formally executing the business process corresponding to the consumer message. If the simulation fails, the execution time corresponding to the consumer message will be extended. In this case, the consumer message cannot reach the execution time, and the distributed lock cannot be obtained, and the formal execution process of the business process corresponding to the consumer message cannot be carried out.

[0091] See also Figure 1D , Figure 1D A schematic diagram of a simulated execution business process provided in an embodiment of the present application, such as Figure 1D As shown, each unique identifier generates a business ID, and then the simulation execution table in the simulation execution system is traversed according to the business ID to determine whether the business ID is included in the simulation execution table. If it is included, it means that the business process of the consumption message corresponding to the business ID has been simulated and executed; if it is not included, it means that the business process of the consumption message corresponding to the business ID has not been simulated and executed, and it can be simulated and executed. If the execution is successful, it means that there is no logical error in the business process and it can be executed smoothly. Then, the distributed lock can be waited according to the execution time of the consumption message corresponding to the business process to carry out the subsequent execution process. If the execution is unsuccessful, it means that there is a logical error in the business process and it cannot be executed smoothly. The execution time of the consumption message corresponding to the business process is postponed. For example, the consumption message can be executed after t1 time, and now it is postponed for t2 time. Then, the consumption message needs to be executed after t1+t2. At the same time, the logical error of the business process of the consumption message is diagnosed and modified. The unique identifier corresponding to the modified consumption message remains unchanged, but a new business ID needs to be generated, and then matched with the simulation execution table according to the new business ID for the subsequent simulation execution process. Until it is determined that the simulation execution of the business process is successful and the execution time of the consumer message corresponding to the business process is no longer delayed, the consumer message can successfully obtain the distributed lock and perform subsequent business process execution.

[0092] Optionally, traversing the simulation execution table in the simulation execution system according to the service ID includes: traversing the simulation execution table in reverse order according to the service ID.

[0093] If the simulated execution of the consumer message business process corresponding to a business ID fails, a new business ID will be generated for the consumer message. Under normal circumstances, as long as the simulated consumer message has not been simulated before, the corresponding business ID will not be matched in the simulated execution table. One case where the business ID is matched is that the generation of a new business ID for the failed consumer message fails, and the consumer message that has been simulated is simulated again with the original business ID. In this case, the business ID can be matched to the original business ID in the simulated execution table as soon as possible during the reverse traversal, thereby improving the matching efficiency. Furthermore, when traversing the simulated execution table in reverse order according to the business ID, the number of traversals or the traversal percentage is set, for example, traversing 10, or traversing 1 / 3 of the business IDs in the table. If no match is found, it is determined that the business ID is not included in the simulated execution table.

[0094] Optionally, the method further includes: obtaining a business ID corresponding to a consumption message of successful simulation execution, and generating a simulation execution table according to the business ID.

[0095] The simulation execution table includes the business IDs corresponding to all consumer messages that have been successfully simulated, because only consumer messages that have failed to execute will regenerate business IDs. Therefore, the business IDs corresponding to each consumer message that has been successfully simulated can be found in the simulation execution table, while the business IDs of consumer messages that have not been successfully executed, at least the latest business IDs, cannot be found in the simulation execution table. The consumer messages that have not been successfully executed can be simulated again. In addition, when the initial consumer message is simulated, the simulation execution table is blank and does not need to match the business ID. In some cases, after executing a complete business logic, the original simulation execution table can be cleared and a new simulation execution table can be generated, which can effectively improve the efficiency of the simulation execution process.

[0096] It can be seen that in the embodiment of the present application, the execution condition is set to reach the execution time. Before the consumer message obtains the distributed lock, the business process corresponding to the consumer message is simulated and debugged. If there is no logical error in the business process, the consumer message can reach the execution time normally and obtain the distributed lock. Otherwise, the consumer message cannot reach the execution time and cannot obtain the distributed lock. This process avoids the situation where the business process corresponding to the consumer message cannot be completed because the distributed lock is only allocated once for the consumer message, thereby improving the efficiency of business process execution.

[0097] 103. The consumer end obtains a distributed lock corresponding to the to-be-executed consumption message from the second server, wherein the unique identifier corresponding to the to-be-executed consumption message is not recorded by the second server.

[0098] After the consumer obtains the consumption message and the unique identifier corresponding to the consumption message and determines that it is a consumption message to be executed, it needs to obtain the access resource of the consumption message. In order to prevent congestion caused by multiple consumption messages obtaining access resources at the same time, it is first necessary to obtain the distributed lock corresponding to the consumption message. Only the consumption message that obtains the distributed lock can obtain access resources and execute subsequent business processes.

[0099] When obtaining the distributed lock corresponding to the consumer message, it is necessary to make sure that the consumer message has not obtained the distributed lock in history, that is, the same consumer message can only obtain the distributed lock once. Because the unique identifier is used to distinguish different consumer messages, the unique identifier of the consumer message can be matched with other unique identifiers that have obtained the distributed lock. If the match fails, it indicates that the consumer message corresponding to the unique identifier has not obtained the distributed lock, and the distributed lock can be allocated to the consumer message corresponding to the unique identifier.

[0100] 104. The consumer end executes the business process corresponding to the to-be-executed consumption message according to the obtained distributed lock;

[0101] 105. The consumer terminal notifies the first server to modify the execution status of the to-be-executed consumption message according to the execution result of the business process.

[0102] The business process corresponding to the consumption message of the obtained distributed lock can be executed normally. If the execution is successful, the business process is completed, and the second server deletes the corresponding distributed lock and records the unique identifier. If the execution fails, similarly, the second server deletes the corresponding distributed lock and records the unique identifier. That is, the unique identifiers corresponding to all executed business processes will be recorded, and the corresponding distributed locks will be deleted. The same unique identifier has only one chance to obtain a distributed lock, and only one chance to access resources.

[0103] After the execution of the business process is completed, the first server (distributed messaging system) needs to be notified of the execution result of the consumption message so that the first server can record and determine whether to send the consumption message to the consumer end later. Figure 1C In the lottery activity, if the consumer end A obtains the consumption message for the first time that the lottery is successful and the reward is issued, after the consumer end obtains the distributed lock from the distributed lock server and executes the business process, the reward is not issued successfully. This result needs to be fed back to the first server so that the first server can determine whether to issue a reward to the consumer end A again.

[0104] Optionally, the first server is notified to modify the execution status of the consumption message according to the execution result of the business process, including: if the execution result of the business process is execution failure, a first prompt message is sent to the first server to prompt the first server to generate a new unique identifier corresponding to the consumption message; if the execution result of the business process is execution success, a second prompt message is sent to the first server to prompt the first server to mark the execution status of the consumption message as executed.

[0105] Optionally, obtaining a consumption message and a unique identifier corresponding to the consumption message from the first server includes: obtaining a consumption message whose execution status is unexecuted and a unique identifier corresponding to the consumption message from the first server.

[0106] If the business process corresponding to the consumption message fails to execute, the consumer can prompt the first server to generate a new unique identifier for the consumption message, so that when the consumer initiates a request again, the first server will issue the original consumption message again, and the consumption message can also obtain a distributed lock based on the new unique identifier, and then execute the subsequent business process. If the business process corresponding to the consumption message is successfully executed, the consumer can also prompt the first server, and the first server will mark the execution status corresponding to the consumption message as executed, so that when the consumer initiates a request again, the first server can issue the executed consumption message. Alternatively, the consumer obtains the executed consumption message and automatically discards it without obtaining a distributed lock.

[0107] It can be seen that in the embodiment of the present application, firstly, the consumption message and the unique identifier corresponding to the consumption message are obtained from the first server; then, the distributed lock corresponding to the consumption message that meets the execution condition is obtained from the second server, wherein the unique identifier corresponding to the consumption message is not recorded by the second server; and the business process corresponding to the consumption message is executed according to the obtained distributed lock; finally, the first server is notified to modify the execution status of the consumption message according to the execution result of the business process. In this process, by setting the execution condition and determining that the distributed lock can be obtained only when the consumption message meets the execution condition, the pertinence and efficiency of obtaining the distributed lock are improved, and the distributed lock is allocated to the consumption message according to the unique identifier, which can effectively prevent the repeated execution of the consumption message, reduce the dirty data of the system, and improve the correctness of the system data.

[0108] See also Figure 2 , Figure 2 is a flow chart of another distributed data processing method provided by an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0109] 201. The consumer terminal obtains a consumption message and a unique identifier corresponding to the consumption message from the first server, and obtains a service ID corresponding to the unique identifier;

[0110] 202. The consumer end traverses the simulation execution table in the simulation execution system according to the service ID;

[0111] 203. If it is determined that the simulation execution table does not include the service ID, the consumer end obtains a consumption message corresponding to the service ID, and simulates and executes the service process corresponding to the consumption message;

[0112] 204. If the execution result of the business process is execution failure, the execution time corresponding to the consumption message is postponed;

[0113] 205. The consumer terminal diagnoses and modifies the business logic corresponding to the consumption message, and repeats the simulation execution process;

[0114] 206. Determine that the consumption message that has reached the execution time is a consumption message to be executed, and the consumer end obtains a distributed lock corresponding to the consumption message to be executed from the second server, wherein the unique identifier corresponding to the consumption message is not recorded by the second server;

[0115] 207. The consumer end executes the business process corresponding to the to-be-executed consumer message according to the obtained distributed lock, and notifies the first server to modify the execution status of the to-be-executed consumer message according to the execution result of the business process.

[0116] After the consumer end obtains the consumption message and its corresponding unique identifier from the first server, it generates a business ID corresponding to the unique identifier locally, and then simulates the execution of the consumption message according to the business ID. If the simulation is successful, the execution time of the consumption message will not be postponed. Otherwise, the execution time of the consumption message will be postponed, and the consumption message will be diagnosed and modified, and simulated again until it is determined that the simulation of the consumption message is successful. This can effectively avoid the situation where the business process corresponding to the consumption message cannot be completed because the distributed lock is only allocated once for the consumption message, thereby improving the efficiency of business process execution. For other descriptions, please refer to the corresponding description of the distributed data processing method described in steps 101-104, which will not be repeated here.

[0117] See also Figure 3 , Figure 3 is a flow chart of another distributed data processing method provided by an embodiment of the present application, such as Figure 3 As shown, the method comprises the following steps:

[0118] 301. The consumer terminal obtains a consumption message and a unique identifier corresponding to the consumption message from the first server, and obtains a service ID corresponding to the unique identifier;

[0119] 302. The consumer end traverses the simulation execution table in the simulation execution system according to the service ID;

[0120] 303. If it is determined that the simulation execution table does not include the service ID, the consumer end obtains a consumption message corresponding to the service ID, and simulates and executes the service process corresponding to the consumption message;

[0121] 304. If the execution result of the business process is execution failure, the execution time corresponding to the consumption message is postponed;

[0122] 305. The consumer terminal diagnoses and modifies the business logic corresponding to the consumption message, and repeats the simulation execution process;

[0123] 306. Determine the number of consumption messages that arrive at the execution time;

[0124] 307. When the number of the consumption messages arriving at the execution time is greater than the preset number, multiple waiting times of the multiple consumption messages arriving at the execution time are obtained;

[0125] 308. Sort the messages according to the waiting time, and determine the consumption message corresponding to the waiting time at the top of the sort as the highest priority;

[0126] 309. Obtain the highest priority consumption message as the consumption message to be executed;

[0127] 310. The consumer end obtains a distributed lock corresponding to the to-be-executed consumption message from the second server, wherein the unique identifier corresponding to the to-be-executed consumption message is not recorded by the second server;

[0128] 311. The consumer end executes the business process corresponding to the to-be-executed consumer message according to the obtained distributed lock, and notifies the first server to modify the execution status of the to-be-executed consumer message according to the execution result of the business process.

[0129] After the consumer end obtains the consumption message and its corresponding unique identifier from the first server, it generates a business ID locally corresponding to the business ID and the unique identifier, and then simulates the execution of the consumption message according to the business ID, and determines whether to postpone the execution time of the consumption message according to the simulated consumption result; after determining that the consumption message arrives at the execution time, the priority of the consumption message is determined according to the waiting time of the consumption message. If the consumption message has the highest priority, it is determined that the consumption message meets the execution conditions, and the distributed lock corresponding to the consumption message is obtained and the subsequent business process is executed. In this process, by setting the execution time to be reached and the highest priority to determine whether the consumption message meets the execution conditions, the efficiency of obtaining distributed locks is improved. For other descriptions, please refer to the corresponding description of the distributed data processing method described in steps 101-104, which will not be repeated here.

[0130] See also Figure 4 , Figure 4 is a flow chart of another distributed data processing method provided by an embodiment of the present application, such as Figure 4 As shown, the method comprises the following steps:

[0131] 401. The consumer terminal obtains a consumption message whose execution status is unexecuted and a unique identifier corresponding to the consumption message from the first server;

[0132] 402. The consumer end obtains a distributed lock corresponding to the consumption message that meets the execution condition from the second server, wherein the unique identifier corresponding to the consumption message is not recorded by the second server;

[0133] 403. The consumer executes the business process corresponding to the consumption message according to the obtained distributed lock;

[0134] 404. If the execution result of the business process is execution failure, the consumer terminal sends a first prompt message to the first server to prompt the first server to generate a new unique identifier corresponding to the consumption message;

[0135] 405. If the execution result of the business process is successful, the consumer end sends a second prompt message to the first server to prompt the first server to mark the execution status of the consumption message as executed.

[0136] The consumer end obtains the consumption message with the execution status of unexecuted and the unique identifier corresponding to the consumption message from the first server. After obtaining the distributed lock according to the unique identifier and executing the business process, if the execution is successful, it is necessary to notify the first server to change the execution status of the consumption message to executed, and the executed consumption message can no longer be obtained by the consumer end; if the execution fails, it is necessary to notify the first server to generate a new unique identifier for the consumption message, so that when the consumer end obtains the consumption message again, it can obtain the distributed lock corresponding to the consumption message from the second server according to the new unique identifier. This process gives the consumption message the opportunity to execute multiple times, while eliminating the possibility of repeated execution of the successfully executed consumption message, ensuring the accuracy of the system data while increasing the probability of successful execution of the business process.

[0137] in addition, Figure 4 The corresponding embodiment scheme can be Figure 2 or Figure 3 The corresponding implementation schemes are combined, that is, the business process corresponding to the consumption message is successfully executed when the simulation is executed, and the consumption message obtains its corresponding distributed lock, but the business process corresponding to the consumption message fails to execute during formal execution. By allocating a new unique identifier to the consumption message on the first server to make up for the leak, the probability of successful execution of the business process is further improved.

[0138] As above, see Figure 5 , Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 5 As shown, the electronic device includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the following steps:

[0139] Acquire a consumption message and a unique identifier corresponding to the consumption message from the first server;

[0140] Determining that the consumption message is a to-be-executed consumption message that meets an execution condition;

[0141] Acquire, from the second server, a distributed lock corresponding to the to-be-executed consumption message, wherein the unique identifier corresponding to the to-be-executed consumption message is not recorded by the second server;

[0142] Execute the business process corresponding to the to-be-executed consumption message according to the obtained distributed lock;

[0143] The first server is notified to modify the execution status of the to-be-executed consumption message according to the execution result of the business process.

[0144] It can be seen that in the embodiment of the present application, firstly, the consumption message and the unique identifier corresponding to the consumption message are obtained from the first server; then, the distributed lock corresponding to the consumption message that meets the execution condition is obtained from the second server, wherein the unique identifier corresponding to the consumption message is not recorded by the second server; and the business process corresponding to the consumption message is executed according to the obtained distributed lock; finally, the first server is notified to modify the execution status of the consumption message according to the execution result of the business process. In this process, by setting the execution condition and determining that the distributed lock can be obtained only when the consumption message meets the execution condition, the pertinence and efficiency of obtaining the distributed lock are improved, and the distributed lock is allocated to the consumption message according to the unique identifier, which can effectively prevent the repeated execution of the consumption message, reduce the dirty data of the system, and improve the correctness of the system data.

[0145] In a possible example, determining that the consumption message is a to-be-executed consumption message that satisfies an execution condition includes:

[0146] Obtaining the execution time corresponding to the consumption message, wherein the execution time includes determining according to a timer or determining according to the duration of generating the consumption message;

[0147] When the time from the execution time is less than a first preset time threshold, determining that the consumption message is a consumption message that reaches the execution time;

[0148] Determine the consumption message that reaches the execution time as the to-be-executed consumption message that meets the execution condition.

[0149] In a possible example, after determining that the consumption message is a consumption message that reaches the execution time, the method further includes:

[0150] Determine the number of consumption messages that arrive at the execution time;

[0151] When the number of consumption messages arriving at the execution time is greater than a preset number, obtaining multiple waiting times of the multiple consumption messages arriving at the execution time;

[0152] Sort the messages according to the waiting time, and determine the consumption message corresponding to the waiting time at the top of the sort as the highest priority;

[0153] Get the consumption message with the highest priority as the consumption message to be executed.

[0154] In a possible example, before obtaining the distributed lock corresponding to the to-be-executed consumption message from the second server, the method further includes:

[0155] Obtaining a service ID corresponding to the unique identifier;

[0156] Traversing the simulation execution table in the simulation execution system according to the business ID;

[0157] If it is determined that the simulation execution table does not include the service ID, obtaining a consumption message corresponding to the service ID, and simulating the execution of the service process corresponding to the consumption message;

[0158] If the execution result of the business process is execution failure, the execution time corresponding to the consumption message is postponed, and a business ID corresponding to the unique identifier of the consumption message is regenerated;

[0159] Diagnose and modify the business logic corresponding to the consumption message, and repeat the simulation execution process.

[0160] In a possible example, the method further includes:

[0161] The service ID corresponding to the consumption message of the successful simulation execution is obtained, and a simulation execution table is generated according to the service ID.

[0162] In a possible example, notifying the first server to modify the execution state of the consumption message according to the execution result of the business process includes:

[0163] If the execution result of the business process is execution failure, sending a first prompt message to the first server to prompt the first server to generate a new unique identifier corresponding to the consumption message;

[0164] If the execution result of the business process is successful, a second prompt message is sent to the first server to prompt the first server to mark the execution status of the consumption message as executed.

[0165] In a possible example, obtaining the consumption message and the unique identifier corresponding to the consumption message from the first server includes:

[0166] A consumption message with an execution status of unexecuted and a unique identifier corresponding to the consumption message are obtained from the first server.

[0167] Figure 6 6 is a functional unit block diagram of a service execution device 600 involved in an embodiment of the present application. The service execution device 600 is applied to an electronic device, and the service execution device 600 includes:

[0168] A first acquisition unit 601 is used to acquire a consumption message and a unique identifier corresponding to the consumption message from a first server, and determine that the consumption message is a to-be-executed consumption message that meets an execution condition;

[0169] A second acquisition unit 602 acquires a distributed lock corresponding to the to-be-executed consumption message from a second server, wherein the unique identifier corresponding to the to-be-executed consumption message is not recorded by the second server;

[0170] An execution unit 603 is used to execute the business process corresponding to the consumption message to be executed according to the obtained distributed lock;

[0171] The notification unit 604 is used to notify the first server to modify the execution status of the to-be-executed consumption message according to the execution result of the business process.

[0172] It can be seen that in the embodiment of the present application, the business execution device first obtains the consumption message and the unique identifier corresponding to the consumption message from the first server; then obtains the distributed lock corresponding to the consumption message that meets the execution condition from the second server, wherein the unique identifier corresponding to the consumption message is not recorded by the second server; and executes the business process corresponding to the consumption message according to the obtained distributed lock; finally, notifies the first server to modify the execution status of the consumption message according to the execution result of the business process. In this process, by setting the execution condition and determining that only the consumption message that meets the execution condition can obtain the distributed lock, the pertinence and efficiency of obtaining the distributed lock are improved, and the distributed lock is allocated to the consumption message according to the unique identifier, which can effectively prevent the repeated execution of the consumption message, reduce the dirty data of the system, and improve the correctness of the system data.

[0173] In an optional example, the first acquiring unit 601 is specifically configured to:

[0174] Obtaining the execution time corresponding to the consumption message, wherein the execution time includes determining according to a timer or determining according to the duration of generating the consumption message;

[0175] When the time from the execution time is less than a first preset time threshold, determining that the consumption message is a consumption message that reaches the execution time;

[0176] Determine the consumption message that reaches the execution time as the to-be-executed consumption message that meets the execution condition.

[0177] In an optional example, the first acquiring unit 601 is further configured to:

[0178] Determine the number of consumption messages that arrive at the execution time;

[0179] When the number of consumption messages arriving at the execution time is greater than a preset number, obtaining multiple waiting times of the multiple consumption messages arriving at the execution time;

[0180] Sort the messages according to the waiting time, and determine the consumption message corresponding to the waiting time at the top of the sort as the highest priority;

[0181] Get the consumption message with the highest priority as the consumption message to be executed.

[0182] In an optional example, the first acquiring unit 601 is further configured to:

[0183] Obtaining a service ID corresponding to the unique identifier;

[0184] Traversing the simulation execution table in the simulation execution system according to the business ID;

[0185] If it is determined that the simulation execution table does not include the service ID, obtaining a consumption message corresponding to the service ID, and simulating the execution of the service process corresponding to the consumption message;

[0186] If the execution result of the business process is execution failure, the execution time corresponding to the consumption message is postponed, and a business ID corresponding to the unique identifier of the consumption message is regenerated;

[0187] Diagnose and modify the business logic corresponding to the consumption message, and repeat the simulation execution process.

[0188] In an optional example, the first acquiring unit 601 is further configured to:

[0189] The service ID corresponding to the consumption message of the successful simulation execution is obtained, and a simulation execution table is generated according to the service ID.

[0190] In an optional example, the notification unit 604 is specifically configured to:

[0191] If the execution result of the business process is execution failure, sending a first prompt message to the first server to prompt the first server to generate a new unique identifier corresponding to the consumption message;

[0192] If the execution result of the business process is successful, a second prompt message is sent to the first server to prompt the first server to mark the execution status of the consumption message as executed.

[0193] In an optional example, the first acquiring unit 601 is specifically configured to:

[0194] A consumption message with an execution status of unexecuted and a unique identifier corresponding to the consumption message are obtained from the first server.

[0195] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments, and the above computer includes a mobile terminal.

[0196] The present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes a mobile terminal.

[0197] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0198] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0199] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0200] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0202] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0203] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable memory, which may include a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0204] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A distributed data processing method, characterized in that: Applied to the consumer end, the method includes: Acquire a consumption message and a unique identifier corresponding to the consumption message from the first server; Determining that the consumption message is a to-be-executed consumption message that meets an execution condition; Acquire, from the second server, a distributed lock corresponding to the to-be-executed consumption message, wherein the unique identifier corresponding to the to-be-executed consumption message is not recorded by the second server; Execute the business process corresponding to the to-be-executed consumption message according to the obtained distributed lock; Notifying the first server to modify the execution status of the to-be-executed consumption message according to the execution result of the business process; Before obtaining the distributed lock corresponding to the to-be-executed consumption message from the second server, the method further includes: Obtain the business ID corresponding to the unique identifier, and generate a business ID for each unique identifier; traverse the simulation execution table in the execution system in reverse order according to the business ID, and the simulation execution table is generated according to the business ID corresponding to the consumption message of the successful simulation execution; if it is determined that the simulation execution table does not include the business ID, obtain the consumption message corresponding to the business ID, and simulate the execution of the business process corresponding to the consumption message; if the execution result of the business process is execution failure, postpone the execution time corresponding to the consumption message, and regenerate the business ID corresponding to the unique identifier of the consumption message; diagnose and modify the business logic corresponding to the consumption message, and repeat the simulation execution process; Get the business ID corresponding to the consumption message of the successful simulation execution, and generate a simulation execution table based on the business ID.

2. The method according to claim 1, characterized in that Determining that the consumption message is a consumption message to be executed that meets the execution condition includes: Obtaining the execution time corresponding to the consumption message, wherein the execution time includes determining according to a timer or determining according to the duration of generating the consumption message; When the time from the execution time is less than a first preset time threshold, determining that the consumption message is a consumption message that reaches the execution time; Determine the consumption message that reaches the execution time as the to-be-executed consumption message that meets the execution condition.

3. The method according to claim 2, characterized in that After determining that the consumption message is a consumption message that reaches the execution time, the method further includes: Determine the number of consumption messages that arrive at the execution time; When the number of consumption messages arriving at the execution time is greater than the preset number, multiple waiting times of the consumption messages arriving at the execution time are obtained; Sort the messages according to the size of the waiting time, and determine the consumption message corresponding to the waiting time at the top of the sort as the highest priority; Get the consumption message with the highest priority as the consumption message to be executed.

4. The method according to claim 1, characterized in that The notifying the first server to modify the execution status of the to-be-executed consumption message according to the execution result of the business process includes: If the execution result of the business process is execution failure, sending a first prompt message to the first server to prompt the first server to generate a new unique identifier corresponding to the consumption message; If the execution result of the business process is successful, a second prompt message is sent to the first server to prompt the first server to mark the execution status of the consumption message as executed.

5. The method according to claim 4, characterized in that The acquiring the consumption message and the unique identifier corresponding to the consumption message from the first server includes: A consumption message with an execution status of unexecuted and a unique identifier corresponding to the consumption message are obtained from the first server.

6. A distributed data processing device, the device being used to execute the method according to any one of claims 1 to 5, characterized in that: The device comprises: A first acquisition unit, configured to acquire a consumption message and a unique identifier corresponding to the consumption message from a first server, and determine that the consumption message is a to-be-executed consumption message that meets an execution condition; A second acquisition unit acquires, from a second server, a distributed lock corresponding to the to-be-executed consumption message, wherein the unique identifier corresponding to the to-be-executed consumption message is not recorded by the second server; An execution unit, used to execute the business process corresponding to the to-be-executed consumption message according to the acquired distributed lock; A notification unit is used to notify the first server to modify the execution status of the to-be-executed consumption message according to the execution result of the business process.

7. An electronic device, characterized in that: The method comprises a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 5.

Images