A data processing method and system for concurrent data

Abstract

The application discloses a data processing method and system for concurrent data, which comprises the following steps: sending the to-be-processed concurrent data of each business type in a business end to a concurrent data pool in parallel; when it is determined that a processing module is not in an active state, sending a distributed lock to the processing module, so that the processing module sends the first to-be-processed concurrent data obtained from the concurrent data pool to a business plug-in end, so that the business plug-in end processes the first to-be-processed concurrent data based on a first plug-in instance, and sends the first data processing result obtained to the processing module, so that the processing module updates the first to-be-processed concurrent data in the concurrent data pool, and re-obtains the first to-be-processed concurrent data from the concurrent data pool until the number of acquisition times reaches a preset threshold, and the first to-be-processed concurrent data cannot be obtained from the concurrent data pool at present. Compared with the prior art, the technical scheme of the application can improve the processing efficiency of the concurrent data.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to a data processing method and system for concurrent data. Background Technology

[0002] Workflow tasks and message statistics calculations are complex scenarios with overlapping changes in process, task, and message lifecycles. The frequency of asynchronous differential data state changes is high, which can easily generate concurrent data.

[0003] Currently, concurrent data processing generally uses locking directly. Locks have a waiting mechanism, which can easily lead to a waste of server resources and a decrease in system performance. Furthermore, due to business instability, distributed locks may be occupied for a long time. If the locks are not released, deadlocks can easily occur. A large influx of business can also exhaust distributed lock resources, resulting in low data processing efficiency. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a data processing method and system for concurrent data, so as to improve the processing efficiency of concurrent data.

[0005] To address the aforementioned technical problems, this invention provides a method for processing concurrent data, comprising:

[0006] Obtain the concurrent data to be processed corresponding to each business type in the business terminal, and send the concurrent data to be processed to the concurrent data pool in parallel;

[0007] Determine whether the current state of the processing module corresponding to each business type is active;

[0008] When it is determined that the processing module is not in an active state, a distributed lock is sent to the processing module so that after receiving the distributed lock, the processing module updates its current state to an active state, retrieves the corresponding first concurrent data to be processed from the concurrent data pool based on a preset first business type, and sends the first concurrent data to be processed to the business plugin end. The business plugin end then processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, and sends the obtained first data processing result to the processing module so that the processing module updates the first concurrent data to be processed in the concurrent data pool according to the received first data processing result.

[0009] In one possible implementation, determining whether the current state of the processing module corresponding to each business type is active includes:

[0010] Obtain the first cached data of the processing module corresponding to each business type, determine whether there is a string marker in the first cached data, and if the string marker exists in the first cached data, determine that the current state of the processing module is active; otherwise, determine whether there is a distributed lock in the processing module.

[0011] If a distributed lock exists in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

[0012] In one possible implementation, before sending the first concurrent data to be processed to the business plugin terminal, so that the business plugin terminal processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, the method further includes:

[0013] The processing module, based on a preset first business type and a preset mapping relationship between business types and business plugins, searches for the first business plugin corresponding to the first business type from the business plugin side, instantiates the first business plugin, and obtains a first plugin instance.

[0014] In one possible implementation, the instantiation of the first business plugin to obtain a first plugin instance specifically includes:

[0015] The processing module obtains the first deployment node corresponding to the first business plugin, and compares the first deployment node with the first service node, wherein the first service node is the first service node corresponding to the processing module.

[0016] If the first deployment node is the same as the first service node, then a first plugin instance corresponding to the first business plugin is generated based on the reflection mechanism; otherwise, the instantiation processing request is routed to the first deployment node so that a first plugin instance corresponding to the first business plugin is generated in the first deployment node.

[0017] In one possible implementation, the processing module updates the first concurrent data to be processed in the concurrent data pool based on the received first data processing result, specifically including:

[0018] The first data processing result includes a first data ID corresponding to each first concurrent data to be processed, and a deletion identifier corresponding to each first data ID, wherein the deletion identifier includes deletion and non-deletion;

[0019] After receiving the first data processing result, the processing module retrieves the target first concurrent data to be processed corresponding to the first data ID from the concurrent data pool according to the first data ID in the first data processing result.

[0020] Obtain the deletion identifier corresponding to the first data ID. When the deletion identifier indicates deletion, delete the target first concurrent data to be processed. When the deletion identifier indicates no deletion, update the target first concurrent data to be processed to the first processed concurrent data.

[0021] In one possible implementation, after sending the concurrent data to be processed to the concurrent data pool in parallel, the method further includes:

[0022] The concurrent data to be processed in the concurrent data pool is atomically incremented to obtain the number of concurrent data to be processed in the concurrent data pool.

[0023] In one possible implementation, after the processing module updates the first concurrent data to be processed in the concurrent data pool based on the received first data processing result, it further includes:

[0024] The processing module records the number of times the first concurrent data to be processed is acquired, and re-acquires the first concurrent data to be processed from the concurrent data pool until the number of acquisitions reaches a preset threshold and the first concurrent data to be processed cannot be acquired from the concurrent data pool. Then, the activation state of the processing module is adjusted to the deactivated state.

[0025] The present invention also provides a method for processing concurrent data, comprising:

[0026] When the current state is inactive, a distributed lock is received from the collection module so that the current state is adjusted to an active state based on the distributed lock;

[0027] Based on a preset second business type, the second concurrent data to be processed corresponding to the second business type is obtained from the concurrent data pool. The second concurrent data to be processed is obtained by the collection module collecting the second concurrent data to be processed sent by the business terminal and sending the second concurrent data to be processed into the concurrent data pool.

[0028] The second concurrent data to be processed is sent to the business plugin terminal so that the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type and generates the second data processing result.

[0029] The system receives the second data processing result sent by the business plugin and updates the second concurrent data to be processed in the concurrent data pool according to the second data processing result.

[0030] In one possible implementation, before receiving the distributed lock sent by the collection module, the following is also included:

[0031] The collection module determines whether the current state of the processing module corresponding to each business type is active. When it is determined that the processing module is not active, a distributed lock is sent to the processing module.

[0032] The collection module determines whether the current state of the processing module corresponding to each service type is active, including:

[0033] Obtain the second cache data of the processing module corresponding to each business type, and determine whether there is a string marker in the second cache data. If the string marker exists in the second cache data, determine that the current state of the processing module is active; otherwise, determine whether there is a distributed lock in the processing module.

[0034] If a distributed lock exists in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

[0035] In one possible implementation, before sending the second concurrent data to be processed to the business plugin terminal, so that the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type, the method further includes:

[0036] Based on the preset second business type and the preset mapping relationship between business type and business plugin, the second business plugin corresponding to the second business type is found from the business plugin end, and the second business plugin is instantiated to obtain the second plugin instance.

[0037] Specifically, the second business plugin is instantiated to obtain a second plugin instance, including:

[0038] Obtain the second deployment node corresponding to the second business plugin, and compare the second deployment node with the second service node, wherein the second service node is the second service node corresponding to the processing module;

[0039] If the second deployment node is the same as the second service node, a second plugin instance corresponding to the second business plugin is generated based on the reflection mechanism; otherwise, the instantiation processing request is routed to the second deployment node so that a second plugin instance corresponding to the second business plugin is generated in the second deployment node.

[0040] In one possible implementation, updating the second concurrent data to be processed in the concurrent data pool based on the second data processing result specifically includes:

[0041] The data processing result includes a second data ID corresponding to each second concurrent data to be processed, and a deletion identifier corresponding to each second data ID, wherein the deletion identifier includes deletion and no deletion;

[0042] After receiving the second data processing result, the target second concurrent data to be processed corresponding to the second data ID is obtained from the concurrent data pool according to the second data ID in the second data processing result;

[0043] Obtain the deletion identifier corresponding to the second data ID. When the deletion identifier indicates deletion, delete the target second concurrent data to be processed. When the deletion identifier indicates no deletion, update the target second concurrent data to be processed with the first processed concurrent data.

[0044] In one possible implementation, after the collection module collects the second concurrent data to be processed sent by the service terminal and sends the second concurrent data to the concurrent data pool, it further includes:

[0045] The collection module performs atomic incrementing of the number of concurrent data to be processed in the concurrent data pool to obtain the number of concurrent data to be processed in the concurrent data pool.

[0046] In one possible implementation, after updating the second concurrent data to be processed in the concurrent data pool according to the second data processing result, the method further includes:

[0047] After updating the first concurrent data to be processed in the concurrent data pool, the number of times the first concurrent data to be processed is obtained is recorded. The first concurrent data to be processed is then obtained from the concurrent data pool again until the number of times it is obtained reaches a preset threshold and the first concurrent data to be processed cannot be obtained from the concurrent data pool. Then, the activation state of the processing module is adjusted to the deactivated state.

[0048] The present invention also provides a collection module, comprising: a first concurrent data collection unit, an activation state determination unit, and a first concurrent data processing unit;

[0049] The first concurrent data collection unit is used to acquire concurrent data to be processed corresponding to each business type in the business terminal, and send the concurrent data to be processed to the concurrent data pool in parallel.

[0050] The activation status determination unit is used to determine whether the current status of the processing module corresponding to each business type is active.

[0051] The first concurrent data processing unit is configured to send a distributed lock to the processing module when it is determined that the processing module is not in an active state, so that after receiving the distributed lock, the processing module updates its current state to an active state, obtains first concurrent data to be processed from the concurrent data pool based on a preset first business type, sends the first concurrent data to be processed to the business plugin end, so that the business plugin end processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, and sends the obtained first data processing result to the processing module, so that the processing module updates the first concurrent data to be processed in the concurrent data pool according to the received first data processing result.

[0052] In one possible implementation, the first concurrent data processing unit, after enabling the processing module to update the first concurrent data to be processed in the concurrent data pool according to the received first data processing result, further includes:

[0053] Record the number of times the first concurrent data to be processed is obtained, and retrieve the first concurrent data to be processed from the concurrent data pool again until the number of retrievals reaches a preset threshold and the first concurrent data to be processed cannot be retrieved from the concurrent data pool. Then, adjust the activation state of the processing module to the deactivated state.

[0054] In one possible implementation, the first concurrent data collection unit is used to determine whether the current state of the processing module corresponding to each business type is active, specifically including:

[0055] Obtain the first cached data of the processing module corresponding to each business type, determine whether there is a string marker in the first cached data, and if the string marker exists in the first cached data, determine that the current state of the processing module is active; otherwise, determine whether there is a distributed lock in the processing module.

[0056] If a distributed lock exists in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

[0057] In one possible implementation, the first concurrent data processing unit, configured to send the first concurrent data to be processed to the business plugin terminal, further includes the following before the business plugin terminal processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type:

[0058] The processing module, based on a preset first business type and a preset mapping relationship between business types and business plugins, searches for the first business plugin corresponding to the first business type from the business plugin side, instantiates the first business plugin, and obtains a first plugin instance.

[0059] In one possible implementation, the first concurrent data processing unit, used for instantiating the first business plugin to obtain a first plugin instance, specifically includes:

[0060] The processing module obtains the first deployment node corresponding to the first business plugin, and compares the first deployment node with the first service node, wherein the first service node is the first service node corresponding to the processing module.

[0061] If the first deployment node is the same as the first service node, then a first plugin instance corresponding to the first business plugin is generated based on the reflection mechanism; otherwise, the instantiation processing request is routed to the first deployment node so that a first plugin instance corresponding to the first business plugin is generated in the first deployment node.

[0062] In one possible implementation, the first concurrent data processing unit is configured to cause the processing module to update the first concurrent data to be processed in the concurrent data pool according to the received first data processing result, specifically including:

[0063] The first data processing result includes a first data ID corresponding to each first concurrent data to be processed, and a deletion identifier corresponding to each first data ID, wherein the deletion identifier includes deletion and non-deletion;

[0064] After receiving the first data processing result, the processing module retrieves the target first concurrent data to be processed corresponding to the first data ID from the concurrent data pool according to the first data ID in the first data processing result.

[0065] Obtain the deletion identifier corresponding to the first data ID. When the deletion identifier indicates deletion, delete the target first concurrent data to be processed. When the deletion identifier indicates no deletion, update the target first concurrent data to be processed to the first processed concurrent data.

[0066] In one possible implementation, after the first concurrent data collection unit sends the concurrent data to be processed to the concurrent data pool in parallel, it further includes:

[0067] The concurrent data to be processed in the concurrent data pool is atomically incremented to obtain the number of concurrent data to be processed in the concurrent data pool.

[0068] The present invention also provides a processing module, including: a state adjustment unit, a concurrent data acquisition unit, a second concurrent data processing unit, and a concurrent data repetition acquisition unit;

[0069] The state adjustment unit is used to receive a distributed lock sent by the collection module when the current state is an inactive state, so as to adjust the current state to an active state based on the distributed lock.

[0070] The concurrent data acquisition unit is used to acquire second concurrent data to be processed corresponding to the second business type from the concurrent data pool based on the preset second business type. The second concurrent data to be processed is acquired by the collection module collecting the second concurrent data to be processed sent by the business terminal and sending the second concurrent data to be processed to the concurrent data pool.

[0071] The second concurrent data processing unit is used to send the second concurrent data to be processed to the business plugin terminal, so that the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type and generates a second data processing result.

[0072] The concurrent data repetition acquisition unit is used to receive the second data processing result sent by the business plugin end, and update the second concurrent data to be processed in the concurrent data pool according to the second data processing result.

[0073] In one possible implementation, the concurrent data repetition acquisition unit, after updating the second concurrent data to be processed in the concurrent data pool according to the second data processing result, further includes:

[0074] The concurrent data re-acquisition unit is used to record the second acquisition count of the current second concurrent data to be processed, and re-acquire the second concurrent data to be processed from the concurrent data pool until the second acquisition count reaches a preset threshold and the second concurrent data to be processed cannot be acquired from the concurrent data pool at present, and then adjust the current state to an inactive state.

[0075] In one possible implementation, the state adjustment unit, before receiving the distributed lock sent by the collection module, further includes:

[0076] The state adjustment unit is used to enable the collection module to determine whether the current state of the processing module corresponding to each service type is active. When it is determined that the processing module is not active, a distributed lock is sent to the processing module.

[0077] The state adjustment unit is used to enable the collection module to determine whether the current state of the processing module corresponding to each service type is active, including:

[0078] Obtain the second cache data of the processing module corresponding to each business type, and determine whether there is a string marker in the second cache data. If the string marker exists in the second cache data, determine that the current state of the processing module is active; otherwise, determine whether there is a distributed lock in the processing module.

[0079] If a distributed lock exists in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

[0080] In one possible implementation, the second concurrent data processing unit, used to send the second concurrent data to be processed to the business plugin terminal, further includes the following before the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type:

[0081] Based on the preset second business type and the preset mapping relationship between business type and business plugin, the second business plugin corresponding to the second business type is found from the business plugin end, and the second business plugin is instantiated to obtain the second plugin instance.

[0082] Specifically, the second business plugin is instantiated to obtain a second plugin instance, including:

[0083] Obtain the second deployment node corresponding to the second business plugin, and compare the second deployment node with the second service node, wherein the second service node is the second service node corresponding to the processing module;

[0084] If the second deployment node is the same as the second service node, a second plugin instance corresponding to the second business plugin is generated based on the reflection mechanism; otherwise, the instantiation processing request is routed to the second deployment node so that a second plugin instance corresponding to the second business plugin is generated in the second deployment node.

[0085] In one possible implementation, the concurrent data re-acquisition unit is used to update the second concurrent data to be processed in the concurrent data pool according to the second data processing result, specifically including:

[0086] The data processing result includes a second data ID corresponding to each second concurrent data to be processed, and a deletion identifier corresponding to each second data ID, wherein the deletion identifier includes deletion and no deletion;

[0087] After receiving the second data processing result, the target second concurrent data to be processed corresponding to the second data ID is obtained from the concurrent data pool according to the second data ID in the second data processing result;

[0088] Obtain the deletion identifier corresponding to the second data ID. When the deletion identifier indicates deletion, delete the target second concurrent data to be processed. When the deletion identifier indicates no deletion, update the target second concurrent data to be processed with the first processed concurrent data.

[0089] In one possible implementation, the concurrent data acquisition unit, after enabling the collection module to collect the second concurrent data to be processed sent by the business terminal and sending the second concurrent data to be processed into the concurrent data pool, further includes:

[0090] The concurrent data acquisition unit is used by the collection module to perform atomic increment processing on the concurrent data to be processed in the concurrent data pool to obtain the number of concurrent data to be processed in the concurrent data pool.

[0091] The present invention also provides a data processing system for concurrent data, comprising: a business terminal, a business plug-in terminal, a collection module, and a processing module;

[0092] The service terminal is communicatively connected to the collection module, the collection module is communicatively connected to the processing module, and the processing module is communicatively connected to the service plug-in terminal.

[0093] The collection module performs the data processing method for concurrent data as described above;

[0094] The processing module executes the data processing method for concurrent data as described in any of the above.

[0095] The present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the data processing method for concurrent data as described in any of the preceding claims.

[0096] The present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a data processing method for concurrent data as described in any of the preceding claims.

[0097] The present invention provides a data processing method and system for concurrent data, which has the following advantages compared with the prior art:

[0098] The system acquires concurrent data to be processed corresponding to each business type in the business terminal, sends the concurrent data to be processed in parallel to the concurrent data pool, and determines whether the current state of the processing module corresponding to each business type is active. If the processing module is not active, a distributed lock is sent to the processing module so that the processing module updates its current state to active after receiving the distributed lock. Based on a preset first business type, the system acquires the first concurrent data to be processed from the concurrent data pool and sends the first concurrent data to be processed to the business plugin terminal. The business plugin terminal processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type and sends the obtained first data processing result to the processing module so that the processing module updates the first concurrent data to be processed in the concurrent data pool according to the received first data processing result.

[0099] Compared with existing technologies, the technical solution of this invention adopts a distributed parallel collection method for the concurrent data to be processed corresponding to each business type in the business terminal through the collection module, and centrally stores it based on the concurrent data pool. Furthermore, a corresponding processing module is configured for each business type to achieve efficient serial processing of the concurrent data to be processed. This effectively solves the problems of complex business operations and multi-threaded concurrent data, improving the processing efficiency of concurrent data. Simultaneously, based on the processing module, the first plugin instance corresponding to each business type is found on the business plugin side, and the concurrent data to be processed is sent to the first plugin instance so that the first plugin instance can process the concurrent data. This separates the data collection process from the processing process and decouples the specific business scenario from the concurrent data module process, achieving complete functional decoupling. Moreover, because the processing module adopts an efficient single-threaded mode, only one distributed lock is used throughout the entire processing lifecycle. Concurrent businesses no longer need to worry about the use of the data lock, greatly improving the efficiency of distributed lock usage and also increasing the fault tolerance of the distributed lock. Attached Figure Description

[0100] Figure 1 This is a flowchart illustrating an embodiment of a concurrent data processing method provided by the present invention;

[0101] Figure 2This is a flowchart illustrating another embodiment of a concurrent data processing method provided by the present invention;

[0102] Figure 3 This is a schematic diagram of one embodiment of a collection device provided by the present invention.

[0103] Figure 4 This is a schematic diagram of one embodiment of a processing device provided by the present invention.

[0104] Figure 5 This is a schematic diagram of the structure of an embodiment of a concurrent data processing system provided by the present invention. Detailed Implementation

[0105] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0106] Example 1, see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of a concurrent data processing method provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 101-103, as detailed below:

[0107] Step 101: Obtain the concurrent data to be processed corresponding to each business type in the business terminal, and send the concurrent data to be processed in parallel to the concurrent data pool.

[0108] In one embodiment, data interaction between the business terminal and the collection module is realized based on the data collection interface, so that the collection module collects the concurrent data to be processed corresponding to each business type in the business terminal based on the data collection interface.

[0109] Specifically, the business side uses the data collection interface to assemble and package concurrent data, and indicates the business type corresponding to the concurrent data. It integrates the concurrent data and its corresponding business type to generate concurrent data to be processed, and sends the concurrent data to be processed to the collection module through the data collection interface.

[0110] Preferably, the generated concurrent data to be processed includes data (concurrent data), bizType (business type), state (data status), time (time and param (others).

[0111] In one embodiment, the collection module collects concurrent data to be processed corresponding to each business type in the business terminal using an asynchronous parallel collection method.

[0112] Specifically, an asynchronous task or thread is created for each business type, and the asynchronous task or thread is used to collect concurrent data to be processed for that business type in parallel.

[0113] In one embodiment, a concurrent data pool is created and initialized to store concurrent data to be processed corresponding to each business type, so as to achieve unified aggregation and centralized management of concurrent data to be processed corresponding to each business type.

[0114] In one embodiment, after the collection module sends the collected service types to the concurrent data pool, it also performs atomic increment processing on the concurrent data to be processed in the concurrent data pool to obtain the concurrent data quantity of the concurrent data to be processed in the concurrent data pool, so as to ensure accurate counting of concurrent data in a concurrent environment.

[0115] Specifically, a global counter variable is initialized to record the number of data items and set to 0. After the concurrent data to be processed is sent to the concurrent data pool in parallel, the current data volume of the concurrent data to be processed is obtained. The counter variable is updated based on the current data volume. A mutex lock or atomic operation is used to perform atomic operations on the counter variable to atomically increment the value of the counter variable, and the increment value of the counter variable is used as the number of concurrent data items and written into the concurrent data pool.

[0116] Preferably, the Redis command `INCR` is used to atomically increment the number of data entries in the cache. The steps are as follows: First, ensure a connection to the Redis server is established. A suitable Redis client can be used to establish the connection, typically requiring specifying the Redis server's host address, port number, and database ID. Define a key name to store the data count; a meaningful key name, such as `data_count`, can be chosen. Use the `INCR` command to perform the atomic increment operation, passing the defined key name as a parameter. If the key does not exist, Redis will automatically create it and initialize its value to 0. After executing the `INCR` command, the incremented data count is obtained. Because the increment operation is atomic, even if multiple concurrent requests call the `INCR` command simultaneously, data inconsistency will not occur. This is because Redis locks the data during the `INCR` command execution, ensuring that each request is processed sequentially.

[0117] Step 102: Determine whether the current status of the processing module corresponding to each business type is active.

[0118] In one embodiment, the collection module sends the concurrent data to be processed corresponding to each business type to the concurrent data pool. After the concurrent data to be processed is stored in the business data pool, the concurrent data pool sends a successful data storage signal to the collection module, so that the collection module can determine whether the current state of the processing module corresponding to each business type is active after receiving the successful data storage signal.

[0119] In one embodiment, when the collection module determines whether the current state of the processing module corresponding to each service type is active, it obtains the first cached data of the processing module corresponding to each service type, and determines whether there is a string marker in the first cached data. If the string marker exists in the first cached data, the current state of the processing module is determined to be active; otherwise, it determines whether there is a distributed lock in the processing module. If there is a distributed lock in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

[0120] Specifically, after completing data collection, the collection module automatically determines whether the processing module is working, i.e., whether the processing module is in an active state. Before being activated, the processing module places a string marker "sign" in the cached data to indicate that the processing module is already activated. After the processing module is activated, it acquires a distributed lock and then begins the actual business processing logic. Based on this, the collection module's judgment logic order is as follows: first, it acquires the cached marker "sign", then checks whether the cached data records the existence of the processing module, and then uses getLockInfo() to determine whether a distributed lock is already present. If any of these checks are true, it indicates that the processing module is working and is in an active state. If both checks are false, it indicates that the processing module is not working and is in an inactive state. In this case, the processing module needs to be activated. After activation, the collection module stops collecting concurrent data.

[0121] In this embodiment, a caching + distributed lock query mechanism is used for the processing thread of the activation processing module to ensure the high efficiency of program execution.

[0122] Step 103: When it is determined that the processing module is not in an active state, a distributed lock is sent to the processing module so that after receiving the distributed lock, the processing module updates its current state to an active state, obtains the corresponding first concurrent data to be processed from the concurrent data pool based on a preset first business type, and sends the first concurrent data to be processed to the business plugin end, so that the business plugin end processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, and sends the obtained first data processing result to the processing module, so that the processing module updates the first concurrent data to be processed in the concurrent data pool according to the received first data processing result.

[0123] In one embodiment, when it is determined that the processing module is not in an active state, a distributed lock is obtained based on the data collection interface and sent to the processing module, so that when the processing module obtains the distributed lock, it updates its current state from an inactive state to an active state based on the distributed lock.

[0124] In one embodiment, since each business type has its corresponding processing module for the pending business data, that is, each processing module is set with a corresponding first business type, when the current state of the processing module is active, it actively obtains all the first pending concurrent data corresponding to the first business type from the concurrent data pool based on its corresponding first business type.

[0125] In one embodiment, after obtaining all the first concurrent data to be processed corresponding to the first business type, the processing module further searches for the first business plugin corresponding to the first business type from the business plugin end based on the preset first business type and the preset mapping relationship between business type and business plugin, and instantiates the first business plugin to obtain the first plugin instance.

[0126] In one embodiment, when the processing module instantiates the first business plugin to obtain a first plugin instance, the processing module obtains the first deployment node corresponding to the first business plugin and compares the first deployment node with the first service node, wherein the first service node is the first service node corresponding to the processing module; if the first deployment node is the same as the first service node, the first plugin instance corresponding to the first business plugin is generated based on the reflection mechanism; otherwise, the instantiation processing request is routed to the first deployment node so that the first plugin instance corresponding to the first business plugin is generated in the first deployment node.

[0127] Specifically, the business plugin registration factory provides plugin registration logic and plugin instantiation functionality, and is responsible for processing specific types of business data. Based on the mapping relationship between business types and business plugins recorded in the business plugin registration factory, it searches for and retrieves business plugins. After finding the corresponding first business plugin according to the first business type, it instantiates the plugin through microservice discovery. If the first business plugin and the processing module are on the same service node, the first instance of the plugin is directly generated by reflection. If they are not on the same deployment node, the routing is to the target node and then the first instance of the plugin is generated by reflection. This business plugin belongs to the singleton pattern, which improves the performance of object usage, and actively searches the data pool for unprocessed concurrent data belonging to this business.

[0128] In one embodiment, the processing module sends all the first concurrent data to be processed corresponding to the first business type to the business plugin end through an interface, so that the first plugin instance generated in the business plugin end can process all the first concurrent data to be processed according to the business rules.

[0129] In one embodiment, a business registration mechanism is adopted to separate and decouple data collection from business processing logic. Businesses no longer need to worry about data lock issues, which reduces business complexity and enhances business universality.

[0130] In one embodiment, after the business plugin terminal processes all the first concurrent data to be processed based on the first plugin instance, it generates a first data processing result and sends the first data processing result to the processing module so that the processing module can be informed of the processing result of all the first concurrent data to be processed in a timely manner.

[0131] Specifically, after processing all the first concurrent data to be processed, the business logic of the first plugin instance will add a set of data IDs and whether to delete the data label to the returned first data processing result.

[0132] In one embodiment, the first data processing result includes a first data ID corresponding to each first concurrent data to be processed, and a deletion identifier corresponding to each first data ID, wherein the deletion identifier includes deletion and non-deletion.

[0133] In one embodiment, after receiving the first data processing result, the processing module further parses the first data processing result and updates the first concurrent data to be processed in the concurrent data pool based on the parsing result.

[0134] Specifically, after receiving the first data processing result, the processing module retrieves the target first concurrent data to be processed corresponding to the first data ID from the concurrent data pool based on the first data ID in the first data processing result; retrieves the deletion identifier corresponding to the first data ID; when the deletion identifier is "delete", the target first concurrent data to be processed is deleted; when the deletion identifier is "do not delete", the target first concurrent data to be processed is updated to first processed concurrent data, that is, the data status of the target first concurrent data to be processed is updated from the pending state to the processed state, which can avoid the reuse of the target first concurrent data to be processed.

[0135] In one embodiment, after updating the first concurrent data to be processed in the concurrent data pool based on the parsing result, the processing module also uses the Redis command `decr` to decrement the number of concurrent data in the concurrent business pool in its corresponding first cached data, thereby realizing data interaction between the processing module and the concurrent data pool.

[0136] In one embodiment, after updating the first concurrent data to be processed in the concurrent data pool, the processing module records the number of times the first concurrent data to be processed is acquired, and retrieves all the first concurrent data to be processed from the concurrent data pool again, until the number of acquisitions reaches a preset threshold and the first concurrent data to be processed cannot be retrieved from the concurrent data pool at present, the activation state of the processing module is adjusted to the inactive state.

[0137] In one embodiment, after the processing module completes the processing of the first batch of concurrent data of the first business type, in order to prevent new batches of concurrent data from being collected into the concurrent data pool, the processing module will not stop working immediately. Instead, it will loop N times according to the configuration to minimize the delay in data use. During the loop, it will record the first acquisition number of the first batch of concurrent data. If no data is found in the end, it indicates that the processing module has completed the processing of all concurrent data in the pool within this life cycle. The activation state of the processing module will be adjusted to the inactive state, and then the current life cycle will end, waiting for the subsequent concurrent data to reactivate the processing module.

[0138] In summary, the concurrent data processing method provided in this embodiment collects concurrent data of various business types in the business terminal in a distributed and parallel manner through the collection module, stores it centrally based on the concurrent data pool, and configures a corresponding processing module for each business type. This achieves efficient serial processing of concurrent data, effectively solving the problems of complex business and multi-threaded concurrent data, and improving the processing efficiency of concurrent data. At the same time, the processing module finds the first plugin instance corresponding to each business type on the business plugin side and sends the concurrent data to be processed to the first plugin instance so that the first plugin instance can process the concurrent data. This separates the data collection process from the processing process and separates the specific business scenario from the concurrent data module process, achieving complete functional decoupling. Furthermore, since the processing module adopts an efficient single-threaded mode, it only occupies one distributed lock throughout the entire processing lifecycle. Concurrent business no longer needs to worry about the use of data locks, which greatly improves the efficiency of distributed lock usage and also improves the fault tolerance of distributed locks.

[0139] Example 2, see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of a concurrent data processing method provided by the present invention, as shown below. Figure 2 As shown, the method includes steps 201-204, as detailed below:

[0140] Step 201: When the current state is inactive, receive the distributed lock sent by the collection module so that the current state is adjusted to an active state based on the distributed lock.

[0141] Step 202: Based on the preset second business type, obtain the second concurrent data to be processed corresponding to the second business type from the concurrent data pool. The second concurrent data to be processed is obtained by the collection module collecting the second concurrent data to be processed sent by the business terminal and sending the second concurrent data to be processed to the concurrent data pool.

[0142] Step 203: Send the second concurrent data to be processed to the business plugin terminal, so that the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type and generates the second data processing result.

[0143] Step 204: Receive the second data processing result sent by the business plugin terminal, and update the second concurrent data to be processed in the concurrent data pool according to the second data processing result.

[0144] In one embodiment, before receiving the distributed lock sent by the collection module, the method further includes: the collection module determining whether the current state of the processing module corresponding to each service type is active; and when it is determined that the processing module is not active, sending the distributed lock to the processing module.

[0145] In one embodiment, the collection module determines whether the current state of the processing module corresponding to each service type is active, including: obtaining the second cache data of the processing module corresponding to each service type; determining whether there is a string marker in the second cache data; if the string marker exists in the second cache data, determining that the current state of the processing module is active; otherwise, determining whether there is a distributed lock in the processing module; if there is a distributed lock in the processing module, determining that the current state of the processing module is active; otherwise, determining that the current state of the processing module is inactive.

[0146] In one embodiment, before sending all the second concurrent data to be processed to the business plugin end so that the business plugin end can process all the second concurrent data to be processed based on the second plugin instance corresponding to the second business type, the method further includes: based on the preset second business type and the preset business type-business plugin mapping relationship, searching for the second business plugin corresponding to the second business type in the business plugin end, and instantiating the second business plugin to obtain the second plugin instance.

[0147] In one embodiment, instantiating the second business plugin to obtain a second plugin instance includes: obtaining a second deployment node corresponding to the second business plugin; comparing the second deployment node with a second service node, wherein the second service node is the second service node corresponding to the processing module; if the second deployment node and the second service node are the same, then generating a second plugin instance corresponding to the second business plugin based on a reflection mechanism; otherwise, routing the instantiation request to the second deployment node so that a second plugin instance corresponding to the second business plugin is generated in the second deployment node.

[0148] In one embodiment, the data processing result includes a second data ID corresponding to each second concurrent data to be processed, and a deletion identifier corresponding to each second data ID, wherein the deletion identifier includes deletion and non-deletion.

[0149] In one embodiment, updating the second concurrent data to be processed in the concurrent data pool according to the second data processing result specifically includes: after receiving the second data processing result, obtaining the target second concurrent data to be processed corresponding to the second data ID from the concurrent data pool according to the second data ID in the second data processing result; obtaining the deletion identifier corresponding to the second data ID; when the deletion identifier is "delete", deleting the target second concurrent data to be processed; when the deletion identifier is "not delete", updating the target second concurrent data to the first processed concurrent data.

[0150] In one embodiment, after the collection module collects the second concurrent data to be processed sent by the service terminal and sends the second concurrent data to be processed into the concurrent data pool, the method further includes: the collection module performs atomic increment processing on the concurrent data to be processed in the concurrent data pool to obtain the concurrent data quantity of the concurrent data to be processed in the concurrent data pool.

[0151] In one embodiment, after updating the first concurrent data to be processed in the concurrent data pool, the processing module records the number of times the first concurrent data to be processed is acquired, and retrieves all the first concurrent data to be processed from the concurrent data pool again, until the number of acquisitions reaches a preset threshold and the first concurrent data to be processed cannot be retrieved from the concurrent data pool at present, the activation state of the processing module is adjusted to the inactive state.

[0152] In summary, the concurrent data processing method provided by this invention activates the working state of the processing module by acquiring the distributed lock sent by the collection module. Because the processing module adopts an efficient single-threaded mode, it only occupies one distributed lock throughout the entire processing lifecycle, eliminating the need for concurrent business operations to worry about data lock usage, thus greatly improving the efficiency of distributed lock usage and its fault tolerance. Furthermore, based on the setting that the processing module only processes concurrent data corresponding to one business type, it achieves efficient serial processing of concurrent data, effectively solving the problems of complex business operations and multi-threaded concurrent data, and improving the processing efficiency of concurrent data. Additionally, based on the processing module finding the first plugin instance corresponding to each business type at the business plugin end and sending the concurrent data to be processed to the first plugin instance, the first plugin instance processes the concurrent data, thus separating the data collection process from the processing process and decoupling the specific business scenario from the concurrent data module process, achieving complete functional decoupling.

[0153] Example 3, see Figure 3 , Figure 3This is a schematic diagram of the structure of one embodiment of the collection module provided by the present invention, as shown below. Figure 3 As shown, the collection module includes a first concurrent data collection unit 301, an activation status judgment unit 302, and a first concurrent data processing unit 303, as detailed below:

[0154] The first concurrent data collection unit 301 is used to acquire concurrent data to be processed corresponding to each business type in the business terminal, and send the concurrent data to be processed to the concurrent data pool in parallel.

[0155] The activation status determination unit 302 is used to determine whether the current status of the processing module corresponding to each service type is active.

[0156] The first concurrent data processing unit 303 is configured to send a distributed lock to the processing module when it is determined that the processing module is not in an active state, so that after receiving the distributed lock, the processing module updates its current state to an active state, obtains first concurrent data to be processed from the concurrent data pool based on a preset first business type, sends the first concurrent data to be processed to the business plugin end, so that the business plugin end processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, and sends the obtained first data processing result to the processing module, so that the processing module updates the first concurrent data to be processed in the concurrent data pool according to the received first data processing result.

[0157] In one embodiment, the first concurrent data processing unit 303 is configured to enable the processing module to update the first concurrent data to be processed in the concurrent data pool according to the received first data processing result, record the number of times the first concurrent data to be processed is acquired, and re-acquire the first concurrent data to be processed from the concurrent data pool until the number of acquisitions reaches a preset threshold and the first concurrent data to be processed cannot be acquired from the concurrent data pool at present, and then adjust the activation state of the processing module to the inactive state.

[0158] In one embodiment, the activation state determination unit 203 is used to determine whether the current state of the processing module corresponding to each service type is active. Specifically, it includes: obtaining the first cached data of the processing module corresponding to each service type; determining whether there is a string marker in the first cached data; if the string marker exists in the first cached data, determining that the current state of the processing module is active; otherwise, determining whether there is a distributed lock in the processing module; if there is a distributed lock in the processing module, determining that the current state of the processing module is active; otherwise, determining that the current state of the processing module is inactive.

[0159] In one embodiment, the first concurrent data processing unit 303 is used to send the first concurrent data to be processed to the business plugin terminal, so that before the business plugin terminal processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, the processing module further includes: based on the preset first business type and the preset business type-business plugin mapping relationship, searching for the first business plugin corresponding to the first business type in the business plugin terminal, and instantiating the first business plugin to obtain the first plugin instance.

[0160] In one embodiment, the first concurrent data processing unit 303 is used to instantiate the first business plugin to obtain a first plugin instance, specifically including: the processing module obtaining the first deployment node corresponding to the first business plugin, comparing the first deployment node with the first service node, wherein the first service node is the first service node corresponding to the processing module; if the first deployment node and the first service node are the same, then the first plugin instance corresponding to the first business plugin is generated based on the reflection mechanism; otherwise, the instantiation processing request is routed to the first deployment node so that the first plugin instance corresponding to the first business plugin is generated in the first deployment node.

[0161] In one embodiment, the first concurrent data processing unit 303 is configured to cause the processing module to update the first concurrent data to be processed in the concurrent data pool according to the received first data processing result. Specifically, the first data processing result includes a first data ID corresponding to each first concurrent data to be processed, and a deletion identifier corresponding to each first data ID, wherein the deletion identifier includes deletion and non-deletion; after receiving the first data processing result, the processing module obtains the target first concurrent data to be processed corresponding to the first data ID from the concurrent data pool according to the first data ID in the first data processing result; obtains the deletion identifier corresponding to the first data ID; when the deletion identifier is deletion, the target first concurrent data to be processed is deleted; when the deletion identifier is non-deletion, the target first concurrent data to be processed is updated to first processed concurrent data.

[0162] In one embodiment, the first concurrent data collection unit 301, after sending the concurrent data to be processed to the concurrent data pool in parallel, further includes: performing atomic increment processing on the concurrent data to be processed in the concurrent data pool to obtain the concurrent data quantity of the concurrent data to be processed in the concurrent data pool.

[0163] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the collection module described above can be referred to the corresponding process in the aforementioned method embodiment 2, and will not be repeated here.

[0164] It should be noted that the above-described embodiments of the collection module are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0165] In summary, the collection module provided in this embodiment collects concurrent data of various business types in the business terminal in a distributed and parallel manner, stores it centrally based on a concurrent data pool, and configures a corresponding processing module for each business type. This achieves efficient serial processing of the concurrent data, effectively solving the problems of complex business and multi-threaded concurrent data, and improving the processing efficiency of concurrent data. At the same time, based on the processing module, the first plugin instance corresponding to each business type is found on the business plugin side, and the concurrent data to be processed is sent to the first plugin instance so that the first plugin instance can process the concurrent data. The data collection process is separated from the processing process, and the specific business scenario is separated from the concurrent data module process, achieving complete functional decoupling. Moreover, since the processing module adopts an efficient single-threaded mode, only one distributed lock is used throughout the entire processing lifecycle. Concurrent business no longer needs to worry about the use of data locks, which greatly improves the efficiency of distributed lock usage and also improves the fault tolerance of distributed locks.

[0166] Example 4, see Figure 4 , Figure 4 This is a schematic diagram of the structure of one embodiment of a processing module provided by the present invention, as shown below. Figure 4 As shown, the processing module includes a state adjustment unit 401, a concurrent data acquisition unit 402, a second concurrent data processing unit 403, and a concurrent data repetition acquisition unit 404, as detailed below:

[0167] The state adjustment unit 401 is used to receive a distributed lock sent by the collection module when the current state is an inactive state, so as to adjust the current state to an active state based on the distributed lock.

[0168] The concurrent data acquisition unit 402 is used to acquire second pending concurrent data corresponding to the second business type from the concurrent data pool based on the preset second business type. The second pending concurrent data is acquired by the collection module collecting the second pending concurrent data sent by the business terminal and sending the second pending concurrent data to the concurrent data pool.

[0169] The second concurrent data processing unit 403 is used to send the second concurrent data to be processed to the business plugin terminal, so that the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type and generates a second data processing result.

[0170] The concurrent data repetition acquisition unit 404 is used to receive the second data processing result sent by the business plug-in terminal, and update the second concurrent data to be processed in the concurrent data pool according to the second data processing result.

[0171] In one embodiment, the concurrent data re-acquisition unit 404 is used to update the second concurrent data to be processed in the concurrent data pool according to the second data processing result, record the second acquisition number of the current second concurrent data to be processed, re-acquire the second concurrent data to be processed from the concurrent data pool, until the second acquisition number reaches a preset threshold and the second concurrent data to be processed cannot be acquired from the concurrent data pool, and then adjust the current state to an inactive state.

[0172] In one embodiment, before receiving the distributed lock sent by the collection module, the state adjustment unit 401 further includes: the state adjustment unit is used to enable the collection module to determine whether the current state of the processing module corresponding to each service type is active, and when it is determined that the processing module is not active, to send a distributed lock to the processing module.

[0173] In one embodiment, the status adjustment form 401 is used to enable the collection module to determine whether the current status of the processing module corresponding to each business type is active. This includes: obtaining second cached data of the processing module corresponding to each business type; determining whether a string marker exists in the second cached data; if the string marker exists in the second cached data, determining that the current status of the processing module is active; otherwise, determining whether a distributed lock exists in the processing module; if a distributed lock exists in the processing module, determining that the current status of the processing module is active; otherwise, determining that the current status of the processing module is inactive.

[0174] In one embodiment, the second concurrent data processing unit 403 is used to send the second concurrent data to be processed to the business plugin terminal, so that before the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type, it further includes: based on the preset second business type and the preset business type-business plugin mapping relationship, searching for the second business plugin corresponding to the second business type in the business plugin terminal, and instantiating the second business plugin to obtain the second plugin instance.

[0175] In one embodiment, the second concurrent data processing unit 403 is used to instantiate the second business plugin to obtain a second plugin instance, including: obtaining the second deployment node corresponding to the second business plugin; comparing the second deployment node with the second service node, wherein the second service node is the second service node corresponding to the processing module; if the second deployment node and the second service node are the same, then generating the second plugin instance corresponding to the second business plugin based on a reflection mechanism; otherwise, routing the instantiation processing request to the second deployment node so that the second plugin instance corresponding to the second business plugin is generated in the second deployment node.

[0176] In one embodiment, the concurrent data repetition acquisition unit 404 is used to update the second concurrent data to be processed in the concurrent data pool according to the second data processing result. Specifically, it includes: the data processing result includes a second data ID corresponding to each second concurrent data to be processed, and a deletion identifier corresponding to each second data ID, wherein the deletion identifier includes deletion and non-deletion; after receiving the second data processing result, according to the second data ID in the second data processing result, the target second concurrent data to be processed corresponding to the second data ID is obtained from the concurrent data pool; the deletion identifier corresponding to the second data ID is obtained, and when the deletion identifier is deletion, the target second concurrent data to be processed is deleted; when the deletion identifier is non-deletion, the target second concurrent data to be processed is updated to the first processed concurrent data.

[0177] In one embodiment, the concurrent data acquisition unit 402 is used to enable the collection module to collect the second concurrent data to be processed sent by the service terminal and send the second concurrent data to be processed to the concurrent data pool. The concurrent data acquisition unit 402 is also used to enable the collection module to perform atomic increment processing on the concurrent data to be processed in the concurrent data pool to obtain the concurrent data quantity of the concurrent data to be processed in the concurrent data pool.

[0178] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the processing module described above can be referred to the corresponding process in the aforementioned method embodiment 2, and will not be repeated here.

[0179] It should be noted that the above embodiments of the processing modules are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0180] In summary, the processing module provided in this embodiment activates its working state by acquiring the distributed lock sent by the collection module. Because the processing module adopts an efficient single-threaded mode, it only occupies one distributed lock throughout its entire processing lifecycle. Concurrent business operations no longer need to worry about data lock usage, greatly improving the efficiency of distributed lock usage and its fault tolerance. Furthermore, based on the setting that the processing module only processes concurrent data corresponding to one business type, it achieves efficient serial processing of concurrent data, effectively solving the problems of complex business operations and multi-threaded concurrent data, and improving the processing efficiency of concurrent data. Simultaneously, based on the processing module finding the first plugin instance corresponding to each business type on the business plugin side and sending the concurrent data to be processed to the first plugin instance, the first plugin instance processes the concurrent data, separating the data collection process from the processing process, and decoupling the specific business scenario from the concurrent data module process, achieving complete functional decoupling.

[0181] Example 5, see Figure 5 , Figure 5 This is a schematic diagram of the structure of an embodiment of a concurrent data processing system provided by the present invention, as shown below. Figure 5 As shown, the processing module includes a business terminal 501, a business plugin terminal 502, a collection module 503, and a processing module 504, as detailed below:

[0182] The service terminal 501 is communicatively connected to the collection module 503, the collection module 503 is communicatively connected to the processing module 504, and the processing module 504 is communicatively connected to the service plug-in terminal 502.

[0183] The collection module 503 executes the data processing method for concurrent data as described in Embodiment 1 above;

[0184] The processing module 504 executes the shared account authentication login method as described in Embodiment 2 above.

[0185] In summary, the concurrent data processing system provided by this invention, through information interaction between the business end, business plug-in end, collection module and processing module, presents a global and modular concept, which can improve the processing efficiency of concurrent data.

[0186] Furthermore, embodiments of the present invention also provide a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the data processing method for concurrent data as described in any of the above.

[0187] Furthermore, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a data processing method for concurrent data as described in any of the above.

[0188] Furthermore, this embodiment of the invention also provides a computer program product, which, when run on a computer device, enables the computer device to implement the concurrent data processing method as described in any of the above embodiments.

[0189] In summary, the concurrent data processing method and system provided by this invention, by sending concurrent data of various business types in the business terminal to a concurrent data pool in parallel, and when it is determined that the processing module is not in an active state, sending a distributed lock to the processing module, so that the processing module sends the first concurrent data to be processed obtained from the concurrent data pool to the business plugin terminal, so that the business plugin terminal processes the first concurrent data to be processed based on the first plugin instance, and sends the obtained first data processing result to the processing module, so that the processing module updates the first concurrent data to be processed in the concurrent data pool, and re-obtains the first concurrent data to be processed from the concurrent data pool, until the number of acquisitions reaches a preset threshold, and the first concurrent data to be processed cannot be obtained from the concurrent data pool at present. Compared with the prior art, the technical solution of this invention can improve the processing efficiency of concurrent data.

[0190] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A method for processing concurrent data, characterized in that, include: Obtain the concurrent data to be processed corresponding to each business type in the business terminal, and send the concurrent data to be processed to the concurrent data pool; Determine whether the current state of the processing module corresponding to each business type is active; When it is determined that the processing module is not in an active state, a distributed lock is sent to the processing module so that after receiving the distributed lock, the processing module updates its current state to an active state, retrieves the corresponding first concurrent data to be processed from the concurrent data pool based on a preset first business type, and sends the first concurrent data to be processed to the business plugin end so that the business plugin end processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, and sends the obtained first data processing result to the processing module so that the processing module updates the first concurrent data to be processed in the concurrent data pool according to the received first data processing result. Determine whether the current state of the processing module corresponding to each business type is active, specifically including: Obtain the first cached data of the processing module corresponding to each business type, determine whether there is a string marker in the first cached data, and if the string marker exists in the first cached data, determine that the current state of the processing module is active; otherwise, determine whether there is a distributed lock in the processing module. If a distributed lock exists in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

2. The data processing method for concurrent data as described in claim 1, characterized in that, Before sending the first concurrent data to be processed to the business plugin terminal, so that the business plugin terminal processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, the method further includes: The processing module, based on a preset first business type and a preset mapping relationship between business types and business plugins, searches for the first business plugin corresponding to the first business type from the business plugin side, instantiates the first business plugin, and obtains a first plugin instance.

3. The data processing method for concurrent data as described in claim 2, characterized in that, The instantiation of the first service plugin to obtain a first plugin instance specifically includes: The processing module obtains the first deployment node corresponding to the first business plugin, and compares the first deployment node with the first service node, wherein the first service node is the first service node corresponding to the processing module. If the first deployment node is the same as the first service node, then a first plugin instance corresponding to the first business plugin is generated based on the reflection mechanism; otherwise, the instantiation processing request is routed to the first deployment node so that a first plugin instance corresponding to the first business plugin is generated in the first deployment node.

4. The data processing method for concurrent data as described in claim 1, characterized in that, The processing module updates the first concurrent data to be processed in the concurrent data pool according to the received first data processing result, specifically including: The first data processing result includes a first data ID corresponding to each first concurrent data to be processed, and a deletion identifier corresponding to each first data ID, wherein the deletion identifier includes deletion and non-deletion; After receiving the first data processing result, the processing module retrieves the target first concurrent data to be processed corresponding to the first data ID from the concurrent data pool according to the first data ID in the first data processing result. Obtain the deletion identifier corresponding to the first data ID. When the deletion identifier indicates deletion, delete the target first concurrent data to be processed. When the deletion identifier indicates no deletion, update the target first concurrent data to be processed to the first processed concurrent data.

5. The data processing method for concurrent data as described in claim 1, characterized in that, After sending the concurrent data to be processed to the concurrent data pool in parallel, the process also includes: The concurrent data to be processed in the concurrent data pool is atomically incremented to obtain the number of concurrent data to be processed in the concurrent data pool.

6. The data processing method for concurrent data as described in claim 1, characterized in that, After updating the first concurrent data to be processed in the concurrent data pool according to the received first data processing result, the processing module further includes: The processing module records the number of times the first concurrent data to be processed is acquired, and re-acquires the first concurrent data to be processed from the concurrent data pool until the number of acquisitions reaches a preset threshold and the first concurrent data to be processed cannot be acquired from the concurrent data pool. Then, the activation state of the processing module is adjusted to the deactivated state.

7. A method for processing concurrent data, characterized in that, include: When the current state is inactive, a distributed lock is received from the collection module so that the current state is adjusted to an active state based on the distributed lock; Based on a preset second business type, the second concurrent data to be processed corresponding to the second business type is obtained from the concurrent data pool. The second concurrent data to be processed is obtained by the collection module collecting the second concurrent data to be processed sent by the business terminal and sending the second concurrent data to be processed into the concurrent data pool. The second concurrent data to be processed is sent to the business plugin terminal so that the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type and generates the second data processing result. Receive the second data processing result sent by the business plugin terminal, and update the second concurrent data to be processed in the concurrent data pool according to the second data processing result; Before receiving the distributed lock sent by the collection module, the following is also included: The collection module determines whether the current state of the processing module corresponding to each business type is active. When it is determined that the processing module is not active, a distributed lock is sent to the processing module. The collection module determines whether the current state of the processing module corresponding to each service type is active, including: Obtain the second cache data of the processing module corresponding to each business type, and determine whether there is a string marker in the second cache data. If the string marker exists in the second cache data, determine that the current state of the processing module is active; otherwise, determine whether there is a distributed lock in the processing module. If a distributed lock exists in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

8. A collection module, characterized in that, include: The system comprises a first concurrent data collection unit, an activation status judgment unit, and a first concurrent data processing unit. The first concurrent data collection unit is used to acquire concurrent data to be processed corresponding to each business type in the business terminal, and send the concurrent data to be processed to the concurrent data pool in parallel. The activation status determination unit is used to determine whether the current status of the processing module corresponding to each business type is active. The first concurrent data processing unit is configured to send a distributed lock to the processing module when it is determined that the processing module is not in an active state, so that after receiving the distributed lock, the processing module updates its current state to an active state, obtains the corresponding first concurrent data to be processed from the concurrent data pool based on a preset first business type, sends the first concurrent data to be processed to the business plugin end, so that the business plugin end processes the first concurrent data to be processed based on the first plugin instance corresponding to the first business type, and sends the obtained first data processing result to the processing module, so that the processing module updates the first concurrent data to be processed in the concurrent data pool according to the received first data processing result; Determine whether the current state of the processing module corresponding to each business type is active, specifically including: Obtain the first cached data of the processing module corresponding to each business type, determine whether there is a string marker in the first cached data, and if the string marker exists in the first cached data, determine that the current state of the processing module is active; otherwise, determine whether there is a distributed lock in the processing module. If a distributed lock exists in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

9. A processing module, characterized in that, include: The system includes a state adjustment unit, a concurrent data acquisition unit, a second concurrent data processing unit, and a concurrent data repetition acquisition unit. The state adjustment unit is used to receive a distributed lock sent by the collection module when the current state is an inactive state, so as to adjust the current state to an active state based on the distributed lock. The concurrent data acquisition unit is used to acquire second concurrent data to be processed corresponding to the second business type from the concurrent data pool based on the preset second business type. The second concurrent data to be processed is acquired by the collection module collecting the second concurrent data to be processed sent by the business terminal and sending the second concurrent data to be processed to the concurrent data pool. The second concurrent data processing unit is used to send the second concurrent data to be processed to the business plugin terminal, so that the business plugin terminal processes the second concurrent data to be processed based on the second plugin instance corresponding to the second business type and generates a second data processing result. The concurrent data repetition acquisition unit is used to receive the second data processing result sent by the business plugin end, and update the second concurrent data to be processed in the concurrent data pool according to the second data processing result; Before receiving the distributed lock sent by the collection module, the state adjustment unit further includes: The collection module determines whether the current state of the processing module corresponding to each business type is active. When it is determined that the processing module is not active, a distributed lock is sent to the processing module. The collection module determines whether the current state of the processing module corresponding to each service type is active, including: Obtain the second cache data of the processing module corresponding to each business type, and determine whether there is a string marker in the second cache data. If the string marker exists in the second cache data, determine that the current state of the processing module is active; otherwise, determine whether there is a distributed lock in the processing module. If a distributed lock exists in the processing module, the current state of the processing module is determined to be active; otherwise, the current state of the processing module is determined to be inactive.

10. A data processing system for concurrent data, characterized in that, include: Business side, business plugin side, collection module and processing module; The service terminal is communicatively connected to the collection module, the collection module is communicatively connected to the processing module, and the processing module is communicatively connected to the service plug-in terminal. The collection module performs the data processing method for concurrent data as described in any one of claims 1-6; The processing module executes the data processing method for concurrent data as described in claim 7.

11. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the data processing method for concurrent data as described in any one of claims 1 to 7.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform a data processing method for concurrent data as described in any one of claims 1 to 7.

13. A computer program product, characterized in that, When the computer program product is run on a computer device, the computer device enables the computer device to implement the data processing method for concurrent data as described in any one of claims 1 to 7.

Images