High-concurrency data processing method
By generating globally unique identifier (GUID) for each thread to lock and manage data, concurrent conflicts and dirty reading problems during data transmission in a multi-threaded environment are solved, and efficient and secure data transmission is achieved.
Patent Information
- Application Number
- CN202411728117.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-03-21
AI Technical Summary
In a multi-threaded environment, concurrent conflicts and dirty reading problems are prone to occur during data transmission, resulting in inconsistent data.
By generating a globally unique identifier (GUID) for each thread, locking and managing data, ensuring that each thread processes only its unique data.
It effectively solves the concurrency and dirty reading problems during data transmission in multi-threaded environments, significantly improving the efficiency and security of data transmission.
Smart Images

Figure CN119669241A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of data transmission, and specifically relates to a high-concurrency data processing method. Background Art
[0002] As the business scope of enterprises continues to expand, their demand for online payments has also increased. Especially during business peak periods, enterprises need to process nearly one million payment transactions every day. Therefore, the requirements for payment processing efficiency are increasing day by day.
[0003] Traditional payment methods are unable to cope with a large number of payment transactions, especially during peak hours. Single-threaded data transmission methods can no longer meet the needs of enterprises. In order to improve payment processing capabilities, multi-threaded interfaces are introduced for data transmission. This method can process a large number of payment documents in a short period of time, thereby improving the overall processing speed.
[0004] Although multi-threaded data transmission can significantly increase the data processing capabilities, it also brings some new challenges. When multiple threads operate on the same data at the same time, concurrency conflicts may occur, resulting in inconsistent data or so-called "dirty reads". In addition, due to the huge amount of data, the real-time and security requirements of online payments are also more stringent, and traditional solutions perform poorly in these aspects. Summary of the invention
[0005] The present application provides a high-concurrency data processing method to solve the technical problems of concurrent processing and dirty reading in the data transmission process in a multi-threaded environment in the prior art.
[0006] The technical solution adopted in this application is:
[0007] The present application embodiment provides a high-concurrency data processing method, comprising:
[0008] Generate a thread unique code for the read data corresponding to each of the threads, the threads being used to transfer the data of the first database to the second database;
[0009] Filtering the data in the first database according to the thread unique code so that the thread only contains the read data;
[0010] The filtered data is transferred from the first database to the second database.
[0011] According to an embodiment of the present application, filtering the data in the first database according to the thread unique code so that the thread only includes the read data is specifically:
[0012] Before the read data is transmitted, the read data read again by other threads during the locking period is excluded by comparing the unique codes of each thread;
[0013] The thread only retains the read data corresponding to the thread unique code.
[0014] According to an embodiment of the present application, the generation of a thread unique code for the read data corresponding to each of the threads is specifically:
[0015] Acquire the read data that meets the preset conditions in the first database through each of the threads, and generate a thread unique code for the corresponding read data;
[0016] The thread unique code is updated to the lock field of the read data.
[0017] According to an embodiment of the present application, the read data meeting the preset conditions includes: the read data whose transmission status is not transmitted, the read data whose transmission status is transmission failed, and the read data without the thread unique code in the lock field.
[0018] According to one embodiment of the present application, after transferring the filtered data from the first database to the second database, the method further includes:
[0019] The transmission status of the read data in the first database is updated.
[0020] A high-concurrency data processing system, comprising:
[0021] A reading module, used for generating a thread unique code for the read data corresponding to each of the threads;
[0022] A filtering module, used for filtering the data in the first database according to the thread unique code, so that the thread only contains the read data;
[0023] A transmission module is used to transfer the filtered data from the first database to the second database.
[0024] According to one embodiment of the present application, it also includes:
[0025] An updating module is used to update the transmission status of the read data in the first database.
[0026] A computer program product containing instructions, when running on a device, enables the device to execute steps in a high-concurrency data processing method.
[0027] A computer-readable storage medium stores a program, which implements the steps in a high-concurrency data processing method when executed by a processor.
[0028] An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor. When the processor executes the program, steps in a high-concurrency data processing method are implemented.
[0029] Due to the adoption of the above technical solution, the beneficial effects achieved by this application are as follows:
[0030] This application uses a globally unique identifier (GUID) to lock and manage data, effectively solving concurrency problems and dirty read problems that may occur during data transmission in a multi-threaded environment. Specifically, the thread first reads qualified data records from the first database and generates a GUID to lock the data to prevent interference from other threads. Before data transmission, the thread will reconfirm the validity of the lock to ensure that the data has not been tampered with by other threads during reading and locking. After the transmission is completed, the data status is updated to "transmitted" to ensure data consistency and accuracy. This solution significantly improves the efficiency and security of data transmission. It is suitable for enterprises with large business volumes, especially for large data volume transmission scenarios across systems, ensuring the timeliness of data transmission and the office efficiency of the enterprise. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0032] Figure 1 A flowchart of a high-concurrency data processing method provided in an embodiment of the present application. DETAILED DESCRIPTION
[0033] In order to more clearly illustrate the overall concept of the present application, a detailed description is given below in an illustrative manner in conjunction with the accompanying drawings.
[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited by the specific embodiments disclosed below. It should be noted that the embodiments of the present application and the features in each embodiment may be combined with each other without conflict.
[0035] In the present application, unless otherwise clearly specified and limited, a first feature "above" or "below" a second feature may be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in an appropriate manner in any one or more embodiments or examples.
[0036] Example 1
[0037] like Figure 1 As shown, a high-concurrency data processing method includes:
[0038] A thread unique code is generated for the read data corresponding to each of the threads, and the threads are used to transfer the data of the first database to the second database.
[0039] Specifically, in a high-concurrency environment, multiple threads may access and process the same data at the same time. In order to ensure that the data processed by each thread is independent and avoid data competition and dirty read problems, a unique identifier (thread unique code) needs to be generated for each thread. This unique code can be used to identify and filter data belonging to the current thread in subsequent data processing.
[0040] The process of generating a thread unique code
[0041] Generate strategy
[0042] There are many strategies for generating thread unique codes, the most common ones include:
[0043] Thread ID: Each thread has a unique ID that can be used directly.
[0044] Timestamp + Random Number: Combine timestamp and random number to generate a unique code that is almost impossible to repeat.
[0045] UUID: Use UUID (Universally Unique Identifier) to generate a globally unique identifier.
[0046] Read data and mark the thread unique code
[0047] Reading Data
[0048] Each thread first reads the data to be processed from the first database, which may be user records, transaction records, log records, etc.
[0049] Mark thread unique code
[0050] After reading the data, the thread will mark each data item with the unique code of the current thread. This marking process is usually completed in memory, that is, setting an attribute on the read data object to store the thread unique code.
[0051] Data Lock
[0052] In order to ensure that data is not modified by other threads while being processed by the current thread, the database locking mechanism needs to be used. Common locking mechanisms include:
[0053] Pessimistic locking: Lock the data row immediately when reading data to ensure that other threads cannot modify the data.
[0054] Optimistic locking: When updating data, check whether the data has been modified by other threads. If it is found that the data has been modified, give up or try again.
[0055] For example, suppose there is an e-commerce platform that needs to transfer order data from a first database (primary database) to a second database (backup database) to ensure data security and redundancy.
[0056] background
[0057] The first database (main database): stores all real-time order data.
[0058] Second database (backup database): stores backup order data for data analysis and disaster recovery.
[0059] Target
[0060] Ensure that multiple threads will not have data competition and dirty read problems when transferring order data from the first database to the second database.
[0061] Specific steps
[0062] Generate a thread unique code
[0063] Suppose we have three threads (Thread A, Thread B, Thread C) responsible for data transmission tasks. Each thread generates a unique identifier (thread unique code) before starting work.
[0064] The unique thread code generated by Thread A is UUID-A
[0065] The unique thread code generated by Thread B is UUID-B
[0066] The unique thread code generated by Thread C is UUID-C
[0067] Read data and mark the thread unique code
[0068] Each thread reads a batch of order data from the first database and marks each piece of data with the unique code of the current thread.
[0069] Thread A reads the order data [Order1, Order2, Order3] and marks UUID-A on each order data.
[0070] Thread B reads the order data [Order4, Order5, Order6] and marks UUID-B on each order data.
[0071] Thread C reads the order data [Order7, Order8, Order9] and marks UUID-C on each order data.
[0072] Data Lock
[0073] To ensure that data is not modified by other threads while being processed by the current thread, each thread uses the database's locking mechanism when reading data.
[0074] Thread A locks [Order1, Order2, Order3] using pessimistic locking.
[0075] Thread B locks [Order4, Order5, Order6] using pessimistic locking.
[0076] Thread C locks [Order7, Order8, Order9] using pessimistic locking.
[0077] Furthermore, data integrity verification can be performed, and a data integrity verification step can be added during the data transmission from the first database to the second database, for example, calculating a hash value of the data to ensure that the data has not been tampered with or damaged during the transmission process.
[0078] Furthermore, data consistency verification can be performed after data transmission is completed to ensure that the data in the first database and the second database are completely consistent. This can be achieved by comparing the number of data records, key field values, etc. in the two databases.
[0079] The data in the first database is filtered according to the thread unique code so that the thread only includes the read data.
[0080] Specifically, data filtering
[0081] Filter purpose
[0082] After the data is locked, filtering is performed to ensure that only data marked with the current thread's unique code is processed. This step is to exclude the situation where other threads read it again during the period from when the thread reads it to when it is locked.
[0083] Filtration process
[0084] The thread will traverse the locked data and check the thread unique code on each data item. Only those data marked with the current thread unique code will be retained, and other data will be filtered out.
[0085] For example, generate a thread unique code
[0086] Each thread generates a unique identifier (thread unique code) before starting work. For example:
[0087] The unique thread code generated by Thread A is UUID-A
[0088] The unique thread code generated by Thread B is UUID-B
[0089] The unique thread code generated by Thread C is UUID-C
[0090] Read data and mark the thread unique code
[0091] Each thread reads a batch of data from the first database and marks each piece of data with the unique code of the current thread. For example:
[0092] Thread A reads the order data [Order1, Order2, Order3] and marks UUID-A on each order data.
[0093] Thread B reads the order data [Order4, Order5, Order6] and marks UUID-B on each order data.
[0094] Thread C reads the order data [Order7, Order8, Order9] and marks UUID-C on each order data.
[0095] Data Lock
[0096] To ensure that data is not modified by other threads while being processed by the current thread, each thread uses the database's locking mechanism when reading data. For example:
[0097] Thread A locks [Order1, Order2, Order3] using pessimistic locking.
[0098] Thread B locks [Order4, Order5, Order6] using pessimistic locking.
[0099] Thread C locks [Order7, Order8, Order9] using pessimistic locking.
[0100] Filter data based on thread unique code
[0101] After the data is locked, each thread will filter the data to ensure that only the data marked with the current thread's unique code is processed. This process can be divided into the following steps:
[0102] Read locked data
[0103] Each thread reads the locked data from the first database. For example:
[0104] Thread A reads the locked [Order1, Order2, Order3].
[0105] Thread B reads the locked [Order4, Order5, Order6].
[0106] Thread C reads the locked [Order7, Order8, Order9].
[0107] Filtering Data
[0108] Each thread filters out its own data based on its own thread unique code. The specific steps are as follows:
[0109] Thread A checks the thread unique code of each piece of data and only retains the data marked with UUID-A.
[0110] Check Order1 and find that its thread unique code is UUID-A, which is retained.
[0111] Check Order2 and find that its thread unique code is UUID-A, which is retained.
[0112] Check Order3 and find that its thread unique code is UUID-A, which is retained.
[0113] The final retained data is [Order1, Order2, Order3].
[0114] Thread B checks the thread unique code of each data and only retains the data marked with UUID-B.
[0115] Check Order4 and find that its thread unique code is UUID-B, which is retained.
[0116] Check Order5 and find that its thread unique code is UUID-B, which is retained.
[0117] Check Order6 and find that its thread unique code is UUID-B, which is retained.
[0118] The final retained data is [Order4, Order5, Order6].
[0119] Thread C checks the thread unique code of each data and only retains the data marked with UUID-C.
[0120] Check Order7 and find that its thread unique code is UUID-C, which is retained.
[0121] Check Order8 and find that its thread unique code is UUID-C, which is retained.
[0122] Check Order9 and find that its thread unique code is UUID-C, which is retained.
[0123] The final retained data is [Order7, Order8, Order9].
[0124] Furthermore, transaction management can be performed, using database transactions to ensure data consistency. If an exception occurs during data transmission, the transaction can be rolled back to ensure that the data is not in an inconsistent state.
[0125] Furthermore, a retry mechanism can be added. During the data transmission process, if there are abnormal situations such as network interruption and database connection failure, a retry mechanism can be set to automatically re-execute the failed operation. The number of retries and the interval time can be configured according to the actual situation.
[0126] The filtered data is transferred from the first database to the second database.
[0127] Specifically, the transfer process
[0128] The filtered data is then transferred to the second database. This is usually done through an insert or update operation on the database. The thread inserts the filtered data into the second database one by one, or in batches to improve performance.
[0129] Exception handling
[0130] During the entire data processing process, various abnormal situations need to be considered, such as network interruption, database connection failure, etc. When an exception occurs, there should be an appropriate exception handling mechanism to ensure that the system can be restored to a consistent state. Common exception handling measures include:
[0131] Transaction management: Use database transactions to ensure data consistency. If an exception occurs during processing, the transaction can be rolled back.
[0132] Retry mechanism: In some cases, a retry mechanism can be set up to automatically re-execute failed operations.
[0133] Performance Optimization
[0134] In order to improve the performance of the system, the following optimization measures can be taken:
[0135] Batch processing: Minimize the number of database reads and writes and improve efficiency through batch processing.
[0136] Cache: Use cache to reduce frequent access to the database.
[0137] Asynchronous processing: Use asynchronous processing mechanisms, such as message queues, to improve the system's responsiveness.
[0138] For example, data transfer
[0139] Each thread transfers the filtered data from the first database to the second database.
[0140] Thread A inserts [Order1, Order2, Order3] into the second database.
[0141] Thread B inserts [Order4, Order5, Order6] into the second database.
[0142] Thread C inserts [Order7, Order8, Order9] into the second database.
[0143] Furthermore, data encryption can be performed. During data transmission, sensitive data can be encrypted to ensure the security of the data during transmission.
[0144] Furthermore, you can also set up access control and set up a strict access control mechanism to ensure that only authorized threads and users can access and operate the database. You can use the database's permission management function to limit the operating permissions of different users.
[0145] In some embodiments of the present application, filtering the data in the first database according to the thread unique code so that the thread only includes the read data is specifically:
[0146] Before the read data is transmitted, the read data read again by other threads during the locking period is excluded by comparing the unique codes of each thread;
[0147] The thread only retains the read data corresponding to the thread unique code.
[0148] Specifically, data is filtered based on the thread unique code.
[0149] After the data is locked, each thread will filter the data to ensure that only the data marked with the current thread's unique code is processed. The specific steps are as follows:
[0150] Read locked data:
[0151] Thread A reads the locked [Order1, Order2, Order3].
[0152] Thread B reads the locked [Order4, Order5, Order6].
[0153] Thread C reads the locked [Order7, Order8, Order9].
[0154] Compare thread unique code:
[0155] Thread A checks the thread unique code of each piece of data and only retains the data marked with UUID-A.
[0156] Check Order1 and find that its thread unique code is UUID-A, which is retained.
[0157] Check Order2 and find that its thread unique code is UUID-A, which is retained.
[0158] Check Order3 and find that its thread unique code is UUID-A, which is retained.
[0159] The final retained data is [Order1, Order2, Order3].
[0160] Thread B checks the thread unique code of each data and only retains the data marked with UUID-B.
[0161] Check Order4 and find that its thread unique code is UUID-B, which is retained.
[0162] Check Order5 and find that its thread unique code is UUID-B, which is retained.
[0163] Check Order6 and find that its thread unique code is UUID-B, which is retained.
[0164] The final retained data is [Order4, Order5, Order6].
[0165] Thread C checks the thread unique code of each data and only retains the data marked with UUID-C.
[0166] Check Order7 and find that its thread unique code is UUID-C, which is retained.
[0167] Check Order8 and find that its thread unique code is UUID-C, which is retained.
[0168] Check Order9 and find that its thread unique code is UUID-C, which is retained.
[0169] The final retained data is [Order7, Order8, Order9].
[0170] Exclude data that is read again by other threads during the lock period:
[0171] Assume that while Thread A is locking [Order1, Order2, Order3], Thread B also reads Order1 and marks it as UUID-B.
[0172] When filtering data, Thread A checks the thread unique code of Order1 and finds that it is marked as UUID-B instead of UUID-A, so Order1 is excluded.
[0173] The data finally retained by Thread A is [Order2, Order3].
[0174] Assume that while Thread B is locking [Order4, Order5, Order6], Thread C also reads Order5 and marks it as UUID-C.
[0175] When filtering data, Thread B checks the thread unique code of Order5 and finds that it is marked as UUID-C instead of UUID-B, so Order5 is excluded.
[0176] The data finally retained by Thread B is [Order4, Order6].
[0177] Assume that while Thread C is locking [Order7, Order8, Order9], Thread A also reads Order8 and marks it as UUID-A.
[0178] When filtering data, Thread C checks the thread unique code of Order8 and finds that it is marked as UUID-A instead of UUID-C, so Order8 is excluded.
[0179] The data finally retained by Thread C is [Order7, Order9].
[0180] In some embodiments of the present application, the generation of a thread unique code for the read data corresponding to each thread is specifically:
[0181] Acquire the read data that meets the preset conditions in the first database through each of the threads, and generate a thread unique code for the corresponding read data;
[0182] The thread unique code is updated to the lock field of the read data.
[0183] Specifically, read the data and mark the thread unique code
[0184] Assume that there are three threads (Thread A, Thread B, Thread C), which respectively read a batch of data from the first database and mark their own thread unique code on each data.
[0185] Thread A reads the order data [Order1, Order2, Order3] and marks UUID-A on each order data.
[0186] Thread B reads the order data [Order4, Order5, Order6] and marks UUID-B on each order data.
[0187] Thread C reads the order data [Order7, Order8, Order9] and marks UUID-C on each order data.
[0188] Data Lock
[0189] To ensure that data is not modified by other threads while being processed by the current thread, each thread uses the database's locking mechanism when reading data.
[0190] Thread A locks [Order1, Order2, Order3] using pessimistic locking.
[0191] Thread B locks [Order4, Order5, Order6] using pessimistic locking.
[0192] Thread C locks [Order7, Order8, Order9] using pessimistic locking.
[0193] In some embodiments of the present application, the read data meeting the preset conditions includes: the read data whose transmission status is not transmitted, the read data whose transmission status is transmission failed, and the read data without the thread unique code in the lock field.
[0194] Specifically, the transfer status is the read data that has not been transferred
[0195] Definition: Read data with a transmission status of "not transmitted" refers to data that has not been processed by any thread. This data has a field (such as transmission_status) marked as "not transmitted" in the database.
[0196] Example:
[0197] Assume there is order data [Order1, Order2, Order3], their transaction_status fields are:
[0198] Order1: Not transferred
[0199] Order2: Not transferred
[0200] Order3: Transmitted
[0201] In this case, Order1 and Order2 are read data that meet the preset condition because their transmission status is Not Transmitted.
[0202] Read data with a transfer status of Failed
[0203] Definition: Read data with a transfer status of "failed" refers to data that failed in the previous transfer attempt. This data has a field (such as "transmission_status") marked as "failed" in the database.
[0204] Example:
[0205] Assume there are order data [Order4, Order5, Order6], their transaction_status fields are:
[0206] Order4: Transmission failed
[0207] Order5: Transmission failed
[0208] Order6: Transmitted
[0209] In this case, Order4 and Order5 are read data that meet the preset condition because their transmission status is transmission failure.
[0210] The lock field does not contain the read data of the thread unique code.
[0211] Definition: The read data without the thread unique code in the lock field refers to the data that is not marked by any thread in the lock field. These data have a field (such as lock_thread_id) in the database that is empty or does not contain the unique code of the current thread.
[0212] Example:
[0213] Assume there is order data [Order7, Order8, Order9], their lock_thread_id fields are:
[0214] Order7: empty
[0215] Order8: UUID-A
[0216] Order9: empty
[0217] In this case, Order7 and Order9 are read data that meet the preset conditions because their lock_thread_id fields are empty and are not marked by any thread.
[0218] In some embodiments of the present application, after transferring the filtered data from the first database to the second database, the method further includes:
[0219] The transmission status of the read data in the first database is updated.
[0220] Specifically, update the transmission status in the first database
[0221] After the data transmission is completed, each thread needs to update the transmission status of the corresponding data in the first database. The specific steps are as follows:
[0222] Determine the data that needs to be updated
[0223] Data that has been successfully transferred: The transfer status of this data needs to be updated from "Not transferred" or "Transfer failed" to "Transferred".
[0224] Data that failed to be transmitted: If an exception occurs during the data transmission process, the transmission status of this data needs to be updated from "Not transmitted" or "Transmitting" to "Transmission failed".
[0225] Update transfer status
[0226] Data successfully transferred:
[0227] Update the transmission_status field to "Transferred".
[0228] The lock_thread_id field is cleared, indicating that the data is no longer locked by any thread.
[0229] Data that failed to transfer:
[0230] Update the transmission_status field to "Transmission Failed".
[0231] The lock_thread_id field is retained so that these data can be identified during subsequent retries.
[0232] Example 2
[0233] A high-concurrency data processing system, comprising:
[0234] A reading module, used for generating a thread unique code for the read data corresponding to each of the threads;
[0235] A filtering module, used for filtering the data in the first database according to the thread unique code, so that the thread only contains the read data;
[0236] A transmission module is used to transfer the filtered data from the first database to the second database.
[0237] In some embodiments of the present application, it also includes:
[0238] An updating module is used to update the transmission status of the read data in the first database.
[0239] A computer program product containing instructions, when running on a device, enables the device to execute steps in a high-concurrency data processing method.
[0240] A computer-readable storage medium stores a program, which implements the steps in a high-concurrency data processing method when executed by a processor.
[0241] An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor. When the processor executes the program, steps in a high-concurrency data processing method are implemented.
[0242] Anything not described in this application can be achieved by adopting or drawing on existing technologies.
[0243] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
[0244] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A high-concurrency data processing method, characterized in that: The method is applied to a multi-threaded data system, and the method comprises: Generate a thread unique code for the read data corresponding to each of the threads, the threads being used to transfer the data of the first database to the second database; Filtering the data in the first database according to the thread unique code so that the thread only contains the read data; The filtered data is transferred from the first database to the second database.
2. The method according to claim 1, characterized in that The filtering of the data in the first database according to the thread unique code so that the thread only includes the read data is specifically: Before the read data is transmitted, the read data read again by other threads during the locking period is excluded by comparing the unique codes of each thread; The thread only retains the read data corresponding to the thread unique code.
3. The method according to claim 1, characterized in that: The thread unique code is generated for the read data corresponding to each thread, specifically: Acquire the read data that meets the preset conditions in the first database through each of the threads, and generate a thread unique code for the corresponding read data; The thread unique code is updated to the lock field of the read data.
4. The method according to claim 3, characterized in that: The read data meeting the preset conditions includes: the read data whose transmission status is not transmitted, the read data whose transmission status is transmission failed, and the read data without the thread unique code in the lock field.
5. The method according to claim 1, characterized in that After transferring the filtered data from the first database to the second database, the method further includes: The transmission status of the read data in the first database is updated.
6. A high-concurrency data processing system, characterized in that: include: A reading module, used to generate a thread unique code for the read data corresponding to each thread; A filtering module, used for filtering the data in the first database according to the thread unique code, so that the thread only contains the read data; A transmission module is used to transfer the filtered data from the first database to the second database.
7. The system according to claim 6, characterized in that Also includes: An updating module is used to update the transmission status of the read data in the first database.
8. A computer program product comprising instructions, which, when executed on a device, is characterized in that: Enable the device to execute the steps in the high-concurrency data processing method as described in any one of claims 1-5.
9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the high-concurrency data processing method as described in any one of claims 1 to 5 are implemented.
10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the high-concurrency data processing method as described in any one of claims 1-5 are implemented.