A high-concurrency transaction data processing system and method combined with edge computing

Abstract

The present invention discloses a high-concurrency transaction data processing system and method based on edge computing to improve the efficiency and security of transaction data processing. The method receives transaction data from multiple user terminals simultaneously through the multi-threaded concurrent receiving queue of the edge device, and performs strict integrity and legality verification on the data. The verified data is processed in pieces and intelligently distributed to multiple processing modules for parallel processing. The edge device uses a real-time parallel reduction algorithm to intelligently integrate the processing results of each module to generate transaction response data. Subsequently, the response data is compressed using an adaptive compression algorithm, synchronized to the central server through a multiple encrypted secure transmission channel, and the synchronization status is fed back to the user terminal in real time. After the data is successfully synchronized, the edge device safely deletes the local cache to ensure efficient, secure and reliable data processing.

Description

Technical Field

[0001] The present invention relates to the field of edge computing data processing technology, and in particular to a high-concurrency transaction data processing system and method combined with edge computing. Background Art

[0002] With the rapid development of the digital economy, transaction data processing systems in areas such as internet transactions, financial services, and e-commerce face unprecedented challenges. Traditional centralized data processing architectures face increasingly prominent performance bottlenecks in high-concurrency, low-latency transaction scenarios. Traditional centralized servers struggle to handle a large number of concurrent requests, leading to system response delays and limited processing capacity. When user requests surge, the system is prone to congestion and crashes, severely impacting user experience and business continuity.

[0003] Traditional architectures also have numerous shortcomings in terms of data security and network transmission. Centralized architectures present a single point of failure risk. If the central server is attacked or fails, the entire system will be paralyzed, exposing data to significant security risks. Large-scale data transmission requires significant bandwidth, and traditional transmission methods lack effective compression and security mechanisms, increasing network burdens and data transmission risks, while also hindering effective resource utilization.

[0004] Faced with these challenges, edge computing technology has emerged, offering a new path to address the limitations of traditional data processing systems. By moving computing, storage, and network resources to edge nodes close to data sources, the system's concurrent processing capabilities can be improved, latency can be reduced, data security can be enhanced, and network resource utilization can be optimized. However, existing edge computing solutions still have many limitations and have yet to fully address the complex technical challenges of processing highly concurrent transaction data. Summary of the Invention

[0005] In view of this, an embodiment of the present invention aims to provide a high-concurrency transaction data processing system and method based on edge computing to solve the above technical problems.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a high-concurrency transaction data processing method based on edge computing, the method comprising:

[0007] The edge device receives transaction data from multiple user terminals simultaneously through multi-threaded concurrent receiving queues;

[0008] The edge device performs integrity verification and legality verification on the transaction data, and stores the verified transaction data in a local cache;

[0009] The edge device shards the verified transaction data to obtain multiple data shards, and intelligently distributes the multiple data shards to multiple processing modules within the edge device;

[0010] Each processing module of the edge device processes according to the allocated data slices to obtain a processing result;

[0011] The edge device uses a real-time parallel reduction algorithm to intelligently integrate the processing results obtained by each processing module to obtain transaction response data;

[0012] The edge device compresses the transaction response data using an adaptive compression algorithm, synchronizes the compressed transaction response data to the central server through a multi-encrypted secure transmission channel, and feeds back the synchronization status to the user terminal in real time;

[0013] After confirming that the transaction response data is successfully synchronized to the central server, the edge device safely deletes the verified transaction data stored in the local cache.

[0014] In a second aspect, a high-concurrency transaction data processing system based on edge computing is provided, the system comprising:

[0015] multiple user terminals;

[0016] At least one edge device, wherein the edge device is configured with:

[0017] A receiving module is used to simultaneously receive transaction data from multiple user terminals through a multi-threaded concurrent receiving queue;

[0018] A verification storage module is used to perform integrity verification and legality verification on the transaction data and store the verified transaction data in a local cache;

[0019] A data sharding module is used to shard the verified transaction data to generate multiple data shards, and intelligently distribute the data shards to multiple processing modules within the edge device;

[0020] The parallel processing module is configured with independent computing units for parallel processing according to the allocated data slices to generate processing results;

[0021] The result reduction module uses a real-time parallel reduction algorithm to intelligently integrate the processing results obtained by each processing module to generate transaction response data;

[0022] A data transmission module is configured to compress the transaction response data using an adaptive compression algorithm; synchronize the compressed transaction response data to a central server via a multi-encrypted secure transmission channel; and provide real-time feedback of the synchronization status to the user terminal;

[0023] A data cleaning module, after confirming that the transaction response data is successfully synchronized to the central server, securely deletes the verified transaction data stored in the local cache;

[0024] Central server, configured with:

[0025] A data receiving module is used to receive transaction response data synchronized by the edge device;

[0026] The data verification module is used to perform secondary verification on the received transaction response data;

[0027] The data storage module is used to persistently store transaction response data that has passed the secondary verification;

[0028] A network communication device is used to establish a secure communication link between the user terminal, the edge device and the central server.

[0029] The above technical solution has the following beneficial effects:

[0030] The high-concurrency transaction data processing method based on edge computing proposed in this invention improves the performance and security of the transaction data processing system through innovative technologies such as multi-threaded concurrent reception, intelligent sharding, and parallel processing. The multi-threaded concurrent reception queue can simultaneously process transaction requests from multiple user terminals, significantly improving system throughput; strict integrity and legitimacy verification ensures data reliability; data sharding intelligent allocation technology achieves dynamic optimization and load balancing of computing resources; real-time parallel reduction algorithms can quickly integrate processing results and reduce system latency; adaptive compression algorithms and multiple encrypted transmission channels not only optimize network transmission efficiency, but also enhance data transmission security; and the secure deletion mechanism of transaction data further protects user privacy and system data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 This is a functional block diagram of a high-concurrency transaction data processing system based on edge computing according to an embodiment of the present invention;

[0033] Figure 2 This is a flow chart of a high-concurrency transaction data processing method based on edge computing according to an embodiment of the present invention;

[0034] Figure 3This is a flowchart of another high-concurrency transaction data processing method based on edge computing according to an embodiment of the present invention;

[0035] Figure 4 This is a functional block diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] Example 1

[0038] like Figure 1 As shown, this embodiment provides a high-concurrency transaction data processing system based on edge computing, the system comprising:

[0039] multiple user terminals;

[0040] At least one edge device, the edge device is configured with: a receiving module for simultaneously receiving transaction data from multiple user terminals through a multi-threaded concurrent receiving queue; a verification storage module for performing integrity verification and legality verification on the transaction data, and storing the verified transaction data in a local cache; a data sharding module for sharding the verified transaction data to generate multiple data shards, and intelligently assigning the data shards to multiple processing modules within the edge device; a parallel processing module, configured with an independent computing unit, for parallel processing according to the assigned data shards to generate processing results; a result reduction module, which uses a real-time parallel reduction algorithm to intelligently integrate the processing results obtained by each processing module to generate transaction response data; a data transmission module, which uses an adaptive compression algorithm to compress the transaction response data; synchronizes the compressed transaction response data to a central server through a multi-encrypted secure transmission channel; and feeds back the synchronization status to the user terminal in real time; a data cleaning module, which securely deletes the verified transaction data stored in the local cache after confirming that the transaction response data has been successfully synchronized to the central server;

[0041] The central server is configured with: a data receiving module for receiving transaction response data synchronized by edge devices; a data verification module for performing secondary verification on the received transaction response data; and a data storage module for persistently storing the transaction response data that has passed the secondary verification;

[0042] A network communication device is used to establish a secure communication link between the user terminal, the edge device and the central server.

[0043] In some embodiments, the receiving module specifically includes: a thread pool management unit, which is used to establish a multi-threaded concurrent receiving queue and configure thread pool parameters; a connection monitoring unit, which is used to monitor the connection request of the user terminal through the network socket, and assign a thread in the thread pool to each connection request for processing; an identity authentication unit, which is used to authenticate the user terminal that establishes the connection, and create a secure communication channel after the identity authentication is successful; a concurrent receiving unit, which is used to allocate threads to parallelly process transaction data receiving tasks from different user terminals through the configured thread pool, and each thread in the thread pool is bound to a user terminal to receive the data sent by the user terminal in real time; a pre-check unit, which is used to perform basic integrity and transmission security pre-check on the transaction data received through the secure communication channel, and add the transaction data that passes the pre-check to the multi-threaded concurrent receiving queue.

[0044] In some embodiments, the verification storage module specifically includes: a data extraction unit, used to extract transaction data to be verified from the multi-threaded concurrent receiving queue; a format verification unit, used to perform format integrity verification on the extracted transaction data, and verify whether the structure of the transaction data conforms to the expected format through predefined rule matching; a legitimacy verification unit, used to perform legitimacy verification on the target field in the transaction data that has passed the format verification, including data type verification, value range verification and logical consistency verification; a status marking unit, used to mark the data that has passed the verification as a passed state and the data that has failed the verification as a failed state based on the results of the format integrity verification and the legitimacy verification; a cache storage unit, used to write the transaction data marked as a passed state into the local cache and create an index for it; an exception handling unit, used to record an audit log for the transaction data marked as a failed state and store it in an isolation area.

[0045] In some embodiments, the data sharding module specifically includes: a sharding rule determination unit, which is used to determine the sharding rules based on attributes such as the type, size and processing priority of the transaction data; a data sharding unit, which is used to divide the verified transaction data into multiple data shards according to the sharding rules; a sharding label generation unit, which is used to assign a unique sharding label to each data shard, which is used to identify and track the processing status of the data shard; a resource evaluation unit, which is used to dynamically evaluate the current resource occupancy of each processing module inside the edge device to obtain a resource evaluation result; an intelligent allocation unit, which is used to determine an intelligent allocation strategy based on the attributes of the transaction data and the resource evaluation results of each processing module, and allocate multiple data shards to multiple processing modules inside the edge device according to the intelligent allocation strategy.

[0046] In some embodiments, the parallel processing module specifically includes: a shard verification unit, which is used to receive the data shards assigned to the current processing module and verify the integrity of the data shards and the consistency of their shard labels to ensure that the shard data has not been tampered with or lost; a processing algorithm selection unit, which is used to select an adaptive processing algorithm based on the operation type and content of the data shards; a business processing unit, which is used to call the processing algorithm to perform business logic processing on the data shards, including verifying business rules, performing data operations, and triggering related business operations; a result generation unit, which is used to complete the processing of the data shards and generate processing results, including transaction status, data update information, and intermediate data that can be used for subsequent integration; a result marking unit, which is used to attach timestamps and shard labels to the processing results; a result submission unit, which is used to submit the processing results with attached timestamps and shard labels to the processing result cache area, and notify the real-time parallel reduction module to prepare for integration.

[0047] In some embodiments, the result reduction module specifically includes: a result extraction unit, which is used to extract the processing results generated by each processing module from the processing result cache area, and verify the integrity of each processing result and the consistency of its timestamp and shard label; a sorting unit, which is used to perform preliminary sorting according to the original order of the transaction shards based on the shard labels to ensure the logical consistency of the processing results during the reduction process; a result merging unit, which is used to logically merge and verify multiple shard results of the same transaction based on the shard attributes and business rules of the transaction data to generate an intermediate integrated result; a parallel reduction unit, which is used to call a real-time parallel reduction algorithm, perform parallel processing on multiple intermediate integrated results, and generate transaction response data; a consistency verification unit, which is used to perform consistency verification on the transaction response data; a response cache unit, which is used to mark the transaction response data that has passed the consistency verification as available, and store it in the response cache area for subsequent compression and transmission.

[0048] In some embodiments, the data transmission module specifically includes: a data extraction unit, which is used to extract transaction response data from the response cache area and perform integrity and consistency checks on the data to ensure that the data is not damaged or tampered with; an adaptive compression unit, which is used to call an adaptive compression algorithm, dynamically adjust the compression ratio according to the size and type of the transaction response data, and compress the transaction response data; an encryption unit, which is used to securely encrypt the compressed transaction response data using multiple encryption algorithms to generate an encrypted data packet; a transmission unit, which is used to send the encrypted data packet to the central server through a pre-established secure transmission channel, and record the status information of the transmission process in real time; a confirmation unit, which is used to receive transmission confirmation information from the central server and verify whether the transaction response data is successfully synchronized to the central server; a status feedback unit, which is used to feedback the synchronization status to the user terminal in real time based on the transmission confirmation information, indicating whether the data synchronization is successful or failed.

[0049] In some embodiments, the data cleaning module specifically includes: a synchronization verification unit, used to confirm that the transaction response data is successfully synchronized to the central server; a cache cleaning unit, used to securely delete the transaction response data in the local cache of the edge device to free up storage space and ensure data privacy.

[0050] In some embodiments, the system also includes: an anomaly monitoring module, which is used to record anomaly logs and store them in an isolation area for transaction data that fails integrity verification or legality verification; when multiple consecutive anomalies are detected or the amount of abnormal data exceeds a preset threshold within a specified time, an alarm mechanism is triggered, and the abnormal status is synchronized to the central server in real time for further analysis and processing.

[0051] In some embodiments, the system further includes: a fault recovery module, which is used to enable a fault recovery mechanism and reload unfinished transaction tasks when processing is interrupted due to a fault in the edge device; transaction response data that has not been synchronized to the central server is resent through the fault recovery mechanism.

[0052] In some embodiments, the data verification module of the central server specifically includes: a signature verification unit, used to verify the validity of the digital signature of the transaction response data; an integrity check unit, used to check whether the transaction response data has been tampered with during transmission; a consistency check unit, used to compare the consistency of the transaction response data with the original transaction request; and a risk assessment unit, used to perform risk level assessment on the transaction response data and identify potential abnormal or suspicious transactions.

[0053] The high-concurrency transaction data processing system based on edge computing proposed in the present invention improves the performance and security of transaction data processing through multi-threaded concurrent reception, intelligent data sharding, parallel processing and real-time parallel reduction. The multi-threaded concurrent reception queue enables the system to process a large number of transaction requests from multiple user terminals at the same time, improving the system's throughput and response speed; data sharding and intelligent allocation mechanisms ensure the efficient use of computing resources, and maximize the parallel computing capabilities of edge devices through dynamic evaluation and precise allocation; the real-time parallel reduction algorithm realizes the rapid integration of processing results, ensuring the real-time and consistency of transaction data processing; at the same time, multiple encryption and adaptive compression technologies provide security for data transmission, effectively preventing security risks during data transmission; the secondary verification mechanism of the central server further enhances the data reliability and security of the system, ensuring the integrity and consistency of transaction data.

[0054] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0055] Example 2

[0056] like Figure 2 As shown, this embodiment provides a high-concurrency transaction data processing method based on edge computing, the method comprising:

[0057] S10: The edge device receives transaction data from multiple user terminals simultaneously through a multi-threaded concurrent receiving queue;

[0058] In this embodiment, a multi-threaded concurrent receive queue refers to an edge device equipped with a network interface and thread management mechanism capable of high concurrent processing. For example, when 100 user terminals initiate transaction requests simultaneously, the system can quickly and concurrently receive and cache these transaction data through a pre-configured thread pool (e.g., 20-50 worker threads). Each thread independently handles one or more connections, improving the system's concurrent processing capabilities and effectively avoiding the blocking and latency issues associated with traditional single-threaded mode.

[0059] S20: The edge device performs integrity verification and legality verification on the transaction data, and stores the verified transaction data in a local cache;

[0060] Integrity verification verifies whether data has been tampered with during transmission by calculating a hash value or checksum. Legality verification includes checking the transaction data's format, field integrity, data type, and value range. For example, for financial transactions, the system verifies whether the transaction amount is within a reasonable range, the transaction account is valid, and the transaction time is within the permitted range. These rigorous checks effectively filter out illegal or incomplete transaction requests, improving system security and reliability.

[0061] S30: The edge device shards the verified transaction data to obtain multiple data shards, and intelligently distributes the multiple data shards to multiple processing modules within the edge device;

[0062] Edge devices split transaction data into multiple logically independent data shards based on their characteristics and size. An intelligent allocation algorithm considers the current load, computing power, and specialized characteristics of each processing module to assign different types of data shards to the most appropriate processing module. For example, for complex financial transactions, data from different stages, such as risk assessment, account verification, and transaction authorization, is allocated to dedicated processing modules for parallel and efficient processing.

[0063] S40: Each processing module of the edge device processes according to the allocated data slices to obtain a processing result;

[0064] Each processing module is equipped with specialized hardware resources and algorithms. For example, the risk assessment module uses machine learning algorithms to analyze transaction anomalies in real time, the account verification module invokes identity authentication services, and the transaction authorization module verifies transaction compliance. These modules can be powered by GPUs, FPGAs, or dedicated AI chips, enabling high-performance parallel computing and significantly improving processing efficiency.

[0065] S50: The edge device intelligently integrates the processing results obtained by each processing module using a real-time parallel reduction algorithm to obtain transaction response data;

[0066] The real-time parallel reduction algorithm is an efficient result integration technology that quickly merges computational results from different processing modules. This algorithm is similar to the reduction operation in MapReduce, but is more lightweight and real-time. For example, for a transaction, the risk assessment module assigns a risk rating, the account verification module confirms account validity, and the transaction authorization module checks compliance. The reduction algorithm quickly integrates these results according to pre-set rules to generate the final transaction response.

[0067] S60: The edge device compresses the transaction response data using an adaptive compression algorithm, synchronizes the compressed transaction response data to the central server through a multi-encrypted secure transmission channel, and feeds back the synchronization status to the user terminal in real time;

[0068] An adaptive compression algorithm dynamically selects the optimal compression scheme based on the characteristics of transaction response data, minimizing transmission volume while ensuring data integrity. Multiple encryption channels utilize multi-layered encryption protocols, such as TLS and IPSec, combined with custom encryption algorithms to ensure secure data transmission. Real-time synchronization status feedback utilizes WebSocket or similar real-time communication technologies, allowing users to instantly monitor transaction processing progress.

[0069] S70: After confirming that the transaction response data is successfully synchronized to the central server, the edge device securely deletes the verified transaction data stored in the local cache.

[0070] Secure deletion goes beyond simply erasing data; it also involves using multiple overwrites and cryptographic zeroing to ensure that sensitive transaction data cannot be recovered. This step is a crucial mechanism for protecting user privacy and system security, preventing potential data leaks.

[0071] In some embodiments, step S10 specifically includes the following steps:

[0072] S101: Establish a multi-threaded concurrent receiving queue and configure thread pool parameters; the thread pool is used to manage thread resources in the multi-threaded concurrent receiving queue;

[0073] In this embodiment, for example, for high-concurrency transaction scenarios, the edge device can pre-configure a thread pool of 20-50 worker threads and dynamically adjust the number of threads based on real-time load. Thread pool parameters include the number of core threads, maximum number of threads, thread idle time, and task queue length. Through detailed parameter tuning, an optimal balance can be achieved between system resource utilization and concurrent processing capabilities, avoiding the performance overhead of thread creation and destruction.

[0074] S102: monitoring the connection request of the user terminal through the network socket, and assigning a thread in the thread pool to process each connection request;

[0075] Network sockets are the key communication mechanism connecting user terminals and edge devices. Edge devices use high-performance, non-blocking I / O models, such as Netty or epoll, to efficiently monitor and process connection requests from multiple user terminals. When a new connection request arrives, the thread pool immediately allocates an idle thread to establish a dedicated network connection channel. For example, in a financial transaction scenario, a single thread can simultaneously handle different transaction requests from a bank app, a website, and an ATM.

[0076] S103: Authenticate the user terminal establishing the connection, and create a secure communication channel after successful authentication.

[0077] Identity authentication is the first line of defense for transaction security. This embodiment utilizes a multi-factor authentication mechanism, combining traditional passwords, dynamic tokens, biometrics, and other authentication methods. For example, users are not only required to enter their password but also to perform secondary verification via SMS verification code, fingerprint, or facial recognition. After successful authentication, the system establishes an end-to-end encrypted communication channel using the TLS / SSL protocol, ensuring the security and integrity of subsequent transaction data transmission.

[0078] S104: Allocate threads through the configured thread pool to process transaction data receiving tasks from different user terminals in parallel. Each thread in the thread pool is bound to a user terminal and receives data sent by the user terminal in real time.

[0079] Each thread is dedicated to receiving data from a single user terminal, enabling true parallel processing. For example, if 50 users initiate transaction requests simultaneously, 50 independent threads can simultaneously receive and process these requests, improving system throughput. Each thread is equipped with its own independent data buffer and processing logic, minimizing inter-thread contention and blocking.

[0080] S105: Perform a basic integrity and transmission security pre-check on the transaction data received through the secure communication channel, and add the transaction data that passes the pre-check to a multi-threaded concurrent receiving queue.

[0081] Pre-checking is a lightweight security filtering performed before formal data verification. The system quickly checks the basic integrity of the data packet, such as data length, basic format, and transmission protocol consistency. It also performs transmission security checks, including detecting anomalies such as tampering and replay attacks. Only data that passes these pre-checks is added to the multi-threaded concurrent receive queue, effectively blocking most potential security threats.

[0082] The advantages of this solution are: through the multi-threaded concurrent receiving architecture, the system's ability to handle high-concurrency transaction requests is improved, and the transaction needs of a large number of users can be handled simultaneously, greatly improving the user experience; the multi-level security authentication and pre-check mechanism embeds strict security controls in every link from connection establishment to data reception, effectively protecting the security of user transaction data; the dynamic management of the thread pool and the precise binding of threads to user terminals achieve efficient utilization and optimal allocation of computing resources, reducing system resource consumption; the solution has good scalability and adaptability, and can flexibly adjust thread pool parameters and processing strategies according to different business scenarios and load requirements.

[0083] In some embodiments, step S20 specifically includes the following steps:

[0084] S201: extracting transaction data to be verified from the multi-threaded concurrent receiving queue;

[0085] In this embodiment, data extraction is an efficient, non-blocking operation. The system uses lock-free queues (e.g., ConcurrentLinkedQueue) for rapid data extraction, ensuring high performance in a multi-threaded environment. For example, in a financial transaction scenario, the system can extract hundreds or even thousands of transaction data items per second from the concurrent receive queue and pass them to the subsequent verification module. The extraction process utilizes a batch read strategy, enabling the simultaneous acquisition of multiple pieces of data to be verified, further improving system processing efficiency.

[0086] S202: Perform format integrity check on the extracted transaction data, verifying whether the structure of the transaction data conforms to the expected format through predefined rule matching. If the transaction data does not conform to the expected format, it is directly marked as failed;

[0087] The system pre-defines strict data format templates, including mandatory fields, field order, and delimiters. For example, for financial transactions, transaction data must include fixed fields such as transaction type, transaction amount, account ID, and timestamp, each of which has specific format requirements. If any key field is missing or formatted incorrectly, the system will immediately mark the transaction data as failed, preventing non-compliant data from entering subsequent processing.

[0088] S203: Perform a validity check on the target fields in the transaction data that has passed the format check. This includes the following: data type check, used to verify whether the data type of the target field complies with the preset data type; value range check, used to verify whether the numeric field is within the preset range; logical consistency check, used to verify whether the logical relationship between the target fields is reasonable;

[0089] Data type validation ensures that the data type of each field matches precisely. For example, transaction amounts must be numeric and account IDs must be strings. Value range validation prevents data anomalies. For example, transaction amounts must be between 0 and 1,000,000, and transaction times must be within a certain range before and after the current system time. Logical consistency validation is more complex and requires analyzing the relationships between multiple fields. For example, for cross-border transactions, it's necessary to verify the logical consistency of the transaction currency, exchange rate, and actual amount to prevent fraud.

[0090] S204: Based on the results of the format integrity check and the legality check, mark the data that has passed the check as passed, and mark the data that has failed the check as failed;

[0091] The system uses enumerations or status codes to mark the verification results of each transaction data, such as "PASS", "FAIL_FORMAT", "FAIL_TYPE", "FAIL_RANGE", etc. This fine-grained status marking not only facilitates subsequent processing but also provides detailed information for auditing and security analysis.

[0092] S205: Write the transaction data marked as passed into the local cache and create an index for it;

[0093] The local cache uses high-performance key-value storage technology, such as Redis or local memory cache. Each verified transaction is assigned a unique transaction ID as the primary key, and multi-dimensional indexes are established, such as by time, account, and transaction type. This indexing mechanism makes subsequent data processing and retrieval fast and efficient. For example, all transaction records for a specific account within a specific time period can be quickly located in milliseconds.

[0094] S206: Record an audit log for the transaction data marked as failed and store it in an isolated area.

[0095] Audit logs include complete details of data verification failures, including the reason for failure, data content, source IP address, and timestamp. The quarantine area is a secure storage space for data that fails verification, facilitating subsequent security analysis and forensics. The system regularly conducts risk assessments on quarantined data and triggers security alerts when necessary.

[0096] The advantages of this solution are: through a multi-level, fine-grained data verification mechanism, it fundamentally prevents non-compliant and unsafe transaction data from entering the system's core processing flow, thereby improving the system's security and reliability; flexible data status marking and multi-dimensional index design not only facilitate real-time data processing, but also provide a solid data foundation for subsequent big data analysis and risk control; the design of audit logs and isolation areas reflects the system's comprehensive consideration of data security, providing strong support for security traceability and risk management; the entire verification process adopts a high-performance, non-blocking design, which can maintain extremely low latency and high throughput even in high-concurrency scenarios with thousands of transactions per second.

[0097] In some embodiments, step S30 specifically includes the following steps:

[0098] S301: Determine sharding rules based on attributes of transaction data including type, size, and processing priority;

[0099] In this embodiment, sharding rules are a dynamic and intelligent decision-making process. The system formulates fine-grained sharding strategies based on the multi-dimensional attributes of transaction data. For example, for financial transaction scenarios, different sharding granularities can be established based on transaction type (e.g., payment, transfer, wealth management), transaction amount (small, medium, large), and customer level (standard, VIP). Small-value, high-frequency transactions use finer sharding, while large, complex transactions require larger sharding units to ensure that each shard is fully and accurately processed.

[0100] S302: Divide the verified transaction data into multiple data shards according to the sharding rule;

[0101] The system can employ intelligent sharding algorithms, segmenting transaction data into multiple logical blocks based on predefined rules. For example, a complex cross-border transaction could be broken down into multiple independent data shards, such as those for exchange rate calculation, fund verification, and compliance checks. The granularity and method of sharding can be dynamically adjusted to accommodate transaction data of varying types and complexities.

[0102] S303: Assign a unique shard label to each data shard to identify and track the processing status of the data shard;

[0103] Each shard tag contains a globally unique identifier (GUID), the original transaction ID, the shard sequence number, a creation timestamp, and other information. For example, a complete shard tag would look like "TXN-20241201-123456-SHARD-001." This design not only facilitates real-time tracking of each shard's processing status but also provides a reliable tracking mechanism for subsequent data reassembly, auditing, and fault recovery.

[0104] S304: Dynamically evaluate the current resource usage of each processing module within the edge device to obtain a resource evaluation result;

[0105] The system continuously monitors key metrics such as CPU utilization, memory usage, and processing queue length for each processing module. For example, for edge devices with multiple specialized processing modules (such as risk control, encryption, and routing), the system can collect resource usage information for each module in real time and create a dynamic resource usage profile. This real-time, fine-grained resource monitoring enables the system to make optimal allocation decisions.

[0106] S305: Determine an intelligent allocation strategy based on the attributes of the transaction data and the resource evaluation results of each processing module;

[0107] Transaction data attributes are a multidimensional collection of information describing transaction characteristics. These attributes comprehensively cover the essential characteristics of a transaction, including transaction type, financial scenario, transaction amount level, risk characteristics, time dimension, and technical attributes. By meticulously characterizing these attributes, the system constructs a multidimensional transaction feature vector, providing rich and accurate input for subsequent intelligent allocation.

[0108] Transaction data attributes can be specifically categorized into four main dimensions: transaction type, risk, time, and technical attributes. Regarding transaction type, the system considers various transaction scenarios, including payments, transfers, and top-ups. Risk attributes build risk profiles by assessing transaction risk levels, anomalies, and account credibility. Time attributes focus on the timing and frequency of transactions. Technical attributes include key metrics such as data complexity, encryption level, and data size. This multi-dimensional description provides a comprehensive and detailed feature foundation for subsequent intelligent allocation.

[0109] The professionalism of a processing module is precisely characterized through quantitative indicators, primarily encompassing three dimensions: professional capability index, performance index, and learning ability. The professional capability index reflects the module's processing capabilities in specific transaction scenarios; performance index focuses on the module's processing efficiency and resource utilization; and learning ability assesses the module's adaptability to new scenarios and its ability to iterate algorithms.

[0110] The intelligent allocation strategy is a dynamic matching algorithm based on machine learning. Taking transaction data attribute vectors and processing module resource status as input, it achieves the optimal mapping of transactions to processing modules through steps such as feature standardization, multi-dimensional matching scoring, and dynamic weight adjustment. For example, for high-risk, large-value cross-border transactions, the system prioritizes specialized modules with sophisticated risk control capabilities; whereas, for standard, low-value payments, it can be allocated to more general modules with greater processing capabilities.

[0111] By incorporating reinforcement learning, clustering algorithms, and predictive models, the system can dynamically adjust allocation strategies based on real-time feedback from allocation results. Reinforcement learning enables the system to learn from historical allocation experience; clustering algorithms help identify transaction patterns and module matching characteristics; and predictive models predict the optimal allocation plan based on historical data. This multi-algorithm collaborative learning mechanism enables allocation strategies to adapt and continuously optimize, enabling them to rapidly respond to changing business scenarios and technical environments.

[0112] S306: Allocate the multiple data shards to the multiple processing modules inside the edge device according to the intelligent allocation strategy.

[0113] Shard allocation is a precise and efficient process. The system utilizes a lightweight scheduler that intelligently routes data shards in milliseconds. This allocation process not only considers current resource availability but also anticipates each module's processing power and response time. For example, for high-concurrency trading systems, shard allocation can be dynamically adjusted to ensure a balanced workload for each processing module, avoiding single performance bottlenecks.

[0114] The advantages of this solution are: through multi-dimensional, intelligent data sharding and allocation mechanisms, the parallel capability and system throughput of edge devices in processing complex transactions are improved, and it can cope with high-concurrency, heterogeneous transaction scenarios; dynamic resource evaluation and intelligent allocation strategies enable the system to balance computing resources in real time and accurately, maximize system performance, and reduce resource waste; unique shard labels and fine-grained tracking mechanisms not only provide data traceability, but also support system security audits and fault recovery; the solution is highly scalable and adaptable, and can flexibly adjust sharding and allocation strategies according to different business scenarios and load characteristics.

[0115] In some embodiments, step S40 specifically includes the following steps:

[0116] S401: Receive the data shards assigned to the current processing module and verify the integrity of the data shards and the consistency of their shard labels to ensure that the shard data has not been tampered with or lost;

[0117] In this embodiment, the system employs multiple verification mechanisms, including hash verification, digital signatures, and blockchain technology. For example, for each data shard, the system calculates and stores a SHA-256 hash value of its contents and compares it with the original hash value in the shard tag. Simultaneously, the shard's digital signature is verified using public key cryptography to ensure that the data has not been tampered with during transmission. If any inconsistency is detected, the system immediately rejects the shard and triggers a security alert.

[0118] S402: Select an appropriate processing algorithm based on the operation type and content of the data shard;

[0119] The system has established a flexible algorithm mapping library, dynamically selecting the most appropriate processing algorithm based on the characteristics of data shards. For example, in financial transaction scenarios, different types of transactions (such as payments, transfers, and financial management) are matched with different dedicated processing algorithms. The system also supports dynamic loading of machine learning algorithms, intelligently adjusting processing strategies based on real-time data characteristics, achieving more precise business logic processing.

[0120] S403: Calling the processing algorithm to perform business logic processing on the data shards, which includes verifying business rules, performing data operations, and triggering related business operations;

[0121] The processing process isn't just a simple data conversion; it involves a complex, multi-step verification and calculation process. For example, for a cross-border payment transaction, the processing algorithm might include compliance checks (anti-money laundering, sanctions lists), exchange rate calculations, real-time risk assessments, account balance verification, and tax calculations. Each step is rigorously verified by business rules to ensure the legitimacy and accuracy of the transaction.

[0122] S404: Complete the data sharding process and generate a processing result, which includes the transaction status, data update information, and intermediate data that can be used for subsequent integration;

[0123] The results generated by the system include not only the final status of the transaction (e.g., success, failure, pending confirmation), but also a wealth of intermediate processing data. For example, for a transaction, the processing result includes: transaction unique identifier, processing time, involved account information, actual transaction amount, exchange rate information, risk control score, etc.

[0124] S405: Add a timestamp and a fragmentation tag to the processing result to obtain the processing result with the timestamp and fragmentation tag added;

[0125] The system adds a microsecond-accurate timestamp to each processing result, recording the precise moment of processing. Shard tags retain the full context of the original data shard. For example, a complete processing result tag is: "TXN-20241201-123456-SHARD-001-PROCESSED-2024-12-01T12:34:56.789Z." This design not only facilitates precise tracking but also provides a reliable basis for subsequent data auditing and replay.

[0126] S406: Submit the processing result with the timestamp and shard tag attached to the processing result cache area, and notify the real-time parallel reduction module to prepare for integration.

[0127] The system uses a high-performance distributed cache (such as Redis) as a temporary storage area for processing results. Each processing result is accurately stored and indexed, awaiting subsequent reduction processing. The notification mechanism adopts a publish-subscribe model, allowing the real-time parallel reduction module to immediately receive new processing results and prepare for data integration. This design ensures real-time and efficient data processing.

[0128] The advantages of this solution are: through multi-level data integrity verification and intelligent algorithm selection, the security and accuracy of system processing are improved, and it can cope with complex and changing business scenarios; the fine-grained processing result design and rich intermediate data not only meet immediate business needs, but also provide a data foundation for subsequent big data analysis and intelligent decision-making; the precise timestamp and sharding label mechanism realizes the complete tracking and verifiability of the data processing process, and provides support for system auditing and compliance; the flexible caching and notification mechanism ensures the real-time and efficient data processing, and has strong scalability and adaptability.

[0129] In some embodiments, step S50 specifically includes the following steps:

[0130] S501: Extract the processing results generated by each processing module from the processing result cache area, and verify the integrity of each processing result and the consistency of its timestamp and shard tag;

[0131] In this embodiment, the system employs multiple verification mechanisms to not only check the physical integrity of data but also verify its logical consistency. For example, for a complex financial transaction, the system compares the timestamps of each shard result to ensure the proper processing sequence. Simultaneously, the integrity of each result is verified using a cryptographic hash algorithm, ensuring that any minor data tampering is immediately detected and rejected. The verification process also includes cross-referencing shard tags to ensure that all results belong to different processing stages of the same transaction.

[0132] S502: Perform preliminary sorting based on the original order of the transaction shards according to the shard labels to ensure the logical consistency of the processing results during the reduction process;

[0133] The system has designed an intelligent sorting algorithm that not only uses timestamps but also considers the sequence information in shard tags. For example, a cross-border payment transaction may include multiple shards, such as risk assessment, exchange rate calculation, and funds verification. The sorting algorithm accurately restores the original order of transaction processing based on predefined business process logic. This approach ensures that even with parallel processing, the final data integration adheres to a strict logical order.

[0134] S503: Based on the sharding attributes and business rules of the transaction data, multiple sharding results of the same transaction are logically merged and verified to generate an intermediate integration result;

[0135] The system utilizes a sophisticated rules engine and intelligent merging algorithms, going beyond simple data consolidation to include in-depth logical validation. For example, for an international trade payment, the system must merge processing results from various modules, including risk control, compliance, finance, and tax. The merging process checks the consistency of the results across each shard, resolves conflicts, and generates a comprehensive intermediate result. This process, like putting together a jigsaw puzzle, ensures the final result is comprehensive and accurate.

[0136] S504: Invoke a real-time parallel reduction algorithm to process multiple intermediate integration results in parallel and generate transaction response data;

[0137] The system utilizes a real-time parallel reduction algorithm capable of processing multiple intermediate integration results simultaneously. For example, for large-scale financial trading systems, it can simultaneously process the intermediate results of hundreds or even thousands of transactions. This reduction algorithm not only prioritizes computational efficiency but also ensures data processing accuracy. It dynamically adjusts computing resources, flexibly allocating processing power based on the complexity of different transactions, achieving efficient and accurate data integration.

[0138] The Real-time Parallel Reduction Algorithm (RTPRA) is a distributed computing algorithm specifically designed for large-scale, highly concurrent data computation scenarios. Its goal is to quickly and efficiently aggregate dispersed intermediate computation results into a single, accurate final result. The algorithm's uniqueness lies in its parallel processing capabilities, enabling it to process multiple data streams simultaneously and complete complex data reduction operations in milliseconds. This technology is more than just a computational method; it represents an intelligent data processing paradigm that dynamically adjusts computing resources and flexibly allocates processing power based on task complexity.

[0139] The algorithm's operating principles can be summarized in three stages: data sharding and parallel processing, intermediate result aggregation, and dynamic resource scheduling. During the data sharding stage, the algorithm intelligently splits large datasets into multiple relatively independent subsets, each of which can be processed in parallel on distributed computing nodes. This sharding strategy ensures a high degree of computational parallelism, enabling the system to process hundreds or even thousands of data fragments simultaneously. Intermediate result aggregation utilizes a tree-based reduction strategy, gradually integrating distributed computational results into a final unified result through multi-level, step-by-step merging. The dynamic resource scheduling mechanism assesses the load of each computing node in real time and dynamically allocates and balances computing resources based on the complexity of the task and the real-time status of system resources.

[0140] S505: Perform consistency verification on the transaction response data;

[0141] The system incorporates a multi-layered verification mechanism, including mathematical model validation, business rule checking, and machine learning anomaly detection. For example, for financial transactions, the verification process checks the rationality of transaction amounts, the consistency of account balances, and the accuracy of tax calculations. The system also conducts cross-module cross-validation to ensure that data generated by different processing steps is free of logical inconsistencies. This comprehensive verification approach enhances the reliability of the system's output data.

[0142] S506: Mark the transaction response data that has passed the consistency verification as available and store it in the response cache area for subsequent compression and transmission.

[0143] The system utilizes high-performance distributed caching technology to assign a unique identifier to each verified transaction response. This storage process not only ensures data persistence but also optimizes subsequent data compression and transmission. For example, the system can select the most appropriate compression algorithm based on data characteristics, reducing storage space and transmission bandwidth usage. Furthermore, the cache area supports fine-grained access control and audit trails, providing strong assurance for data security and compliance.

[0144] The advantages of this solution are: through a multi-level, intelligent data verification and merging mechanism, the accuracy and reliability of the system in processing complex transactions are improved, and it can cope with highly dynamic and complex business scenarios; advanced parallel reduction algorithms and distributed computing technologies have greatly improved the system's processing performance, and it can handle a large number of complex transactions simultaneously and has scalability; sophisticated consistency verification and intelligent sorting mechanisms not only ensure the logic of data processing, but also provide a high-quality data foundation for subsequent data analysis and decision-making; flexible caching and storage design achieves efficient data compression, secure transmission and traceability.

[0145] In some embodiments, step S60 specifically includes the following steps:

[0146] S601: Extract transaction response data from the response buffer area and perform integrity and consistency checks on the data to ensure that the data is not damaged or tampered with;

[0147] In this embodiment, the system employs a multi-level verification mechanism, including hash value comparison, digital signature verification, and blockchain technology. For example, for a financial transaction response, the system first calculates the data's SHA-256 hash value and compares it with the stored original hash value. Simultaneously, the data's digital signature is verified using a public key infrastructure (PKI) to ensure the data has not been tampered with during transmission. This multi-layered verification approach quickly identifies any data integrity issues, providing a reliable data foundation for subsequent processing.

[0148] S602: Invoke an adaptive compression algorithm to dynamically adjust the compression ratio according to the size and type of the transaction response data and compress the transaction response data;

[0149] The system incorporates an intelligent compression algorithm that dynamically selects the optimal compression strategy based on data characteristics. For example, for different types of transaction response data (e.g., text, images, and videos), the system automatically selects the most appropriate compression algorithm, such as Huffman coding for text and JPEG compression for images. The compression algorithm also considers the sensitivity and importance of the data, minimizing its size while ensuring data quality. This intelligent compression approach not only saves storage space and transmission bandwidth, but also improves overall system performance.

[0150] S603: Securely encrypt the compressed transaction response data using multiple encryption algorithms to generate an encrypted data packet;

[0151] The system employs a layered encryption strategy, combining symmetric and asymmetric encryption techniques. For example, data is first symmetrically encrypted using AES-256, and then the AES key is encrypted using the asymmetric RSA-2048 algorithm. This multi-layered encryption approach provides multi-layered security, ensuring data security even if one layer is compromised. The encryption process also dynamically generates a one-time key, further increasing the difficulty of cracking. For sensitive financial transaction data, this high-strength encryption scheme effectively prevents data leakage and unauthorized access.

[0152] S604: Send the encrypted data packet to the central server through the pre-established secure transmission channel, and record the status information of the transmission process in real time;

[0153] The system utilizes a secure transmission protocol based on TLS 1.3 to establish an end-to-end encrypted communication channel. Dynamic routing and multi-path transmission technologies are used during the transmission process to automatically select the most secure and efficient network path. For example, for cross-border financial transactions, the system can monitor network conditions in real time and dynamically adjust transmission routes. Furthermore, detailed status information is recorded at each key node in the transmission process, including transmission time, network latency, and packet integrity, providing crucial data for subsequent audits and optimization.

[0154] S605: Receive transmission confirmation information from the central server to verify whether the transaction response data is successfully synchronized to the central server;

[0155] The system incorporates an intelligent confirmation mechanism that not only verifies successful data transmission but also checks data integrity and consistency. For example, the central server returns a confirmation message containing a data hash, a transmission timestamp, and a checksum. Edge devices then perform multiple verifications on this information to ensure that the data has been fully and accurately synchronized to the central server. If any anomalies are detected, the system automatically triggers a retransmission mechanism to ensure reliable data synchronization.

[0156] S606: Based on the transmission confirmation information, the synchronization status is fed back to the user terminal in real time, indicating whether the data synchronization is successful or failed;

[0157] The system features an intelligent notification mechanism that generates personalized user notifications based on different synchronization statuses. For example, a successful transaction will receive a concise success notification; failures will be accompanied by detailed error messages and suggested solutions. Notifications are available through various channels, including SMS, in-app notifications, and email, ensuring users are kept up to date on the latest data synchronization status.

[0158] S607: After confirming that the transaction response data is successfully synchronized to the central server, the transaction response data in the local cache of the edge device is securely deleted to free up storage space and ensure data privacy.

[0159] The system employs multiple data erasure techniques to ensure that sensitive data in the local cache is completely and irreversibly deleted. For example, multiple overwrites are used to overwrite the original data with random data multiple times, completely eliminating any traces of the data. The deletion process is also logged to facilitate subsequent compliance audits. This secure deletion method not only frees up storage space on edge devices but also maximizes user data privacy.

[0160] The advantages of this solution are: through multi-level data verification, adaptive compression and multiple encryption technologies, a comprehensive data security protection system is built, which can effectively protect the integrity and confidentiality of sensitive transaction data; the intelligent transmission and synchronization mechanism, combined with dynamic routing and multi-path transmission technology, improves the reliability and efficiency of data transmission, and can ensure accurate data synchronization even in complex network environments; real-time status feedback and personalized notification mechanisms enhance the user experience, allowing users to understand the status of data synchronization in real time; secure deletion technology not only protects data privacy, but also optimizes the storage resources of edge devices, with security, efficiency and user-friendliness.

[0161] like Figure 3 As shown, in some embodiments, the method further includes:

[0162] S80: The edge device records an exception log for transaction data that fails the integrity check or the legality check and stores it in an isolation area; when multiple consecutive exceptions are detected or the amount of abnormal data exceeds a preset threshold within a specified time, an alarm mechanism is triggered and the abnormal status is synchronized to the central server in real time for further analysis and processing.

[0163] In this embodiment, the edge device first establishes a dedicated data isolation zone to store transaction data that fails integrity and legality checks. This isolation zone employs strict access control and encrypted storage technology to ensure that abnormal data does not impact the normal operation of the system. For example, if a financial transaction fails hash value verification or digital signature verification, the system immediately moves the data to the isolation zone and generates a detailed exception log.

[0164] Exception logging not only records basic information about the abnormal data, but also includes detailed information such as the timestamp of the abnormality, the abnormality type, the data source, and the initial diagnosis. The system uses a structured logging format to facilitate subsequent data analysis and security audits. For example, for a detected data integrity anomaly, the log will include key information such as the abnormality time, the data source IP address, the original data hash value, the expected hash value, and the abnormality type (e.g., data tampering, transmission corruption).

[0165] The system incorporates an intelligent anomaly threshold monitoring algorithm that dynamically analyzes the frequency and patterns of abnormal data. Alarms are triggered when any of the following conditions are detected: the number of abnormal data occurrences exceeds 5% of total transaction data within a five-minute period; the cumulative volume of abnormal data exceeds 100 within an hour; or three consecutive occurrences of abnormal data from the same data source are detected. Once an alarm is triggered, the system generates a high-priority alert and synchronizes it to the central server in real time via multiple channels.

[0166] The system utilizes an efficient real-time communication protocol to ensure rapid and reliable transmission of alerts to the central server. Alerts not only include detailed descriptions of abnormal data but also provide preliminary risk assessments and recommended response strategies. For example, for a detected suspicious data source, the alert includes a recommended IP address blocking strategy, a preliminary threat level assessment, and recommended further investigation directions. Upon receiving the alert, the central server can immediately initiate security response processes, such as triggering intrusion detection systems, initiating security audits, or implementing preventative network isolation measures.

[0167] To enhance the system's adaptability and learning capabilities, the anomaly data processing mechanism also integrates machine learning technology. The system continuously analyzes anomaly logs to identify potential attack patterns and system vulnerabilities. Through in-depth analysis of historical anomaly data, the system can continuously optimize anomaly detection algorithms and improve its ability to identify and defend against new security threats. For example, the system can use anomaly detection algorithms such as Isolation Forest or Local Outlier Factor (LOF) to automatically learn and identify characteristic patterns of data anomalies.

[0168] The advantages of this solution are: by establishing a dedicated data isolation area and a detailed exception logging mechanism, the system can comprehensively and accurately capture and manage potential security threats, effectively preventing the negative impact of abnormal data on the system; the intelligent anomaly detection and alarm mechanism, combined with dynamic threshold monitoring and multi-channel alarm synchronization, can achieve rapid response and active defense to security threats, improving the security resilience of the system; by integrating machine learning technology, the system has the ability to continuously learn and self-optimize, and can continuously improve the level of identification and defense against new security threats.

[0169] In some embodiments, the method further comprises:

[0170] S90: When the edge device is interrupted due to a fault, the fault recovery mechanism is enabled to reload the unfinished transaction tasks; the transaction response data that has not been synchronized to the central server is resent through the fault recovery mechanism.

[0171] In this embodiment, the system has designed a multi-layered fault detection and recovery framework that can quickly identify and address various edge device faults. The fault detection module continuously monitors the device's operating status, including key indicators such as hardware performance, system resources, and network connectivity. When an abnormality is detected that could affect system operation, such as excessive CPU load, memory exhaustion, or network loss, the system immediately triggers the fault recovery process.

[0172] The system uses real-time task status snapshot technology to regularly record the execution progress, key parameters, and intermediate status of each transaction task. This snapshot mechanism not only records basic task information but also includes detailed execution context. For example, for a complex financial transaction, the system will record key information such as the initial transaction parameters, completed processing steps, unfinished subtasks, and the current processing stage.

[0173] When an edge device detects a failure, the system accurately locates the interrupted transaction task based on a task state snapshot and automatically initiates a reload process. This reload process includes the following steps: restoring the task execution environment from the most recent valid state snapshot; reinitializing unfinished transaction tasks; automatically adjusting task execution parameters to adapt to the current system state; and resuming unfinished transaction processing from the point of interruption. For example, in the case of an interrupted cross-border payment transaction, the system can precisely restart from the last confirmed processing step, ensuring transaction continuity and accuracy.

[0174] The system has designed an intelligent data retransmission strategy to reliably handle transaction response data that fails to synchronize with the central server. This retransmission mechanism includes the following core functions: establishing a priority queue for unsynchronized data; automatically detecting and filtering data to be retransmitted; and dynamically adjusting the retransmission strategy to avoid network congestion and duplicate transmissions. For example, for a batch of unsynchronized transaction response data, the system intelligently determines the optimal retransmission order and plan based on the data's timestamp, importance, and sensitivity.

[0175] To enhance fault recovery reliability, the system incorporates multiple fault-tolerance technologies. These include a distributed backup mechanism that redundantly stores critical mission status across multiple edge devices; blockchain-based state synchronization technology that ensures the immutability of state snapshots; and an adaptive network transmission protocol that optimizes data retransmission under varying network conditions. For example, when network bandwidth is limited, the system can dynamically adjust data compression and transmission strategies to ensure reliable retransmission of critical data.

[0176] The advantages of this solution are: through detailed task status snapshots and intelligent fault recovery mechanisms, the system can quickly and accurately restore transaction processing when a failure occurs, minimizing the impact of the failure on business continuity; multi-level fault detection and reload technology, combined with intelligent data retransmission strategies, provides a comprehensive and reliable fault recovery solution that can cope with various complex system failure scenarios; by integrating distributed backup, blockchain synchronization and adaptive transmission technology, the system has fault tolerance and data recovery performance; this comprehensive fault recovery solution not only ensures system reliability and data integrity, but also provides a scalable and highly resilient technical solution for complex edge computing environments.

[0177] The present application also provides a machine-readable storage medium having a program stored thereon, which implements the above method when executed by a processor. The present application also provides a computer program product, including the computer program, which implements the above method when executed by a processor.

[0178] The embodiment of the present application further provides a computer device, which may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 4 As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05 and a memory (not shown in the figure) connected via a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A06. The network interface A02 of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor A01, a method for a decision engine is implemented. The display screen A04 of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device A05 of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0179] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0180] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0181] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0182] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0183] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0184] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory includes non-permanent storage in a computer-readable medium, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0185] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0186] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0187] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A high-concurrency transaction data processing method based on edge computing, characterized in that: The method comprises: S10: The edge device receives transaction data from multiple user terminals simultaneously through a multi-threaded concurrent receiving queue; S20: The edge device performs integrity and legitimacy checks on the transaction data and stores the verified transaction data in a local cache. The integrity check verifies whether the data has been tampered with during transmission by calculating a hash value or a checksum of the data. The legitimacy check includes checking the format, field integrity, data type, and value range of the transaction data. S30: The edge device shards the verified transaction data to obtain multiple data shards, and intelligently distributes the multiple data shards to multiple processing modules within the edge device; S40: Each processing module of the edge device processes according to the allocated data slices to obtain a processing result; S50: The edge device uses a real-time parallel reduction algorithm to intelligently integrate the processing results obtained by each processing module to obtain transaction response data; wherein, the real-time parallel reduction algorithm includes three stages: data sharding and parallel processing, intermediate result aggregation, and dynamic resource scheduling; in the data sharding and parallel processing stage, the real-time parallel reduction algorithm intelligently splits the data set into multiple relatively independent subsets, each of which can be processed in parallel on distributed computing nodes; in the intermediate result aggregation stage, a tree reduction strategy is adopted to gradually integrate the distributed computing results into a final unified result through multi-level gradual merging; in the dynamic resource scheduling stage, the load of each computing node can be evaluated in real time, and computing resources can be dynamically allocated and balanced according to the complexity of the task and the real-time status of system resources; S60: The edge device compresses the transaction response data using an adaptive compression algorithm, synchronizes the compressed transaction response data to the central server via a multi-encrypted secure transmission channel, and feeds back the synchronization status to the user terminal in real time; wherein the adaptive compression algorithm dynamically selects an optimal compression scheme based on the characteristics of the transaction response data; S70: After confirming that the transaction response data is successfully synchronized to the central server, the edge device securely deletes the verified transaction data stored in the local cache; Step S10 specifically includes the following steps: S101: Establish a multi-threaded concurrent receiving queue and configure thread pool parameters; the thread pool is used to manage thread resources in the multi-threaded concurrent receiving queue. For high-concurrency transaction scenarios, the edge device pre-configures a thread pool containing 20-50 worker threads and dynamically adjusts the number of threads based on real-time load; the thread pool parameters include the number of core threads, the maximum number of threads, the thread idle time, and the task queue length; S102: monitoring the connection request of the user terminal through the network socket, and assigning a thread in the thread pool to process each connection request; S103: Authenticate the user terminal that has established the connection, and create a secure communication channel after the authentication is successful. The secure communication channel is an end-to-end encrypted communication channel established using the TLS / SSL protocol. S104: Using the configured thread pool, threads are allocated to process transaction data reception tasks from different user terminals in parallel. Each thread in the thread pool is bound to a user terminal and receives data sent by the user terminal in real time. Each thread is equipped with an independent data buffer and processing logic. S105: Perform a basic integrity and transmission security pre-check on the transaction data received through the secure communication channel, and add the transaction data that passes the pre-check to the multi-threaded concurrent receiving queue. The basic integrity check includes checking the data length, basic format, and transmission protocol consistency of the data packet; the transmission security check includes detecting whether the data packet has been tampered with or has been replayed. Step S60 specifically includes the following steps: S601: Extracting transaction response data from the response cache area and performing integrity and consistency checks on the transaction response data to ensure that the transaction response data has not been damaged or tampered with. The integrity and consistency checks on the transaction response data employ a multi-verification mechanism, including calculating a SHA-256 hash value of the transaction response data and comparing it with the stored original hash value. Furthermore, the digital signature of the transaction response data is verified using a public key infrastructure (PKI) to ensure that the transaction response data has not been tampered with during transmission. S602: Invoke an adaptive compression algorithm to dynamically adjust the compression ratio according to the size and type of the transaction response data and compress the transaction response data; S603: Securely encrypt the compressed transaction response data using a multiple encryption algorithm to generate an encrypted data packet; wherein the multiple encryption algorithm includes: dynamically generating a one-time AES key during the encryption process; symmetrically encrypting the compressed transaction response data using the AES-256 algorithm and the AES key, and then encrypting the AES key using the RSA-2048 asymmetric encryption algorithm; S604: Send the encrypted data packet to the central server through the pre-established secure transmission channel, and record the status information of the transmission process in real time; S605: Receive transmission confirmation information from the central server to verify whether the transaction response data is successfully synchronized to the central server; wherein the transmission confirmation information includes a data hash value, a transmission timestamp, and a check code; S606: Based on the transmission confirmation information, the synchronization status is fed back to the user terminal in real time, indicating whether the data synchronization is successful or failed; S607: After confirming that the transaction response data is successfully synchronized to the central server, securely delete the transaction response data in the local cache of the edge device to free up storage space and ensure data privacy; wherein, the secure deletion uses multiple overwrite technology to overwrite the original data with random data multiple times; the secure deletion process is logged to provide a basis for subsequent compliance audits.

2. The method according to claim 1, characterized in that Step S20 specifically includes the following steps: S201: extracting transaction data to be verified from the multi-threaded concurrent receiving queue; S202: Perform format integrity check on the extracted transaction data, verifying whether the structure of the transaction data conforms to the expected format through predefined rule matching. If the transaction data does not conform to the expected format, it is directly marked as failed; S203: Perform a validity check on the target fields in the transaction data that has passed the format check. This includes the following: data type check, used to verify whether the data type of the target field complies with the preset data type; value range check, used to verify whether the numeric field is within the preset range; logical consistency check, used to verify whether the logical relationship between the target fields is reasonable; S204: Based on the results of the format integrity check and the legality check, mark the data that has passed the check as passed, and mark the data that has failed the check as failed; S205: Write the transaction data marked as passed into the local cache and create an index for it; S206: Record an audit log for the transaction data marked as failed and store it in an isolated area.

3. The method according to claim 1, characterized in that Step S30 specifically includes the following steps: S301: Determine sharding rules based on attributes of transaction data including type, size, and processing priority; S302: Divide the verified transaction data into multiple data shards according to the sharding rule; S303: Assign a unique shard label to each data shard to identify and track the processing status of the data shard; S304: Dynamically evaluate the current resource usage of each processing module within the edge device to obtain a resource evaluation result; S305: Determine an intelligent allocation strategy based on the attributes of the transaction data and the resource evaluation results of each processing module; S306: Allocate the multiple data shards to the multiple processing modules inside the edge device according to the intelligent allocation strategy.

4. The method according to claim 1, wherein Step S40 specifically includes the following steps: S401: Receive the data shards assigned to the current processing module and verify the integrity of the data shards and the consistency of their shard labels to ensure that the shard data has not been tampered with or lost; S402: Select an appropriate processing algorithm based on the operation type and content of the data shard; S403: Calling the processing algorithm to perform business logic processing on the data shards, which includes verifying business rules, performing data operations, and triggering related business operations; S404: Complete the data sharding process and generate a processing result, which includes transaction status, data update information, and intermediate data that can be used for subsequent integration; S405: Add a timestamp and a fragmentation tag to the processing result to obtain the processing result with the timestamp and fragmentation tag added; S406: Submit the processing result with the timestamp and shard tag attached to the processing result cache area, and notify the real-time parallel reduction module to prepare for integration.

5. The method according to claim 1, wherein Step S50 specifically includes the following steps: S501: Extract the processing results generated by each processing module from the processing result cache area, and verify the integrity of each processing result and the consistency of its timestamp and shard tag; S502: Perform preliminary sorting based on the original order of the transaction shards according to the shard labels to ensure the logical consistency of the processing results during the reduction process; S503: Based on the sharding attributes and business rules of the transaction data, multiple sharding results of the same transaction are logically merged and verified to generate an intermediate integration result; S504: Invoke a real-time parallel reduction algorithm to process multiple intermediate integration results in parallel and generate transaction response data; S505: Perform consistency verification on the transaction response data; S506: Mark the transaction response data that has passed the consistency verification as available and store it in the response cache area for subsequent compression and transmission.

6. The method according to claim 1, characterized in that Also includes: S80: The edge device records an exception log for transaction data that fails the integrity check or the legality check and stores it in an isolation area; when multiple consecutive exceptions are detected or the amount of abnormal data exceeds a preset threshold within a specified time, an alarm mechanism is triggered and the abnormal status is synchronized to the central server in real time for further analysis and processing.

7. The method according to claim 1, characterized in that Also includes: S90: When an edge device fails and processing is interrupted, the fault recovery mechanism is activated to reload unfinished transaction tasks; Transaction response data that is not synchronized to the central server is resent through the fault recovery mechanism.

8. A high-concurrency transaction data processing system based on edge computing, characterized in that: The system comprises: multiple user terminals; At least one edge device, wherein the edge device is configured with: A receiving module is used to simultaneously receive transaction data from multiple user terminals through a multi-threaded concurrent receiving queue; A verification storage module is used to perform integrity and legitimacy checks on the transaction data and store the verified transaction data in a local cache; the integrity check verifies whether the data has been tampered with during transmission by calculating a hash value or checksum of the data, and the legitimacy check includes checking the format, field integrity, data type, and value range of the transaction data; A data sharding module is used to shard the verified transaction data to generate multiple data shards, and intelligently distribute the data shards to multiple processing modules within the edge device; The parallel processing module is configured with independent computing units for parallel processing according to the allocated data slices to generate processing results; The result reduction module uses a real-time parallel reduction algorithm to intelligently integrate the processing results obtained by each processing module to generate transaction response data. The real-time parallel reduction algorithm consists of three stages: data sharding and parallel processing, intermediate result aggregation, and dynamic resource scheduling. In the data sharding and parallel processing stage, the real-time parallel reduction algorithm intelligently splits the data set into multiple relatively independent subsets, each of which can be processed in parallel on distributed computing nodes. In the intermediate result aggregation stage, a tree reduction strategy is used to gradually integrate the distributed computing results into a final unified result through multi-level step-by-step merging. In the dynamic resource scheduling stage, the load of each computing node can be evaluated in real time, and computing resources can be dynamically allocated and balanced according to the complexity of the task and the real-time status of system resources. a data transmission module, configured to compress the transaction response data using an adaptive compression algorithm; synchronize the compressed transaction response data to a central server via a multi-encrypted secure transmission channel; and provide real-time feedback of the synchronization status to the user terminal; wherein the adaptive compression algorithm dynamically selects an optimal compression scheme based on the characteristics of the transaction response data; A data cleaning module, after confirming that the transaction response data is successfully synchronized to the central server, securely deletes the verified transaction data stored in the local cache; the data cleaning module specifically includes: A synchronization verification unit is used to confirm that the transaction response data has been successfully synchronized to the central server. A cache cleaning unit is used to securely delete the transaction response data in the local cache of the edge device to free up storage space and ensure data privacy. The secure deletion uses a multiple-overwrite technique to overwrite the original data with random data multiple times. The secure deletion process is logged to provide a basis for subsequent compliance audits. Central server, configured with: A data receiving module is used to receive transaction response data synchronized by edge devices; The data verification module is used to perform secondary verification on the received transaction response data; The data storage module is used to persistently store transaction response data that has passed the secondary verification; A network communication device, configured to establish a secure communication link between the user terminal, the edge device, and the central server; The receiving module specifically includes: A thread pool management unit, configured to establish a multi-threaded concurrent receive queue and configure thread pool parameters; the thread pool is configured to manage thread resources in the multi-threaded concurrent receive queue; for high-concurrency transaction scenarios, the edge device pre-configures a thread pool containing 20-50 worker threads and dynamically adjusts the number of threads based on real-time load; the thread pool parameters include the number of core threads, the maximum number of threads, the thread idle time, and the task queue length; The connection monitoring unit is used to monitor the connection request of the user terminal through the network socket and allocate a thread in the thread pool to process each connection request; An identity authentication unit is used to authenticate the user terminal establishing the connection and create a secure communication channel after successful identity authentication. The secure communication channel is an end-to-end encrypted communication channel established using the TLS / SSL protocol; The concurrent receiving unit is used to allocate threads through the configured thread pool to parallelly process transaction data reception tasks from different user terminals. Each thread in the thread pool is bound to a user terminal and receives data sent by the user terminal in real time. Each thread is equipped with an independent data buffer and processing logic. A pre-check unit is used to perform basic integrity and transmission security pre-checks on transaction data received through a secure communication channel, and add transaction data that passes the pre-check to a multi-threaded concurrent receiving queue. The basic integrity check includes checking the data length, basic format, and transmission protocol consistency of the data packet; the transmission security check includes detecting whether the data packet has been tampered with or has been replayed. The data transmission module specifically includes: a data extraction unit, configured to extract transaction response data from the response cache area and perform integrity and consistency checks on the transaction response data to ensure that the transaction response data has not been damaged or tampered with; wherein the integrity and consistency checks on the transaction response data employ a multiple verification mechanism, including calculating a SHA-256 hash value of the transaction response data and comparing it with the stored original hash value; and verifying the digital signature of the transaction response data using a public key infrastructure (PKI) to ensure that the transaction response data has not been tampered with during transmission; An adaptive compression unit is used to call an adaptive compression algorithm, dynamically adjust the compression ratio according to the size and type of the transaction response data, and compress the transaction response data; an encryption unit configured to securely encrypt the compressed transaction response data using a multiple encryption algorithm to generate an encrypted data packet; wherein the multiple encryption algorithm includes: dynamically generating a one-time AES key during the encryption process; symmetrically encrypting the compressed transaction response data using the AES-256 algorithm and the AES key, and then encrypting the AES key using the RSA-2048 asymmetric encryption algorithm; The transmission unit is used to send the encrypted data packets to the central server through a pre-established secure transmission channel and record the status information of the transmission process in real time; A confirmation unit, configured to receive transmission confirmation information from the central server and verify whether the transaction response data has been successfully synchronized to the central server; wherein the transmission confirmation information includes a data hash value, a transmission timestamp, and a check code; The status feedback unit is used to feed back the synchronization status to the user terminal in real time based on the transmission confirmation information, prompting the success or failure of data synchronization.

Images