System for unified data interaction with external system based on asynchronous processing

Abstract

The invention provides a system for unified data interaction with an external system based on asynchronous processing. The system comprises a data interaction service management module, a timer module, a priority adjustment module, an asynchronous processing module and an operation and maintenance management module. The system receives and stores interaction data through the data interaction service management module, the timer module periodically scans a table to be sent and sends the data, and the priority adjustment module dynamically adjusts the data priority to optimize the processing sequence. The asynchronous processing module efficiently processes success and failure states of data transmission, supports automatic retry, state update and asynchronous storage, and avoids main flow blockage. And the operation and maintenance management module supports retransmission of failure and success records through resetting and authority control, so that the stability and flexibility of data interaction are ensured. The system efficiency is remarkably improved through asynchronous parallel processing, the reliability and response speed of data interaction are optimized, and the method is suitable for large-scale high-concurrency scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of data interaction, and in particular to a system for unified data interaction with an external system based on asynchronous processing. Background Art

[0002] In modern information systems, cross-system data interaction is an important means to achieve business collaboration and data sharing. However, existing technologies often face many problems and technical bottlenecks in complex business scenarios, which restrict data interaction efficiency and system stability.

[0003] Traditional data interaction methods usually adopt a synchronous mode, and the business module needs to wait for the confirmation response from the external system before continuing the subsequent operation. Although this mode is relatively simple to implement, the slow response speed of the external system in high-load scenarios will cause the business process to be blocked, thereby significantly increasing the overall delay of the task and reducing the processing efficiency of the system. In addition, this synchronization mechanism lacks effective management of task priorities. In the prior art, tasks are generally processed in the order of submission, and the priority of tasks cannot be dynamically adjusted according to the urgency or importance of business needs. Critical tasks are often blocked by low-priority tasks, which affects business continuity and service quality.

[0004] When data transmission fails, existing technologies usually only support a simple retry mechanism and lack detailed classification and management of failed records. If the retry fails, the task is often discarded, resulting in data loss or failure to complete important tasks. In addition, there is insufficient operation and maintenance support for failed tasks, making it difficult for system administrators to quickly discover problems and fix them, further increasing business risks. The storage and management of success and failure records are also relatively scattered. The process of migrating data from the to-be-sent table to the success table or failure table relies on multiple steps, which increases the complexity of the operation and easily causes data consistency issues.

[0005] In terms of operation and maintenance support, existing technologies have insufficiently designed reset functions for successful or failed tasks. If administrators need to resend data, they usually need to manually modify records or re-trigger operations. This method is inefficient and prone to errors, which is not conducive to long-term maintenance and optimization of the system. Even though some technologies have introduced asynchronous processing mechanisms, the parallelization capabilities they implement are still limited. In terms of data storage, status updates, and result feedback, they still need to be completed step by step in sequence, resulting in a failure to significantly improve system processing efficiency. At the same time, the complexity of the asynchronous mechanism itself also increases the difficulty of development and debugging, limiting its widespread application.

[0006] In general, existing technologies have significant deficiencies in data interaction delay control, task priority management, failure handling, storage complexity, and operation and maintenance support, making it difficult to meet the needs of high-concurrency and dynamic business scenarios for efficient data interaction systems. These problems need to be solved urgently to achieve a more efficient and stable cross-system data interaction mechanism and provide reliable support for complex business scenarios. Summary of the invention

[0007] In order to solve the above problems in the prior art, the present invention proposes a system for unified data interaction with an external system based on asynchronous processing, comprising:

[0008] A data interaction service management module, used to receive interaction data submitted by the business module through an interface, and store the interaction data in a table to be sent;

[0009] The timer module is used to periodically scan the table to be sent, assemble the data in the table to be sent, and send the data to the corresponding external system according to the interface address;

[0010] The priority adjustment module is used to dynamically adjust the priority of the data in the to-be-sent table according to the importance of the business, and the data with high priority is sent first;

[0011] The asynchronous processing module is used to process operations after data is sent successfully or failed, including:

[0012] After the data is sent successfully, the status of the data is automatically updated, changing it from "pending to be sent" to "successful";

[0013] Immediately feed back successful results to the business module without waiting for other operations to complete;

[0014] Parallel processing of data storage and state management, directly recording the success status, avoiding delays caused by transfers between tables;

[0015] When data transmission fails, two automatic retries are performed; if it still fails, the data is marked as "failed" and directly stored as a failure record, and the business module is notified at the same time;

[0016] Asynchronously complete the storage operation of failed data in the background to avoid blocking the main process;

[0017] The operation and maintenance management module is used to support data reset operations, including:

[0018] Reset the failed records, and the data status after reset will be changed to "pending to be sent", and re-enter the pending to be sent process;

[0019] Reset the success records, and the data status after reset will be changed to "pending to be sent" status, and re-enter the pending to be sent process.

[0020] The data interaction service management module is further used to perform field integrity check, data type check and protocol matching check on the received interaction data; the interaction data includes field information, interface address, business priority and technical route.

[0021] The timer module dynamically adjusts the scanning frequency according to a preset time interval to adapt to the change in the number of records in the to-be-sent table.

[0022] The priority adjustment module dynamically updates the data priority value through a priority calculation formula based on record retention time and business criticality indicators.

[0023] The priority adjustment module is further used to set a priority lock flag for high priority data records.

[0024] The asynchronous processing module verifies the availability of the target interface by sending a health check request when data transmission fails, and performs a retry operation after ensuring availability.

[0025] The asynchronous processing module adopts a reliable message mechanism or a distributed transaction framework to ensure atomicity and consistency of the storage operation of the failed records.

[0026] The operation and maintenance management module is further used to perform atomic control on the reset operation of batch data through a transaction mechanism, so that the data state is consistent during the reset process.

[0027] The operation and maintenance management module records the time, data identification, operation status and initiator information of each reset operation through a log.

[0028] The operation and maintenance management module performs authority verification on the reset operation.

[0029] Beneficial effects:

[0030] By introducing an asynchronous processing module, the storage of successful data, status update, and result feedback operations are integrated into a single module, and asynchronous parallel processing is implemented, which greatly improves system efficiency and reduces time delays and operational complexity. At the same time, the management of failed data also reduces the blockage of the main process through an asynchronous mechanism, achieving reliable and fast data interaction functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present application, but do not constitute an improper limitation of the present invention. In the drawings:

[0032] Figure 1 A schematic diagram of the structure of a unified data interaction system with an external system based on asynchronous processing is shown.

[0033] Figure 2 A method flow chart of a unified data interaction system with an external system based on asynchronous processing is shown. DETAILED DESCRIPTION

[0034] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments, wherein the illustrative embodiments and descriptions are only used to explain the present invention but are not intended to limit the present invention.

[0035] In order to better understand the present invention, specific implementation modes are described in detail below with reference to the accompanying drawings.

[0036] like Figure 1 As shown, a system for unified data interaction with external systems based on asynchronous processing of the present invention.

[0037] The system includes the following modules:

[0038] 1. Data interaction service management module 101

[0039] The data interaction service management module 101 receives interaction data from the business module through a standardized interface. The interaction data includes field information, interface address, business priority and technical route. The module stores the received data in the "to be sent table" for subsequent processing.

[0040] 2. Timer module 102

[0041] The timer module 102 scans the table to be sent at a preset time interval. During the scanning process, the timer module extracts data from the table to be sent, assembles data packets according to the interface address, and sends them to the corresponding external system through the interface protocol.

[0042] 3. Priority adjustment module 103

[0043] The priority adjustment module 103 dynamically monitors the business importance of the data in the to-be-sent table. According to the priority setting provided by the business module, the module adjusts the priority of the data in the to-be-sent table to ensure that high-priority data is sent first.

[0044] 4. Asynchronous processing module 104

[0045] The asynchronous processing module 104 is responsible for processing the success and failure status of the data interaction process, including:

[0046] After the data is sent successfully, the module updates the data status to "success" and immediately feeds back the success result to the business module to avoid blocking the main process.

[0047] When data transmission fails, the module automatically retries twice. If it still fails, the data status is marked as "failed" and stored as a failure record. At the same time, a failure notification is sent to the business module.

[0048] The background completes the storage of failed records in an asynchronous manner to ensure the efficient operation of the main process.

[0049] 5. Operation and maintenance management module 105

[0050] The operation and maintenance management module 105 provides data reset functions for the system, including:

[0051] Reset the failed record to the "pending to be sent" status and re-enter the sending process.

[0052] Reset the success record to the "pending" status to meet the repeated sending needs of specific scenarios.

[0053] This embodiment realizes efficient and stable data interaction with external systems through modular design combined with an asynchronous processing mechanism, thereby improving the overall performance and reliability of the system.

[0054] In order to better understand the present invention, the specific implementation of the data interaction service management module 101 is described in detail below.

[0055] The data interaction service management module 101 is used to receive the interaction data submitted by the business module through a standardized interface, and store the received data in the "to be sent table" to support the subsequent data interaction process. The specific technical implementation plan is as follows:

[0056] Receiving interactive data

[0057] The data interaction service management module 101 communicates with the business module through a standardized interface. The interface uses a predefined communication protocol (such as HTTP, gRPC, or an asynchronous communication protocol based on a message queue) to ensure the compatibility and reliability of data transmission.

[0058] The interactive data format received by the data interactive service management module can be JSON or XML structured data, and the data content includes the following fields:

[0059] Field information: identifies the specific content and format of the interaction data to ensure that the semantics of the data are clear. For example, fields include unique identifiers (UUIDs), data categories (such as order information, user information, etc.), and key attribute fields (such as order amounts, user IDs, etc.).

[0060] Interface address: The target interface address of the external system, used to determine the recipient of the data. The interface address includes the host name, port number, and access path (for example, https: / / api.example.com / v1 / data).

[0061] Business priority: The priority value specified by the business module based on actual needs, usually an integer (such as 1 for high priority, 2 for medium priority, and so on).

[0062] Technical route: indicates the specific technical configuration required for data interaction, such as protocol type (HTTP or HTTPS), authentication method (such as OAuth2.0 or API Key), and data encryption requirements (such as AES or RSA encryption).

[0063] Verification and analysis of interactive data

[0064] After receiving the data, the data interaction service management module 101 performs the following operations:

[0065] Data Validation: Check the integrity and validity of the received data, including field integrity check (ensuring that all required fields are provided), data type check (ensuring that the field value conforms to the predefined data type), and protocol matching check (ensuring that the interface address is consistent with the technical route).

[0066] Data parsing: Parse structured interactive data into a standard format that can be recognized within the system. For example, parse JSON into an in-memory data structure (such as Map in Java or d ict in Python) for subsequent processing.

[0067] Storage of interaction data

[0068] The verified interaction data will be stored in the "to be sent table". The specific implementation includes:

[0069] Storage structure: The table structure design of the table to be sent adopts a relational database (such as MySQL) or a NoSQL database (such as MongoDB). The fields of the table include data ID (primary key), field information, interface address, business priority, technical route, data status (such as "to be sent") and timestamp.

[0070] Transaction support: Stored procedures use database transactions to ensure data consistency. For example, if a field fails to be stored during a storage operation, a rollback mechanism will be triggered to restore the original state of the table.

[0071] Performance optimization: To improve writing performance, the module supports batch storage operations, that is, writing multiple interactive data at the same time in one transaction. In addition, query efficiency can be optimized by setting indexes (such as setting single-column indexes or composite indexes based on priority fields or timestamp fields), thereby improving data access performance. In high-concurrency scenarios, the module supports partitioned table storage, which partitions the table to be sent by business category or time period, further improving storage efficiency and query speed.

[0072] To ensure the reliability of data persistence, the data interaction service management module 101 also supports asynchronous write operations. After submitting the data, the module uses the asynchronous task mechanism to put the write operation into the background task queue for execution, thereby avoiding blocking the main thread and ensuring the system response speed in high-concurrency data interaction scenarios.

[0073] Through the above design, the data interaction service management module 101 can efficiently and reliably receive, verify, parse and store interaction data, laying a solid foundation for subsequent data transmission and processing. At the same time, the module ensures the adaptability and stability of the system in complex business scenarios through protocol compatibility design and data storage optimization mechanism.

[0074] Specific implementation of the timer module 102

[0075] The timer module 102 scans the table to be sent at a preset time interval, extracts the interactive data from it, assembles the data packet according to the interface address, and sends it to the corresponding external system through the interface protocol. The specific technical implementation is as follows:

[0076] Scheduled task execution

[0077] The timer module 102 periodically scans the table to be sent based on a time trigger mechanism. The specific implementation of the time trigger mechanism includes:

[0078] Triggering based on fixed time intervals: The module uses a timer to periodically trigger the scan operation. The triggering interval is specified by the system configuration parameters (for example, triggering once every 5 seconds).

[0079] Dynamically adjust the trigger frequency: In high-data-volume scenarios, the timer module can dynamically adjust the trigger frequency based on the number of records in the table to be sent. For example, when the number of records in the table to be sent exceeds a certain threshold, the trigger frequency is increased from 5 seconds to 2 seconds.

[0080] Scanning of tables to be sent

[0081] When a scan is triggered, the timer module 102 performs the following operations:

[0082] Data query: The module extracts data records that meet the conditions from the table to be sent through database query statements. The query conditions include:

[0083] The data status is "pending to be sent";

[0084] Sort by business priority from high to low;

[0085] Sort the records by their timestamps from earliest to latest.

[0086] Batch extraction: To optimize performance, the module uses paging query mode to extract a batch of data (such as 100 records) each time to avoid performance bottlenecks caused by loading too much data in a single query.

[0087] Packet assembly

[0088] For each data record extracted, the timer module assembles the data packet according to the interface address. The specific steps include:

[0089] Data encapsulation: The module encapsulates the field information in the data record into a request data packet according to the format of the target interface. For example, for the RESTful interface, the module serializes the field information into JSON format and generates an HTTP request body.

[0090] Request header configuration: The module configures the HTTP request header (such as Content-Type, Authorization) according to the technical route in the data record to ensure that the data packet meets the authentication and data format requirements of the target interface.

[0091] Interface address matching: The module extracts the interface address field from the data record and combines it with the system configuration file to determine the complete URL of the target system.

[0092] Sending of data packets

[0093] The timer module sends the assembled data packet to the target external system through the specified interface protocol. The specific implementation of the sending operation includes:

[0094] Sending protocol support: The module supports multiple protocols (such as HTTP, HTTPS, gRPC, WebSocket), and selects the appropriate protocol type according to the technical route field.

[0095] Request method selection: The module selects GET, POST, PUT or DELETE request method according to the definition of the target interface. Usually, data interaction uses the POST method to submit request body data.

[0096] Timeout and retry mechanism: The module sets the request timeout (such as 3 seconds) and automatically triggers the retry mechanism when the timeout or connection fails. The default number of retries is 2.

[0097] Processing of sent results

[0098] After the data packet is sent, the timer module records the sending result:

[0099] For records that are sent successfully, the module passes the success result to the asynchronous processing module to update the data status to "success".

[0100] For records that fail to be sent, the module passes the failure result to the asynchronous processing module to trigger subsequent retry or marking operations.

[0101] Through the above technical implementation, the timer module 102 can efficiently and accurately scan the table to be sent and complete the data sending operation, ensuring that the data interaction with the external system has high performance and stability.

[0102] The priority adjustment module 103 is used to dynamically monitor the business importance of the data in the to-be-sent table, and adjust the priority of the data in the to-be-sent table according to the priority setting provided by the business module to ensure that high-priority data is sent first. The specific technical implementation is as follows:

[0103] Prioritization rules

[0104] The priority adjustment module dynamically sets the priority of data based on the priority information provided by the business module. The priority setting follows the following rules:

[0105] Static priority: directly specified by the business module when it is submitted, usually an integer from 1 to N, where 1 is the highest priority.

[0106] Dynamic adjustment rules: Combine the attributes of business data and real-time scenarios to dynamically adjust the priority through algorithms. The dynamic priority P is calculated according to the following formula: final :

[0107] P final =P base +w1·T delay +w2·I critical

[0108] in:

[0109] P base It is the basic priority of the data, which is specified by the business module;

[0110] T delay The retention time of data in the table to be sent, in seconds;

[0111] I critical Key indicators of the data (such as the user level or order amount involved in the data);

[0112] w1, w2 are weight parameters, specified by the system configuration file.

[0113] The priority adjustment module dynamically monitors the importance of data in the following ways:

[0114] Real-time monitoring: The module regularly scans the data in the table to be sent, and counts the retention time, key indicators and basic priority of each record. The scanning frequency can be dynamically adjusted according to the amount of data (for example, once a second).

[0115] Event triggering: When a business module submits a special event (such as a change in system notification priority), the module immediately responds and recalculates the priority of related data.

[0116] Prioritization process

[0117] The priority adjustment module adjusts the data in the table to be sent according to the priority setting rules. The adjustment process is as follows:

[0118] Priority calculation: Calculate the final priority value P of each record using the above formula final .

[0119] Data sorting: re-sort the data records in the waiting table according to the calculated priority value. The sorting algorithm supports multiple methods:

[0120] Quick sort: Applicable to scenarios with small data volumes, ensuring efficient sorting.

[0121] Incremental sorting: insert and sort the newly added data, and only update the order of the affected data to optimize the sorting performance in large data volume scenarios.

[0122] Priority lock: Set a priority lock flag for the adjusted high-priority data records to ensure that these records will not be adjusted again within a short period of time (such as within 30 seconds) to avoid repeated calculations.

[0123] Optimized processing of high-priority data

[0124] To ensure that high-priority data is processed first, the module supports the following optimization strategies:

[0125] Dynamic resource allocation: In the timer module, more sending channel resources are allocated to high-priority data, and high-priority data packets are sent first.

[0126] Early warning mechanism: When the retention time of high-priority data in the to-be-sent table exceeds the preset threshold, the module automatically sends an early warning notification to the operation and maintenance management module for manual intervention.

[0127] The adjusted priority results are recorded in the following way:

[0128] Update within the table: Update the calculated priority value directly to the priority field of the table to be sent, and record the adjustment time and reason.

[0129] Logging: The module records the process and results of each priority adjustment in a log file to facilitate subsequent auditing and troubleshooting.

[0130] Through the above technical means, the priority adjustment module 103 can dynamically and accurately manage the priority of data in the to-be-sent table, ensuring that key data is sent first, thereby improving the overall efficiency and reliability of the system.

[0131] The asynchronous processing module 104 is used to efficiently process the success and failure states in the data interaction process, optimize the operating efficiency of the system through a non-blocking method, and ensure the efficient execution of the main process. When the data packet is successfully sent to the target external system, the asynchronous processing module will immediately update the status of the corresponding data record in the to-be-sent table to "success". This operation is completed through the database transaction mechanism to ensure the integrity and consistency of the data, and record the success timestamp and the response information of the target interface at the same time, so as to facilitate subsequent data analysis or debugging. After successful processing, the module notifies the business module of the result through the event triggering mechanism, and the feedback content includes the unique identifier of the successful data, the interface address of the target system, the success time, and possible additional information (such as the response summary of the external system). In order not to block the processing flow of other data, the feedback operation is completed in a non-blocking manner. For example, using a message queue or event-driven architecture, the feedback result is pushed to the business module through an independent asynchronous thread or task. In addition, in order to meet special needs, the feedback notification supports a callback mechanism. When the business module needs to respond immediately, the module can directly trigger the corresponding business operation through a preset callback interface.

[0132] When a data packet fails to be successfully sent to the target external system, the asynchronous processing module triggers an automatic retry mechanism and performs a maximum of two retries. The interval between each retry is a preset value, such as 5 seconds. The interval and the number of retries can be dynamically adjusted according to the system load to avoid continuous failures caused by short-term exceptions. Before each retry, the module verifies the availability of the target interface. For example, it sends an HTTP health check request to confirm whether the target system is operating normally, or verifies network connectivity through a Ping test. In order to avoid abnormal status of the target system due to repeated submission, the retry operation adopts an idempotent design. Idempotence is achieved through verification based on a unique identifier and a request digest to ensure that each retry operation has the same impact on the target system. If all retries fail, the module marks the data status as "failed" and immediately stores the record as failed data, which includes the data's unique identifier, the target interface address and its technical configuration information (such as protocol type and authentication method), the failure reason (such as network timeout or interface response exception), the failure time, and the number of failures.

[0133] After marking the data as "failed", the asynchronous processing module will promptly feedback the failure results to the business module through the event notification mechanism. The feedback content includes the unique identifier of the failed data, the reason for the failure, processing suggestions (such as manual recheck or resend), and detailed information of the failure record. If the target interface is unavailable for a long time, the module can generate an early warning message and push it to the operation and maintenance management module so that the operation and maintenance personnel can troubleshoot the problem in time. For example, when multiple consecutive failures are caused by interface authentication errors, the module can add a prompt in the early warning message and suggest that the operation and maintenance personnel check the authentication configuration.

[0134] In order to ensure that the main thread resources are not blocked, the asynchronous processing module uses background asynchronous storage to manage failure records, thereby achieving high concurrent processing capabilities. Specifically, the module transfers the storage operation of failed records to the background task queue through the asynchronous task scheduling framework. In order to further optimize performance, the background task will adopt a batch operation method, such as merging multiple failed records into a single transaction and writing them to the database in batches, thereby reducing IO overhead. In high data volume or distributed deployment scenarios, the module supports the storage of failed records in a distributed database cluster, and improves data query and storage performance through decentralized storage based on partition keys (such as data identifiers or target interface addresses). In order to prevent data loss or inconsistency during the storage process, the module uses a reliable messaging mechanism or a distributed transaction framework to ensure the atomicity and consistency of storage operations. Even if an exception occurs during the storage process, the system can ensure the accuracy and integrity of data records.

[0135] Through the above mechanism, the asynchronous processing module can accurately and efficiently manage the success and failure status of data interaction. This module not only ensures the non-blocking operation of the main process, but also significantly optimizes the data processing performance through the automatic retry mechanism and asynchronous storage, further improving the stability and response efficiency of the system, and providing reliable technical support for data interaction in large-scale, high-concurrency scenarios.

[0136] The operation and maintenance management module 105 provides the system with a data reset function to support reprocessing of failed and successful records in different scenarios, ensuring the flexibility and controllability of data interaction. This module dynamically adjusts the data status and re-imports the records into the process to be sent, thereby achieving the goal of repeated sending or corrected sending.

[0137] When the failed record needs to be reprocessed, the operation and maintenance management module 105 can reset the selected failed record status to "to be sent" and re-include it in the table to be sent. First, the module provides a query interface for users to filter and locate the failed records that need to be reset. The query conditions include the unique identifier of the data, the reason for failure, the target interface address, the failure time and the number of failures, etc. The user can specify one or more records for reset operation through these conditions. The module performs permission verification on the filtered records to ensure that only users or system components with operation permissions can modify the status of the records. After the verification is passed, the module updates the status field of the failed record from "failed" to "to be sent" and records the current timestamp as the reset time. In order to avoid exceptions caused by repeated data transmission, the module clears temporary fields related to failure in the reset operation, such as the number of failures or the most recent failure reason, to ensure that the records in the table to be sent enter the subsequent sending process with a brand new state.

[0138] In certain specific scenarios, it may be necessary to repeat the sending operation of the success record, such as the target system requesting retransmission or the user actively triggering the retransmission requirement. To this end, the operation and maintenance management module 105 supports resetting the status of the success record to "to be sent" and re-adding it to the to-be-sent table for processing. The module locates the success record that needs to be reset through a similar query and screening mechanism. The user specifies the records that need to be reset through conditional screening (such as data unique identifier, target interface address and success time) or batch selection operations. The module also performs permission verification to prevent unauthorized reset operations. After the permission verification is passed, the module updates the status field of the success record from "success" to "to be sent" and clears non-essential fields related to success (such as success timestamp or response summary of the target interface) to ensure that the record can trigger the complete data sending process again.

[0139] In order to support batch reset operations, the operation and maintenance management module 105 adopts efficient transaction management and batch processing technology. When the user selects multiple records to reset, the module encapsulates all operations into a single database transaction to ensure that the entire transaction can be rolled back to restore a consistent state when any reset operation fails. The module handles large-scale data resets through a paging mechanism to avoid system performance problems caused by excessive data volume in a single operation.

[0140] To improve the transparency and traceability of operations, the module logs each reset operation. The log content includes the time of the reset operation, the unique identifier of the record involved, the original state and target state, the identity information of the initiator of the operation, and any relevant notes or instructions. The log records can be queried by operation and maintenance personnel at any time to facilitate subsequent audits or troubleshooting.

[0141] Through the above design, the operation and maintenance management module 105 realizes the flexible reset management of failure records and success records, provides support for the data interaction process in specific scenarios, and ensures the security and traceability of operations. This module further improves the stability and flexibility of the system by working in collaboration with other modules, and can meet the dynamic adjustment needs in various business scenarios.

[0142] like Figure 2 As shown in the figure, the data interaction process of the system includes the following steps, which ensure the efficiency and reliability of data interaction through modular design and dynamic collaboration. Each step is described in detail below:

[0143] Step 1: The business module submits the interactive data to the data interactive service management module 101 through the interface

[0144] The business module submits interactive data to the data interaction service management module through standardized interfaces (such as HTTP API, RPC interface, etc.). The submitted data includes core elements such as field information, interface address, business priority and technical route. The interface adopts a unified structured format (such as JSON or XML) to ensure the standardization of data transmission. The data interaction service management module performs the following processing on the received data:

[0145] Data parsing: Parse the received structured data into a standard format recognizable within the system (such as key-value pairs or data objects in memory).

[0146] Data verification: including field integrity verification, data type verification and interface address format verification to ensure the legitimacy of the data.

[0147] Data storage: The verified interactive data is stored in the to-be-sent table through the transaction mechanism, and the timestamp and initial status as "to-be-sent" are recorded at the same time.

[0148] Step 2: Data is stored in the waiting table and waits for the scanning process of the timer module 102

[0149] The table to be sent is stored in a database, and the design includes the following fields: data unique identifier, interface address, priority, current status, submission time, etc. After the data is stored, the record is in the "to be sent" state, waiting for the timer module to scan it. The design optimizes access performance through indexes to ensure query efficiency in high-concurrency scenarios.

[0150] Step 3: The timer module processes the high priority data preferentially according to the setting of the priority adjustment module 103 and sends the assembled data to the external system.

[0151] The timer module scans the table to be sent at a preset time interval and processes the data according to the priority order set by the priority adjustment module. The specific implementation includes:

[0152] Scanning and sorting: The timer module performs paging queries, extracts a batch of data records with the highest priority from the table to be sent, and further sorts them according to timestamps to ensure that high-priority, early-submitted data is sent first.

[0153] Data assembly: The module assembles the data packet according to the interface address and field information of each data record. For RESTful interface, the assembled request packet usually includes HTTP method, URL, request header and request body content.

[0154] Data sending: The timer module sends the assembled data packet to the target external system through an external communication protocol (such as HTTP or gRPC) and listens for the synchronous response sent.

[0155] Step 4: The asynchronous processing module 104 monitors the sending result and processes the subsequent operations of the sending success or failure

[0156] The asynchronous processing module monitors the results of the timer module's sending operation in real time, and performs the following operations based on the sending results:

[0157] Send successfully:

[0158] Update the status of the corresponding record in the table to be sent to "success", and record the sending time and external system response information;

[0159] The successful result is fed back to the business module through the message queue or event trigger mechanism, and the feedback content includes the unique identifier, interface address and sending time.

[0160] Send failed:

[0161] The retry mechanism is triggered and the module performs a maximum of two retries.

[0162] If all retries fail, the record status is updated to "failed", and the failure reason and timestamp are stored.

[0163] Step 5: If the sending fails, the system automatically triggers a retry, and the failure record is finally reset by the operation and maintenance management module 105 or manually intervened;

[0164] For failed records, the asynchronous processing module marks their status as "failed" after retry failure and notifies the operation and maintenance management module through the event mechanism. The operation and maintenance management module supports the following operations:

[0165] Record reset: The user or system resets the failed record to the "pending to be sent" status as needed and re-enters the sending process.

[0166] Manual intervention: Query detailed information of failure records (such as failure reasons, target interface addresses, etc.) through the operation interface, and the operation and maintenance personnel manually adjust the configuration or check the abnormal status of the external system.

[0167] Through the above steps, the system realizes dynamic collaboration between modules and efficient processing of data flow. The data interaction process design not only meets the requirements of high concurrency and real-time processing, but also improves the robustness and stability of the system through a flexible failure handling mechanism.

[0168] The above description is only a preferred embodiment of the present invention, so all equivalent changes or modifications made according to the structure, characteristics and principles described in the scope of the patent application of the present invention are included in the scope of the patent application of the present invention.

Claims

1. A system for unified data interaction with external systems based on asynchronous processing, characterized in that: The unified data interaction system includes: A data interaction service management module, used to receive interaction data submitted by the business module through an interface, and store the interaction data in a table to be sent; The timer module is used to periodically scan the table to be sent, assemble the data in the table to be sent, and send the data to the corresponding external system according to the interface address; The priority adjustment module is used to dynamically adjust the priority of the data in the to-be-sent table according to the importance of the business, and the data with high priority is sent first; The asynchronous processing module is used to process operations after data is sent successfully or failed, including: After the data is sent successfully, the status of the data is automatically updated, changing it from "pending to be sent" to "successful"; Immediately feed back successful results to the business module without waiting for other operations to complete; Parallel processing of data storage and state management, directly recording the success status, avoiding delays caused by transfers between tables; When data transmission fails, two automatic retries are performed; if it still fails, the data is marked as "failed" and directly stored as a failure record, and the business module is notified at the same time; Asynchronously complete the storage operation of failed data in the background to avoid blocking the main process; The operation and maintenance management module is used to support data reset operations, including: Reset the failed records, and the data status after reset will be changed to "pending to be sent" status, and re-enter the pending to be sent process; Reset the successful records, and the data status after reset will be changed to "pending to be sent" status, and re-enter the pending to be sent process.

2. A system for unified data interaction with an external system based on asynchronous processing as claimed in claim 1, characterized in that: The data interaction service management module is further used to perform field integrity check, data type check and protocol matching check on the received interaction data; the interaction data includes field information, interface address, business priority and technical route.

3. A system for unified data interaction with external systems based on asynchronous processing as claimed in claim 1, characterized in that: The timer module dynamically adjusts the scanning frequency according to a preset time interval to adapt to the change in the number of records in the to-be-sent table.

4. A system for unified data interaction with external systems based on asynchronous processing as claimed in claim 1, characterized in that: The priority adjustment module dynamically updates the data priority value through a priority calculation formula based on record retention time and business criticality indicators.

5. A system for unified data interaction with an external system based on asynchronous processing as claimed in claim 1 or 4, characterized in that: The priority adjustment module is further used to set a priority lock flag for high priority data records.

6. A system for unified data interaction with external systems based on asynchronous processing as claimed in claim 1, characterized in that: The asynchronous processing module verifies the availability of the target interface by sending a health check request when data transmission fails, and performs a retry operation after ensuring availability.

7. A system for unified data interaction with an external system based on asynchronous processing as claimed in claim 1 or 6, characterized in that: The asynchronous processing module adopts a reliable message mechanism or a distributed transaction framework to ensure atomicity and consistency of the storage operation of the failed records.

8. A system for unified data interaction with external systems based on asynchronous processing as claimed in claim 1, characterized in that: The operation and maintenance management module is further used to perform atomic control on the reset operation of batch data through a transaction mechanism, so that the data state is consistent during the reset process.

9. A system for unified data interaction with external systems based on asynchronous processing as claimed in claim 1, characterized in that: The operation and maintenance management module records the time, data identification, operation status and initiator information of each reset operation through a log.

10. A system for unified data interaction with external systems based on asynchronous processing as claimed in claim 1, characterized in that: The operation and maintenance management module performs authority verification on the reset operation.

Images