A data processing method and device, electronic equipment and computer readable medium

By obtaining auxiliary data of business data and determining the transaction type, starting the task thread and performing automatic retries in case of exceptions, the problems of increased complexity and low data query efficiency caused by the differences between streaming and batch processing methods are solved, and consistency of data processing and efficient query are achieved.

CN120407091BActive Publication Date: 2025-10-17CHINA SECURITIES DEPOSITORY & CLEARING CO LTD
1 Cites 0 Cited by

Patent Information

Application Number
CN202510873042.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-06-26
Publication Date
2025-10-17
Estimated Expiration
2045-06-26

AI Technical Summary

Technical Problem

The existing stream processing and batch processing methods are different, which increases the complexity of development and operation and maintenance, and has low data query efficiency, and cannot guarantee the consistency of data processing after interruption recovery.

Method used

By obtaining auxiliary data of business data, the transaction type and data source to be submitted are determined, the task thread is started and the transaction is submitted; when an exception occurs in the task instance, the transaction submission is automatically retried based on the breakpoint transaction, data source and preset data consistency mode, exception information is generated and the task thread is exited.

Benefits of technology

It reduces the complexity of development and operation, improves data query efficiency, and ensures the consistency of data processing after interruption recovery.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application discloses a data processing method and device, electronic equipment and computer readable medium, relates to the technical field of data processing, and the method comprises the following steps: in response to the completion of business logic execution, acquiring the current batch of business data, calling a multi-level cache memory table to acquire the auxiliary data corresponding to the business data; based on the business data and the auxiliary data, determining the transaction to be submitted and the transaction type; determining the data source corresponding to the transaction, starting the task thread to start and submit the transaction based on the data source and the transaction type; in response to an exception of the task instance, acquiring the transaction submission progress to determine the breakpoint transaction, and performing transaction submission automatic retry based on the breakpoint transaction, the data source corresponding to the breakpoint transaction, the transaction submission progress and the preset data consistency mode; in response to the failure of the transaction submission automatic retry, generating and outputting exception information, and exiting the task thread. The complexity of development and operation and maintenance is reduced, the data query efficiency is improved, and the consistency of data processing is ensured.
Need to check novelty before this filing date? Find Prior Art