Data reporting method and apparatus, storage medium, and terminal

Abstract

The embodiment of the specification discloses a data reporting method and device, a storage medium and a terminal, which realizes stream processing by converting batch data into a structured event stream, adopts a logical double-channel architecture to simultaneously submit events to a main processing channel and a backup cache channel, wherein the main channel is responsible for real-time reporting and updates a globally persistent progress water line after each success to accurately mark the processing boundary; the backup channel stores a complete data copy. When the main channel fails, the system queries the water line and locates the accurate playback starting point in the backup channel according to the water line, and automatically reads the event copy for continuous transmission. The mechanism ensures data reliability through double-channel redundancy, and cooperatively realizes accurate and rapid recovery of breakpoints relying on the water line, finally guaranteeing the efficiency of data processing and system high availability.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a data reporting method, apparatus, storage medium, and terminal. Background Technology

[0002] In high-concurrency, high-SLA scenarios, existing data synchronization architectures based on linear batch processing have shortcomings. They heavily rely on a single processing path, making them susceptible to system-wide interruptions due to network or service anomalies. Furthermore, the lack of precise and persistent global state tracking prevents breakpoint resumption after failures, necessitating inefficient full retries or manual intervention. Traditional, crude retry logic also easily leads to duplicate data submissions to downstream systems, causing service chaos and compliance risks. The entire system is typically script-driven, resulting in insufficient observability, difficulty in monitoring key metrics, and poor operational controllability and auditability, severely impacting the overall continuity and compliance of the process. Summary of the Invention

[0003] This specification provides a data reporting method, apparatus, storage medium, and terminal, which can solve the technical problem of poor disaster recovery when reporting massive amounts of data in related technologies.

[0004] Firstly, embodiments of this specification provide a data reporting method, the method comprising:

[0005] Obtain the source event stream to be reported, where each event in the source event stream corresponds to one piece of data to be reported;

[0006] Each event in the above source event stream is submitted in parallel to the main processing channel and the backup cache channel. The main processing channel is used to report each event in sequence and update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the above source event stream.

[0007] When a reporting failure occurs in the main processing channel mentioned above, check the current progress level.

[0008] Based on the current progress level, the corresponding playback start position in the backup cache channel is determined. Starting from the playback start position, the event copy stored in the backup cache channel is read and retransmitted.

[0009] In one possible implementation, starting from the playback start position, reading the event copies stored in the backup cache channel for continued transmission includes: starting from the playback start position, for each event copy read from the backup cache channel, querying the deduplication table to see if there is a record that has been successfully transmitted; if there is no record that has been successfully transmitted, then performing a transmission operation on the corresponding event copy.

[0010] In one possible implementation, the deduplication table is recorded and maintained based on the idempotent key corresponding to each event.

[0011] In one possible implementation, the information contained in the progress watermark status includes at least an identifier of a successful reporting event, the corresponding reporting channel identifier, and a timestamp.

[0012] In one possible implementation, starting from the playback start position, reading the event copy stored in the backup cache channel for continued transmission includes: determining the playback mode based on the fault type corresponding to the transmission failure, and generating playback conditions corresponding to the playback mode; the playback mode includes at least one of time window playback mode, primary key range playback mode, and error classification playback mode; starting from the playback start position, determining the target event copy that meets the playback conditions in the backup cache channel; and continuing transmission of the target event copy.

[0013] In one possible implementation, the method further includes: if the reported failure is an anomaly that cannot be automatically recovered, then submitting the reported failure event and the corresponding fault information to the anomaly event isolation area, and receiving manual processing instructions for the anomaly event through the management interface maintained in the anomaly event isolation area.

[0014] Secondly, embodiments of this specification provide a data reporting device, the device comprising:

[0015] The data preparation module is used to obtain the source event stream to be reported, where each event in the source event stream corresponds to one piece of data to be reported.

[0016] The primary and backup synchronization module is used to submit each event in the above source event stream to the primary processing channel and the backup cache channel in parallel. The primary processing channel is used to report each event in sequence and update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the above source event stream.

[0017] The progress query module is used to query the current progress level when a reporting failure occurs in the main processing channel mentioned above.

[0018] The playback reporting module is used to determine the corresponding playback start position in the backup cache channel based on the current progress waterline, and to read the event copy stored in the backup cache channel for continued transmission starting from the playback start position.

[0019] Thirdly, embodiments of this specification provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of the method described above.

[0020] Fourthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method described above.

[0021] Fifthly, embodiments of this specification provide a terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is adapted to be loaded by the processor and to execute the steps of the method described above.

[0022] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0023] This specification provides a data reporting method. It involves acquiring a source event stream to be reported, where each event in the source event stream corresponds to a piece of data to be reported. Each event in the source event stream is submitted in parallel to a main processing channel and a backup cache channel. The main processing channel reports each event sequentially and updates the progress watermark status corresponding to each successfully reported event after each successful report. The backup cache channel stores a complete copy of the source event stream. When a reporting failure occurs in the main processing channel, the current progress watermark is queried. Based on the current progress watermark, the corresponding playback start position in the backup cache channel is determined. Starting from the playback start position, the event copy stored in the backup cache channel is read for continued reporting. First, the discrete data packets traditionally processed in batches are abstracted and transformed into a continuous, structured event sequence, where each event corresponds to a piece of data to be reported with a complete context. This event stream transformation mechanism enables the system to process massive amounts of data in a streaming, incremental manner. After acquiring the event stream, each event in the source event stream is submitted in parallel to the main processing channel and the backup cache channel. In this step, the system synchronously distributes copies of the same event to two logical channels with different responsibilities. The primary processing channel handles real-time processing and reporting, while the backup cache channel is used for persistent data backup. This parallel submission mechanism ensures the reliable preservation of original data, avoiding the risk of data loss due to a single channel failure. Secondly, the primary and backup channels do not block each other, guaranteeing both performance and high availability, achieving redundancy and disaster recovery capabilities. The core operation of the primary processing channel is to report each event sequentially and update the progress watermark status corresponding to each successfully reported event after each successful report. The watermark is a globally visible, persistent metadata marker that accurately records the boundary of successfully processed data. This allows any component in the distributed system to have a unified understanding of the system's processing progress based on this progress pointer. This provides a precise basis for subsequent breakpoint resumption and primary / backup failover, preventing duplicate or missed reports due to ambiguous states. When the system detects a failure in the main processing channel, it queries the current progress watermark and determines the corresponding playback start position in the backup buffer channel based on the current progress watermark. It then reads the event copy from this position to resume transmission. Through the globally shared watermark status information, the system can know the last successfully confirmed point before the failure occurred, thus promptly resuming event transmission from a known coordinate on the backup channel. This recovery process is accurate, reliable, and automated, greatly shortening the failure recovery time. It ensures that all data to be reported is successfully processed once and only once from the occurrence of the failure to the completion of the recovery, ensuring the complete continuity of system service capabilities. This meets the strict timeliness requirements of high-level service protocols and improves the overall reliability of the system's data reporting tasks, the accuracy of failure recovery, and the level of automation. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 An exemplary system architecture diagram of a data reporting method provided in the embodiments of this specification;

[0026] Figure 2 A flowchart illustrating a data reporting method provided in an embodiment of this specification;

[0027] Figure 3 A flowchart illustrating a data reporting method according to another embodiment of this specification;

[0028] Figure 4 A structural block diagram of a data reporting device provided in the embodiments of this specification;

[0029] Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of this specification. Detailed Implementation

[0030] To make the features and advantages of the embodiments of this specification more apparent and understandable, the technical solutions of the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the embodiments of this specification.

[0031] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those in this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments in this specification as detailed in the appended claims. Furthermore, in the description of the embodiments in this specification, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, in the description of the embodiments in this specification, "multiple" refers to two or more.

[0032] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0033] In high-concurrency service scenarios involving real-time processing and synchronization of massive amounts of data, systems often need to reliably complete the collection, processing, and final delivery of millions or even tens of millions of data records within a specified timeframe, under strict Service Level Agreement (SLA) constraints. Solutions for such scenarios typically construct a multi-stage batch processing pipeline: raw records are extracted from the source system, cleaned, integrated, and standardized by the data platform, then messages or files conforming to downstream system specifications are generated in a dedicated processing module, and finally, the outputs are submitted one by one to an authoritative, usually external, target system by calling specific interfaces or protocols. Taking the scenario of financial credit data reporting as an example, this scenario typically requires the system to accurately deliver credit transaction records to the financial credit information database before a specified time. However, this widely adopted linear processing architecture, due to its inherent design pattern, exposes several shortcomings when dealing with the requirements of high reliability and strong consistency in modern distributed environments.

[0034] Specifically, the core data processing and delivery links in existing solutions are sequential, highly dependent on a single execution path, network channel, and terminal interface. This design introduces a single point of failure risk into the critical path; any interruption in any link, such as momentary network fluctuations, brief service unavailability, or interface rate limiting, can cause the entire process to stall. More critically, the process lacks precise, persistent, and globally consistent state tracking of processing progress. The system cannot confirm the exact stage of each record at any time (e.g., "generated," "submitted," "confirmed"). If the process is unexpectedly interrupted, the system cannot intelligently resume transmission precisely from the breakpoint, and is forced to either perform a full retry or rely on cumbersome manual intervention to estimate the starting point, which is inefficient and highly prone to data consistency issues. The lack of a state-aware mechanism further leads to rudimentary retry and fault-tolerance logic. When attempting to restore service or handle momentary failures, the system's simple retry behavior, unable to accurately determine whether a record has been successfully processed, easily results in the same data entity being repeatedly submitted to downstream systems. This duplication not only causes data redundancy and confusion at the receiving end but may also lead to logical errors and compliance risks. Furthermore, in primary-backup disaster recovery designs aimed at improving availability, the primary-backup switchover process is often difficult to achieve smoothly and without loss, which can easily lead to data loss or duplicate submissions, making the disaster recovery mechanism itself a potential risk point.

[0035] From a system operations and governance perspective, traditional architectures are typically driven by discrete scripts and job scheduling, making it difficult to monitor and measure core metrics such as processing throughput, queue backlog, end-to-end latency, and error classification and distribution in real time. When anomalies occur, alerts are often delayed and vague, failing to meet the high standards of visibility, controllability, and auditability required by enterprise applications. In high-concurrency scenarios with strong SLA constraints, these issues directly impact the timely, accurate, and reliable completion of core service processes, hindering operational continuity and compliance for enterprises.

[0036] Therefore, this specification provides a data reporting method to solve the aforementioned technical problem of poor disaster recovery when reporting massive amounts of data.

[0037] Please see Figure 1 , Figure 1 This is an exemplary system architecture diagram of a data reporting method provided in the embodiments of this specification.

[0038] like Figure 1 As shown, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 serves as the medium for providing a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth communication links, Wireless-Fidelity (Wi-Fi) communication links, or microwave communication links, etc.

[0039] Terminal 101 can interact with server 103 via network 102 to receive messages from or send messages to server 103. Alternatively, terminal 101 can interact with server 103 via network 102 to receive messages or data sent to server 103 by other users. Terminal 101 can be hardware or software. When terminal 101 is hardware, it can be various electronic devices, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal 101 is software, it can be installed in the aforementioned electronic devices and can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module; no specific limitation is made here.

[0040] In the embodiments of this specification, terminal 101 first obtains the source event stream to be reported, where each event in the source event stream corresponds to a piece of data to be reported. When reporting data, terminal 101 submits each event in the source event stream to the main processing channel and the backup cache channel in parallel. The main processing channel is used to report each event in sequence and update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the source event stream. When a reporting failure occurs in the main processing channel, terminal 101 queries the current progress watermark. Furthermore, based on the current progress watermark, terminal 101 determines the corresponding playback start position in the backup cache channel and reads the event copy stored in the backup cache channel from the playback start position to continue the reporting.

[0041] Server 103 can be a business server providing various services. It should be noted that server 103 can be hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module; no specific limitations are made here.

[0042] Alternatively, the system architecture may not include server 103. In other words, server 103 may be an optional device in the embodiments of this specification. That is, the method provided in the embodiments of this specification can be applied to a system structure that only includes terminal 101. The embodiments of this specification do not limit this.

[0043] It should be understood that Figure 1 The number of terminals, networks, and servers shown is only illustrative; the number can be any number of terminals, networks, and servers depending on the implementation requirements.

[0044] Please see Figure 2 , Figure 2 This is a flowchart illustrating a data reporting method provided in an embodiment of this specification. The execution entity in this embodiment can be a terminal performing data reporting, a processor within the terminal performing the data reporting method, or a data reporting service within the terminal performing the data reporting method. For ease of description, the following example uses a processor within the terminal as the execution entity to illustrate the specific execution process of the data reporting method.

[0045] like Figure 2 As shown, the data reporting method may include at least:

[0046] S202. Obtain the source event stream to be reported. Each event in the source event stream corresponds to one piece of data to be reported.

[0047] Optionally, in the embodiments of this specification, the discrete data to be reported generated in the system is first abstracted and encapsulated into a continuous and ordered event stream. Specifically, by transforming data snapshots from the upstream batch processing system, which exist in the form of tables or files, into a continuous and strictly ordered event stream, data abstraction and paradigm shift are achieved. This constructs a unified, reliable data ingestion layer with clear semantics. This shift is the foundation for achieving subsequent high reliability, accurate status tracking, and streaming disaster recovery.

[0048] Specifically, each record to be reported can be written as an event from an offline data warehouse to an intermediate medium. This intermediate medium is a database, such as a Kafka cluster or OceanBase database table, that facilitates system reading and monitoring of the event stream. The source event stream is a strictly ordered sequence of immutable event objects, where each event is a self-contained atomic data unit, encapsulating a complete reporting record generated by a single operation (such as a loan disbursement) at a specific point in time. The specific information should include at least a service data payload and system metadata. The service data payload is the content to be reported, cleaned and standardized to conform to the interface specifications of the downstream target system; while the system metadata is used for unique identification and end-to-end tracing. Thus, the system can obtain the source event stream to be reported by subscribing to specific topics in the intermediate medium or listening to database change logs.

[0049] Furthermore, in the embodiments of this specification, each reporting record may contain a unique identifier: reportid + reporting date + reporting time + user basic information, which constitutes the primary key of the record. Here, reportid ensures the uniqueness of a single service request; the reporting date and reporting time together define the precise location of the event in the service time dimension, serving as the core basis for sequential reporting in the early stages, and subsequent backtracking, sorting, and fault recovery by time window; user basic information provides the entity's identifier. This composite primary key design is the foundation for accurate deduplication and global ordered perception in a distributed environment. This design decouples data production and consumption, allowing the system to independently pull data from reliable event sources (such as message middleware) on demand, improving the system's flexibility and scalability. The system continuously monitors the data source, and once a new event arrives, it immediately triggers the subsequent processing flow, reducing the system's latency in processing reported data.

[0050] S204. Submit each event in the source event stream to the main processing channel and the backup cache channel in parallel. The main processing channel is used to report each event in sequence and update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the source event stream.

[0051] Optionally, for each event in the input source event stream, the system synchronously replicates and routes it to two functionally separated logical channels, i.e., submitting it in parallel to the main processing channel and the backup cache channel. In one feasible implementation, the stateful stream processing capabilities provided by Flink can be utilized, with an Apache Flink job acting as the reporting execution control center to initiate and consume the source event stream. The Flink job is the core execution unit in the Apache Flink stream processing framework. It continuously pulls reporting tasks from the database storing the source event stream. Internally, the job takes each pulled event as input, replicates it, and simultaneously feeds it to two processing branches. In the main processing channel branch, the event immediately enters the reporting processing pipeline; in the backup cache channel branch, the event copy is efficiently written to another highly persistent storage system (such as another Kafka cluster or a dedicated backup table). This process, whether synchronous or asynchronous, guarantees eventual consistency, ensuring dual-path persistence of the original data.

[0052] Furthermore, when reporting source data streams, the process is constrained by specific service rules. For example, in a credit data reporting scenario, downstream nodes may require daily data to be reported within a specific timeframe. The Flink job will then shard the data according to the date in the message name, ensuring that messages from the same day are processed sequentially, thus maintaining a strict order of events within the instance. This is crucial for time-series-dependent service logic, preventing state chaos that might result from out-of-order processing across shards. The parallelism during event processing can be set according to actual rate requirements. Parallelism is primarily determined by the carrying capacity of the downstream target system to avoid overwhelming downstream services or triggering their rate-limiting policies due to sudden high concurrency requests from the current system.

[0053] Furthermore, Flink jobs periodically checkpoint their state during runtime (including the offset of data consumed by each parallel instance and user-defined states such as intermediate results of processed events). When a job restarts due to a failure, it can recover from the most recent successful checkpoint, achieving precise rollback of the processing state. This mechanism is the foundation for the main processing channel to reliably report and update watermarks in order. Using Flink jobs ensures that even after a job crash or node failure, the recovered job can continue precisely from the breakpoint. By adjusting parameters such as parallelism, memory configuration, and checkpoint interval, the system's throughput, processing latency, and resource consumption can be finely controlled. The job itself acts as a container, uniformly managing computing resources, state, and fault recovery, allowing the entire complex dual-channel disaster recovery logic to exist in the form of a simple, deployable, and easily managed application.

[0054] In the embodiments described in this specification, the main processing channel is responsible for performing a series of prescribed transformations, encapsulations, and remote call operations under normal circumstances. This includes converting events into messages of a specific format, performing encrypted signing, and submitting them to the target system via network protocols. After each successful submission, it atomically updates the progress watermark status corresponding to the successfully submitted event. The progress watermark status is a persistent, versioned metadata record, typically containing at least the identifier of the successfully submitted event, the corresponding submission channel identifier, and a timestamp, representing the work progress that the system has confirmed as persistently completed. In one feasible implementation, the coordination and management of the progress watermark status can be implemented using Redis or ZooKeeper. The backup cache channel is a high-persistence, high-throughput storage service whose sole responsibility is to accurately and completely record copies of the original event stream. Write operations to the backup channel are typically asynchronous or non-blocking to ensure that the processing performance of the main channel is not affected. The storage period of the backup cache channel can be set according to actual needs (e.g., 7 days, 5 days, etc.) to support replay at any subsequent point in time. The data structure of the event copy still retains the original fields and context information, facilitating resubmission after repair. While the dual channels may physically share the underlying infrastructure, they are logically completely independent and isolated, ensuring that a failure in one will not immediately affect the other. This dual-channel mechanism ensures that a copy of the original data exists independently of the processing logic, providing the system with a final source of data recovery and achieving data layer redundancy. Furthermore, through the cooperation of the primary channel and the waterline, a unified and authoritative source of facts is established for the system. This is the foundation for accurate breakpoint location and recovery, thereby supporting the system's disaster recovery capabilities and significantly improving the overall system resilience.

[0055] S206. When a reporting failure occurs in the main processing channel, query the current progress level.

[0056] Optionally, during the continuous data transmission process on the main processing channel, transmission failures may occur due to various reasons such as consecutive timeouts, main channel unavailability, the need for retransmission of missed reports, and partial failures caused by network jitter. Once a transmission failure is confirmed and a disaster recovery mechanism needs to be initiated, the system will immediately send a query request to the storage progress watermark service (such as distributed key-value storage or coordination service) to obtain the last globally confirmed success point before the failure occurred, i.e., the current progress watermark. The current progress watermark value represents the irreversible processing progress that the system has completed for the external world at the moment of failure. Under this mechanism, the disaster recovery behavior of the system no longer depends on a single state within the main channel, but on a strongly consistent state snapshot maintained independently in the system. This eliminates the need for the system to replay a large number of logs to infer the breakpoint and avoids information inaccuracies caused by the loss of the main channel instance's memory state. It can instantly identify the starting point of the recovery operation, shorten the failure recovery time, and provide clear and error-free instructions for subsequent automated failover.

[0057] S208. Based on the current progress water level, determine the corresponding playback start position in the backup cache channel. Starting from the playback start position, read the event copy stored in the backup cache channel for continued transmission.

[0058] Optionally, after retrieving the current progress watermark, the system locates the event in the backup cache channel. The current progress watermark represents the last globally confirmed success point before the failure occurred. The position corresponding to the next event after this watermark is the replay start position. Subsequently, starting from this start position, the system sequentially reads the event copies stored in the backup cache channel and re-executes the reporting process. The continued reporting of these events is not simply a retry, but a completely new, controlled processing flow, also adhering to the rules for updating the watermark, thus forming a closed loop. The backup channel acts as a read-only data source, the recovery instance as a new processing engine, and the watermark as a synchronization point; all three work together. Because the recovery start point is precise and the data source is independent and reliable, it ensures that all events that should have been processed during the failure are processed exactly once, without omissions or duplication due to the uncertain state of the main channel. The system service capability can achieve rapid reconstruction from the breakpoint on another logical path, thus strictly meeting the requirements for data reliability and system availability in the high-level service level agreement.

[0059] This specification provides a data reporting method in its embodiments. The method involves acquiring a source event stream to be reported, where each event in the source event stream corresponds to a piece of data to be reported. Each event in the source event stream is submitted in parallel to a main processing channel and a backup cache channel. The main processing channel reports each event sequentially and updates the progress watermark status corresponding to each successfully reported event after each successful report. The backup cache channel stores a complete copy of the source event stream. When a reporting failure occurs in the main processing channel, the current progress watermark is queried. Based on the current progress watermark, the corresponding playback start position in the backup cache channel is determined. Starting from the playback start position, the event copy stored in the backup cache channel is read for continued reporting. This method first abstracts and transforms the discrete data packets traditionally processed in batches into a continuous, structured event sequence, where each event corresponds to a piece of data to be reported with a complete context. This event stream transformation mechanism enables the system to process massive amounts of data in a streaming, incremental manner. After acquiring the event stream, each event in the source event stream is submitted in parallel to the primary processing channel and the backup cache channel. In this step, the system synchronously distributes copies of the same event to two logical channels with different responsibilities. The primary processing channel handles real-time processing and reporting, while the backup cache channel is used for persistent data backup. This parallel submission mechanism ensures the reliable preservation of the original data, avoiding the risk of source data loss due to a single channel failure. Secondly, the primary and backup channels do not block each other, ensuring high availability while maintaining performance, achieving redundancy and disaster recovery capabilities. The core operation of the primary processing channel is to report each event sequentially and update the progress watermark status corresponding to each successfully reported event after each successful report. The watermark is a globally visible, persistent metadata marker that accurately records the boundary of successfully processed events. This allows any component in the distributed system to have a unified understanding of the system's processing progress based on this progress pointer. This provides a precise basis for subsequent breakpoint resumption and primary / backup switching, preventing duplicate or missed reports due to ambiguous states. When the system detects a failure in the main processing channel, it queries the current progress watermark and determines the corresponding playback start position in the backup buffer channel based on the current progress watermark. It then reads the event copy from this position to resume transmission. Through the globally shared watermark status information, the system can know the last successfully confirmed point before the failure occurred, thus promptly resuming event transmission from a known coordinate on the backup channel. This recovery process is accurate, reliable, and automated, greatly shortening the failure recovery time. It ensures that all data to be reported is successfully processed once and only once from the occurrence of the failure to the completion of the recovery, ensuring the complete continuity of system service capabilities. This meets the strict timeliness requirements of high-level service protocols and improves the overall reliability of the system's data reporting tasks, the accuracy of failure recovery, and the level of automation.

[0060] Please see Figure 3, Figure 3 This is a flowchart illustrating a data reporting method provided in another embodiment of this specification.

[0061] like Figure 3 As shown, the data reporting method may include at least:

[0062] S302. Obtain the source event stream to be reported. Each event in the source event stream corresponds to one piece of data to be reported.

[0063] S304. Submit each event in the source event stream to the main processing channel and the backup cache channel in parallel. The main processing channel is used to report each event in sequence and update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the source event stream.

[0064] S306. When a reporting failure occurs in the main processing channel, query the current progress level.

[0065] For details regarding steps S302-S306, please refer to the detailed descriptions in steps S202-S206, which will not be repeated here.

[0066] S308. Based on the current progress waterline, determine the corresponding playback start position in the backup cache channel. Starting from the playback start position, for each event copy read from the backup cache channel, query the deduplication table to see if there is a record that has been successfully reported. If there is no record that has been successfully reported, then perform the reporting operation for the corresponding event copy.

[0067] Optionally, during the fault recovery execution phase, it is necessary to ensure that the data stream recovered from the backup channel can not only fully cover the work missed during the fault, but also strictly avoid any duplicate processing, thereby achieving the goal of final consistency exactly once for each event. Based on this, in the embodiments of this specification, a progress watermark is used to accurately locate the breakpoint, and an operation filtering mechanism based on idempotency uniqueness is used to jointly complete the breakpoint resume operation.

[0068] Specifically, the current progress watermark is a structured metadata entry containing a unique identifier for each successful event. The system needs to map this identifier to the physical storage location of the backup cache channel. For offset-based log systems, this typically means querying or calculating the precise offset of the identifier within a specific partition, with the next position after that offset being the replay start point. This mapping mechanism ensures that the starting point for the recovery job to read data is precisely the first unacknowledged event after the last acknowledged successful event before the primary channel failure, achieving precision in recovery granularity.

[0069] Furthermore, after reading the event copy from the determined starting position, it is not directly reported. Instead, a preliminary deduplication check determines whether the current event has already been successfully reported and does not require re-reporting. At this point, the system will query a separately maintained deduplication table for each event copy to be processed, using its idempotent key (e.g., reportid + reporting date + reporting time + user basic information) as the query key. The deduplication table is a high-performance external storage (such as Redis) that supports fast key-value queries. It records and maintains data based on the idempotent key corresponding to each event, and only performs the reporting operation on the corresponding copy for events that have not yet been successfully reported.

[0070] In one feasible implementation, when the system decision requires replay, a replay task or recovery processing instance can be initiated. This instance first queries the waterline at the time of the failure and, through interaction with the metadata service of the backup cache channel, completes the parsing from the logical waterline to the physical offset, thereby locating the breakpoint. The recovery instance then enters a cyclical processing flow: first, it sequentially reads event copies from the starting position of the backup channel; then, for each event, it extracts its idempotent key and queries the deduplication table; based on the query results, it routes the data, skipping events if they have been successfully reported, and submitting them to the reporting execution module if they have not been successfully reported. Upon successful processing, the deduplication table and the global waterline status in the collaborative waterline manager are updated synchronously. The deduplication table also has a reasonable cleanup cycle, such as 7 days, controlling storage costs by periodically and automatically cleaning up old data. This precise positioning based on water level lines makes the recovery process fast and targeted. Combined with an idempotent filtering mechanism that queries first and then executes, it avoids events that have already been confirmed to be successful from entering the processing flow again, ensuring that the same data is successfully reported at most once. As a result, the system can concentrate resources on dealing with the real data gaps, greatly shortening the recovery time target and improving resource utilization efficiency.

[0071] S310. Determine the replay mode based on the fault type corresponding to the reported fault, and generate the replay conditions corresponding to the replay mode; the replay mode includes at least one of time window replay mode, primary key range replay mode, and error classification replay mode; starting from the replay start position, determine the target event copy that meets the replay conditions in the backup cache channel; and continue to report the target event copy.

[0072] Optionally, after successfully locating the playback start point and filtering completed events, the recovery operation can be further upgraded to a programmable, targeted, and semantically specific precise operation, thereby achieving targeted and controllable recovery. This step is achieved through multi-dimensional data filtering and routing strategies. The system can identify the target event set corresponding to the filtering conditions in the backup data stream based on one or more filtering conditions (the filtering conditions are passively received manual input instructions or automatically obtained based on the fault context), and only resume transmission of events in the set. That is, the playback mode is determined according to the fault type corresponding to the reported fault, and the playback conditions corresponding to the playback mode are generated. Starting from the playback start position, the system determines the target event copy that meets the playback conditions in the backup cache channel and resumes transmission of the target event copy.

[0073] In one feasible implementation, there are three replay modes. (1) Time window replay mode: The filtering dimension is the data timestamp. For example, within the recovery time window after the main channel network interruption, the system allows specifying a start and end time point (such as "yesterday 18:00 to 20:00"). Then, when the recovery engine is executed, it will compare the timestamp of the event with the specified time window and only process the events within the window. (2) Primary key range replay mode: The filtering dimension is the primary key identifier. By specifying one or more primary keys (such as customer 001), the system can accurately locate all events related to a specific object. (3) Error classification replay mode: The filtering dimension is the status of the historical processing results. This mode depends on the system's classification and marking of failed events during previous processing. For example, when the main channel encounters different errors such as network timeout or signature verification failure, the event and its error type metadata will be recorded together in a specific context. During playback, the system can filter out historical events that failed for specific recoverable reasons and retry them based on instructions (such as "only replay network timeout failures"). Permanent errors (such as invalid data formats) do not need to be retried, but will wait for further exception handling.

[0074] Specifically, during the target dataset location and acquisition phase, after receiving the replay start position (from the waterline) and the determined replay conditions, the system recovery engine interacts with the backup cache channel. For storage that supports advanced queries (such as databases), conditional queries can be executed directly; for simple log storage (such as Kafka), it may be necessary to read sequentially from the start position and apply filtering conditions in memory to select events that meet the "time window," "primary key range," or corresponding "error label," forming the final target event replica stream. This ensures that only data that truly needs to be reprocessed enters the replay process. Through precise filtering, the system avoids replaying a large amount of data that has already been successfully processed or is irrelevant to the current fault, shortening the execution time of the recovery task, reducing unnecessary network bandwidth, computing resource consumption, and redundant requests to downstream systems, making the recovery operation more lightweight and agile. At the same time, this design also enables the system to flexibly respond to diverse faults and operational needs, improving the granularity and professionalism of operations and maintenance.

[0075] S312. If the reported fault is an anomaly that cannot be automatically recovered, the reported anomaly event and the corresponding fault information shall be submitted to the anomaly event isolation area, and manual processing instructions for the anomaly event shall be received through the management interface maintained in the anomaly event isolation area.

[0076] Optionally, for failure events that exceed the automatic recovery capability due to root cause service logic errors or unpredictable complex faults, embodiments of this specification also provide a standardized and controlled handling mechanism. That is, if the reported fault is an anomaly that cannot be automatically recovered, the reported failure event and corresponding fault information are submitted to the anomaly event isolation area, and manual handling instructions for the anomaly event are received through the management interface maintained in the anomaly event isolation area. The anomaly event isolation area is a dedicated subsystem with complete state management, data persistence, and operation interfaces.

[0077] Specifically, for various anomalies, the system predefines clear boundaries between "non-recoverable" and "recoverable" faults based on the nature of the error. For example, errors triggered by data ontology violating service rules (such as encryption failure or invalid fields) are clearly distinguished from errors caused by external environmental fluctuations (such as network outages). When the processing logic identifies the former, it triggers a transfer operation, encapsulating the complete data payload of the current event along with its detailed error context (including error codes, stack traces, timestamps, etc.) and submitting it to an isolation zone supported by highly available persistent storage (such as dedicated message topics or database tables). This process ensures that the abnormal event and its diagnostic information are reliably preserved and retains all necessary context for subsequent processing.

[0078] Furthermore, once an anomaly enters the isolation zone, it enters a handling process primarily based on manual intervention. The system can integrate a management console that provides visualization and in-depth diagnostic capabilities, allowing operations personnel to filter anomalies by time, type, status, and other dimensions, and directly view the original failure data and detailed error stacks to quickly pinpoint the root cause of the problem. Building on this, the console further provides online repair capabilities, allowing authorized personnel to correct and edit data content in a controlled environment. Finally, through commands issued via the console, the repaired event can be reinjected as a new, compliant input into the main processing flow or a designated supplementary data channel, allowing it to undergo complete standardized processing, including idempotent verification. Throughout this process, all operations are fully recorded, forming an auditable operational track.

[0079] Optionally, to ensure that any isolated anomalies receive timely attention and form a closed-loop processing mechanism, the system also incorporates an automated monitoring and alarm-driven mechanism. The background service periodically scans the isolation zone to identify unprocessed items whose dwell time exceeds a preset threshold (e.g., 24 hours). For such overdue events, the system automatically triggers an alarm notification, submitting the issue to relevant personnel via messages integrated into email, instant messaging tools, or the unified operations and maintenance platform. In summary, the anomaly isolation and manual intervention mechanism significantly enhances the system's resilience in dealing with various boundary conditions and unknown risks.

[0080] This specification provides a data reporting method in its embodiments. By combining a water level line with a deduplication table, the recovery process is made more precise and secure. The precise water level line ensures that the data is resumed from the exact breakpoint, avoiding the waste of resources from a full scan. Deduplication verification intercepts all successfully processed events, eliminating duplicate reporting caused by retries or inconsistent states, significantly improving recovery efficiency. The system can accurately filter target data subsets for supplementary reporting based on the spatiotemporal range, subject, or nature of the fault. This minimizes the impact of recovery operations, greatly reduces invalid requests to system resources and downstream services, and can flexibly handle complex operation and maintenance scenarios such as local faults and data correction, improving the agility of recovery and the precision of operation and maintenance. Furthermore, through the established anomaly isolation and manual handling mechanism, a reliable final safety boundary is set for the system. Root causes of errors that cannot be automatically handled are isolated, archived, and provided with a visual processing interface. Complex issues requiring further judgment are directed to manual handling, preventing them from blocking the main automated process. This achieves differentiated fault handling and makes all operation and maintenance operations auditable and traceable.

[0081] Please see Figure 4 , Figure 4 This is a structural block diagram of a data reporting device provided in an embodiment of this specification. Figure 4As shown, the data reporting device 400 includes:

[0082] The data preparation module 410 is used to obtain the source event stream to be reported, where each event in the source event stream corresponds to one piece of data to be reported.

[0083] The primary and backup synchronization module 420 is used to submit each event in the source event stream to the primary processing channel and the backup cache channel in parallel. The primary processing channel is used to report each event in sequence and update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the source event stream.

[0084] The progress query module 430 is used to query the current progress level when a reporting failure occurs in the main processing channel.

[0085] The playback reporting module 440 is used to determine the corresponding playback start position in the backup cache channel based on the current progress waterline, and read the event copy stored in the backup cache channel from the playback start position for continued transmission reporting.

[0086] Optionally, the playback reporting module 440 is also used to query the deduplication table for each event copy read from the backup cache channel, starting from the playback start position, whether there is a record that has been successfully reported; if there is no record that has been successfully reported, then the reporting operation for the corresponding event copy is performed.

[0087] Optionally, the deduplication table is recorded and maintained based on the idempotent key corresponding to each event.

[0088] Optionally, the information included in the progress watermark status includes at least the identifier of the successful reporting event, the corresponding reporting channel identifier, and the timestamp.

[0089] Optionally, the replay reporting module 440 is further configured to determine the replay mode based on the fault type corresponding to the reported fault, and generate the replay conditions corresponding to the replay mode; the replay mode includes at least one of time window replay mode, primary key range replay mode, and error classification replay mode; starting from the replay start position, determine the target event copy that meets the replay conditions in the backup cache channel; and continue to report the target event copy.

[0090] Optionally, the data reporting device 400 further includes an anomaly handling module, which, if the reporting failure is an anomaly that cannot be automatically recovered, submits the reporting failure anomaly event and the corresponding fault information to the anomaly event isolation area, and receives manual processing instructions for the anomaly event through the management interface maintained in the anomaly event isolation area.

[0091] In the embodiments of this specification, a data reporting device is provided, wherein a data preparation module is used to acquire a source event stream to be reported, each event in the source event stream corresponding to a piece of data to be reported; a primary and backup synchronization module is used to submit each event in the source event stream to a primary processing channel and a backup cache channel in parallel, the primary processing channel is used to report each event in sequence, and update the progress watermark status corresponding to the currently successfully reported event after each successful report, the backup cache channel is used to store a complete copy of the source event stream; a progress query module is used to query the current progress watermark when a reporting failure occurs in the primary processing channel; a playback reporting module is used to determine the corresponding playback start position in the backup cache channel based on the current progress watermark, and read the event copy stored in the backup cache channel from the playback start position for continued reporting. Firstly, the discrete data packets of traditional batch processing are abstracted and transformed into a continuous, structured event sequence, each event corresponding to a piece of data to be reported with a complete context. This event stream transformation mechanism enables the system to process massive amounts of data in a streaming, incremental manner. After acquiring the event stream, each event in the source event stream is submitted in parallel to the primary processing channel and the backup cache channel. In this step, the system synchronously distributes copies of the same event to two logical channels with different responsibilities. The primary processing channel handles real-time processing and reporting, while the backup cache channel is used for persistent data backup. This parallel submission mechanism ensures the reliable preservation of the original data, avoiding the risk of source data loss due to a single channel failure. Secondly, the primary and backup channels do not block each other, ensuring high availability while maintaining performance, achieving redundancy and disaster recovery capabilities. The core operation of the primary processing channel is to report each event sequentially and update the progress watermark status corresponding to each successfully reported event after each successful report. The watermark is a globally visible, persistent metadata marker that accurately records the boundary of successfully processed events. This allows any component in the distributed system to have a unified understanding of the system's processing progress based on this progress pointer. This provides a precise basis for subsequent breakpoint resumption and primary / backup switching, preventing duplicate or missed reports due to ambiguous states. When the system detects a failure in the main processing channel, it queries the current progress watermark and determines the corresponding playback start position in the backup buffer channel based on the current progress watermark. It then reads the event copy from this position to resume transmission. Through the globally shared watermark status information, the system can know the last successfully confirmed point before the failure occurred, thus promptly resuming event transmission from a known coordinate on the backup channel. This recovery process is accurate, reliable, and automated, greatly shortening the failure recovery time. It ensures that all data to be reported is successfully processed once and only once from the occurrence of the failure to the completion of the recovery, ensuring the complete continuity of system service capabilities. This meets the strict timeliness requirements of high-level service protocols and improves the overall reliability of the system's data reporting tasks, the accuracy of failure recovery, and the level of automation.

[0092] This specification provides a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of any of the methods described above.

[0093] This specification also provides a computer storage medium that can store multiple instructions adapted for loading by a processor and executing the steps of any of the methods described in the above embodiments.

[0094] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of this specification. Figure 5 As shown, terminal 500 may include: at least one terminal processor 501, at least one network interface 504, user interface 503, memory 505, and at least one communication bus 502.

[0095] The communication bus 502 is used to enable communication between these components.

[0096] The user interface 503 may include a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0097] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0098] The terminal processor 501 may include one or more processing cores. The terminal processor 501 connects to various parts within the terminal 500 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by calling data stored in the memory 505. Optionally, the terminal processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The terminal processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the terminal processor 501 and may be implemented as a separate chip.

[0099] The memory 505 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned terminal processor 501. Figure 5 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a data reporting program.

[0100] exist Figure 5In the terminal 500 shown, the user interface 503 is mainly used to provide an input interface for the user and to obtain the user's input data; while the terminal processor 501 can be used to call the data reporting program stored in the memory 505 and specifically perform the following operations:

[0101] Obtain the source event stream to be reported. Each event in the source event stream corresponds to one piece of data to be reported.

[0102] Each event in the source event stream is submitted in parallel to the main processing channel and the backup cache channel. The main processing channel is used to report each event in order and update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the source event stream.

[0103] When a reporting failure occurs in the main processing channel, query the current progress level.

[0104] Based on the current progress level, determine the corresponding playback start position in the backup cache channel. Starting from the playback start position, read the event copy stored in the backup cache channel for continued transmission.

[0105] In some embodiments, when the terminal processor 501 performs the following steps when reading event copies stored in the backup cache channel from the playback start position for continued transmission: starting from the playback start position, for each event copy read from the backup cache channel, query the deduplication table to see if there is a record that has been successfully transmitted; if there is no record that has been successfully transmitted, then perform the transmission operation for the corresponding event copy.

[0106] In some embodiments, the deduplication table is recorded and maintained based on the idempotent key corresponding to each event.

[0107] In some embodiments, the information included in the progress watermark status includes at least the identifier of the successful reporting event, the corresponding reporting channel identifier, and the timestamp.

[0108] In some embodiments, when the terminal processor 501 performs a follow-up transmission report by reading the event copy stored in the backup cache channel starting from the playback start position, it specifically performs the following steps: determining the playback mode according to the fault type corresponding to the transmission fault, and generating playback conditions corresponding to the playback mode; the playback mode includes at least one of time window playback mode, primary key range playback mode, and error classification playback mode; determining the target event copy that meets the playback conditions in the backup cache channel starting from the playback start position; and performing a follow-up transmission report on the target event copy.

[0109] In some embodiments, the terminal processor 501 further performs the following steps: if the reported fault is an exception that cannot be automatically recovered, the failed reporting exception event and the corresponding fault information are submitted to the exception event isolation area, and manual processing instructions for the exception event are received through the management interface maintained in the exception event isolation area.

[0110] In the several embodiments provided in this specification, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0111] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0112] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0113] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0114] In addition, it should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0115] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0117] The above is a description of a data reporting method, apparatus, storage medium, and terminal provided in the embodiments of this specification. For those skilled in the art, based on the ideas of the embodiments of this specification, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this specification.

Claims

1. A data reporting method, the method comprising: Obtain the source event stream to be reported, where each event in the source event stream corresponds to one piece of data to be reported; Each event in the source event stream is submitted in parallel to the main processing channel and the backup cache channel. The main processing channel is used to report each event in sequence and atomically update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the source event stream, and the data structure of the event copy retains the original fields and context information. The progress watermark status is a persistent metadata record used to characterize the work progress that has been confirmed to be permanently completed in the system. The information contained in the progress watermark status includes at least the identifier of the successfully reported event, the corresponding reporting channel identifier, and the timestamp. When the main processing channel experiences a reporting failure, the system queries the current progress level from the service storing the progress level. The current progress level is structured metadata containing a unique identifier for a success event. Based on the current progress level, the unique identifier of the successful event is mapped to the physical storage location of the backup cache channel. The next position of the physical storage location in the backup cache channel is determined as the playback start position. Starting from the playback start position, the event copy stored in the backup cache channel is read for continued transmission.

2. The method according to claim 1, wherein reading the event copy stored in the backup buffer channel for continued transmission from the playback start position comprises: Starting from the playback start position, for each event copy read from the backup cache channel, check the deduplication table to see if there is a record that has been successfully reported; If no record has been successfully reported, then the reporting operation for the corresponding event copy is performed.

3. The method according to claim 2, wherein the deduplication table is recorded and maintained based on the idempotent key corresponding to each event.

4. The method according to claim 1, wherein reading the event copy stored in the backup buffer channel for continued transmission from the playback start position includes: The playback mode is determined based on the fault type corresponding to the reported fault, and the playback conditions corresponding to the playback mode are generated. The replay modes include at least one of the following: time window replay mode, primary key range replay mode, and error classification replay mode. Starting from the playback start position, a target event copy that meets the playback conditions is determined in the backup cache channel; The target event copy is then continuously transmitted.

5. The method according to claim 1, further comprising: If the reported fault is an anomaly that cannot be automatically recovered, the reported failure anomaly event and the corresponding fault information are submitted to the anomaly event isolation area, and manual processing instructions for the anomaly event are received through the management interface maintained in the anomaly event isolation area.

6. A data reporting device, the device comprising: The data preparation module is used to obtain the source event stream to be reported, where each event in the source event stream corresponds to one piece of data to be reported. The primary and backup synchronization module is used to submit each event in the source event stream to the primary processing channel and the backup cache channel in parallel. The primary processing channel is used to report each event in sequence and atomically update the progress watermark status corresponding to the currently successfully reported event after each successful report. The backup cache channel is used to store a complete copy of the source event stream, and the data structure of the event copy retains the original fields and context information. The progress watermark status is a persistent metadata record used to characterize the work progress that has been confirmed to be permanently completed in the system. The information contained in the progress watermark status includes at least the identifier of the successfully reported event, the corresponding reporting channel identifier, and the timestamp. The progress query module is used to query the current progress level from the service that stores the progress level when the main processing channel experiences a reporting failure. The current progress level is structured metadata containing a unique identifier for a success event. The replay reporting module is used to map the unique identifier of the successful event to the physical storage location of the backup cache channel based on the current progress waterline, determine the next position of the physical storage location in the backup cache channel as the replay start position, and read the event copy stored in the backup cache channel for continued transmission reporting starting from the replay start position.

7. A computer program product comprising instructions that, when run on a computer or processor, causes the computer or processor to perform the steps of the method as described in any one of claims 1 to 5.

8. A computer storage medium storing a plurality of instructions adapted for loading by a processor and performing the steps of the method as claimed in any one of claims 1 to 5.

9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as claimed in any one of claims 1 to 5.

Images