A data rapid tracing method and system based on multi-source log analysis

Abstract

A data rapid tracing method and system based on multi-source log analysis, relates to a data processing system or method specially applicable to administrative, commercial, financial, management, supervision or prediction purposes, in the method, the system first generates a transaction document containing a unique serial number and business attribute parameters in response to a business instruction, establishes the association between the two and generates a to-be-confirmed record and a timestamp. Then send the business message containing the business attribute parameters to the receiving end. When receiving the ownership backtracking request of the receiving end, the system calculates the effective time interval based on the current time and the preset time delay threshold, and filters out the target to-be-confirmed record whose business attribute parameters match and whose generated timestamp is within the interval. Finally, the unique serial number is extracted from the target record, and a confirmation receipt is generated and sent to the receiving end system. The present application is used to reduce the computing overhead and false positive rate of the system in accurately positioning specific abnormal transactions in massive logs, and improve the confidence and timeliness of the automatic tracing result.

Description

Technical Field

[0001] This application belongs to the field of data processing systems or methods specifically applicable to administrative, commercial, financial, management, supervisory or predictive purposes, and in particular relates to a method and system for rapid data tracing based on multi-source log analysis. Background Technology

[0002] Cross-border e-commerce and international financial settlement involve massive amounts of sensitive user privacy and transaction data. Cross-border transaction processing systems are typically divided into multiple high-security domains, such as a domestic core domain, a cross-border data exchange domain, and an overseas settlement domain. Controlled data transfer between these domains is achieved through physical one-way optical gates or logical isolation gateways. Under this complex isolated network architecture, when faced with discrepancies in cross-border fund reconciliation, compliance audits, or investigations into transaction anomalies, it is necessary to trace the entire lifecycle of a transaction across multiple isolated domains.

[0003] A widely adopted approach in related technologies is a method based on centralized log collection and rule-based correlation analysis. This method involves deploying log collection probes in various independent security domains to capture log streams generated by business systems, middleware, and databases in real time. Then, it uses secure data transmission channels to aggregate the dispersed logs to a unified big data analytics platform. On this platform, the system extracts timestamps, source IP addresses, and unique business identifiers from the logs according to preset correlation rules to perform cross-domain log matching and splicing. Through this field-level logical correlation, log fragments scattered across different security domains are linked together, thereby achieving a certain degree of logical reorganization of cross-domain data and providing a feasible method for end-to-end monitoring.

[0004] However, as data flows from domestic to foreign domains via the gateway, it undergoes deep de-identification, encryption, and even business ID remapping, which disrupts the consistency of key related fields in the logs. This significantly reduces the accuracy of static field matching, which is crucial for related technologies. Furthermore, the unpredictable queuing and processing times inherent in the data transfer mechanisms of cross-domain isolation devices, coupled with slight clock deviations between different management entities, make fuzzy matching based on time windows prone to ambiguity under massive transaction concurrency. This increases the computational overhead and false alarm rate for accurately locating specific abnormal transactions in massive logs, reducing the confidence and timeliness of automated traceability results. Summary of the Invention

[0005] This application provides a method and system for rapid data tracing based on multi-source log analysis, which reduces the computational overhead and false alarm rate of accurately locating specific abnormal transactions in massive logs, and improves the confidence and timeliness of automated tracing results.

[0006] Firstly, this application provides a method for rapid data tracing based on multi-source log analysis. In response to a business initiation command, it generates an original transaction document containing a unique serial number and business attribute parameters; establishes a correlation between the unique serial number and the business attribute parameters, generates a record to be confirmed and a generation timestamp; sends a business message to the receiving system, the business message containing business attribute parameters; receives an ownership lookup request from the receiving system and determines the current system time at which the ownership lookup request was received, the ownership lookup request containing the attribute parameters to be queried; calculates a valid time interval based on the current system time and a preset delay threshold; filters out target records to be confirmed that have the same business attribute parameters as the attribute parameters to be queried and whose generation timestamps are within the valid time interval; reads the unique serial number from the target record to be confirmed; generates a confirmation receipt containing the unique serial number and sends it to the receiving system.

[0007] By adopting the above technical solution, and by introducing a unique serial number into the original transaction documents and establishing its association with business attribute parameters, combined with timestamp records and calculation of valid time intervals, the system can quickly locate relevant transaction records upon receiving an ownership lookup request. After the system calculates the valid time interval based on the current time and a preset delay threshold, the query scope can be limited to a specific time window, reducing the overhead of full data scanning. Through matching and filtering business attribute parameters with the attribute parameters to be queried, the system can accurately identify the target record to be confirmed and extract a unique serial number to generate a confirmation receipt, reducing system resource consumption during data tracing, improving query efficiency, and enhancing the accuracy of data tracking. Using a unique serial number as an index identifier improves the uniqueness guarantee of transaction records and reduces the possibility of data confusion in high-concurrency scenarios.

[0008] In conjunction with some implementation methods of the first aspect, in some implementation methods, target records to be confirmed that have business attribute parameters that are consistent with the query attribute parameters and whose generation timestamps are within the valid time interval are selected. Specifically, this includes: extracting key business fields from the query attribute parameters and calculating the key business fields using a preset hash algorithm to obtain a target hash index key; using the target hash index key to locate the target hash bucket in the hash bucket array of the local memory database to determine the target hash bucket corresponding to the key business field, wherein the target hash bucket stores multiple candidate record lists arranged in reverse order of generation timestamps; traversing the candidate record lists in the target hash bucket to compare the overlap between the generation timestamps of the records in the candidate record lists and the valid time intervals; when the generation timestamp of any record in the candidate record list is within the valid time interval and the corresponding business attribute parameters fully match the query attribute parameters, the record corresponding to the generation timestamp is determined as the target record to be confirmed and the traversal stops.

[0009] By adopting the above technical solution, a highly efficient data retrieval mechanism is established by introducing a hash algorithm to calculate key business fields and using a hash bucket array to store candidate records. Since the records in the hash buckets are arranged in reverse chronological order of their generation timestamps, the overlap between timestamps and valid time intervals can be quickly determined when traversing the candidate record list. Once a record that meets the time conditions and whose business attribute parameters are fully matched is found, the system immediately stops traversing, avoiding further invalid data comparisons. This hash index-based positioning method, combined with the reverse chronological order of timestamps and the timely termination traversal strategy, reduces the time complexity of data retrieval. Simultaneously, the use of a local in-memory database to store the hash bucket array reduces disk I / O operations, improving the system's response speed in high-concurrency scenarios. Based on the uniform distribution characteristics of the hash algorithm, the data storage balance is improved, increasing the efficiency of system resource utilization.

[0010] In conjunction with some implementation methods of the first aspect, in some implementation methods, target records to be confirmed that have business attribute parameters that are consistent with the query attribute parameters and whose generated timestamps are within the valid time interval are selected. Specifically, this includes: inputting the query attribute parameters into a preset Bloom filter for mapping detection, where the preset Bloom filter is a set of bit arrays constructed based on the business attribute parameters of historically generated records to be confirmed; if the output existence determination is negative, returning a query failure response to the receiving system; if the output existence determination is positive, constructing a time-sliced ​​query cursor based on the start and end times of the valid time interval; using the time-sliced ​​query cursor to obtain all candidate datasets within the valid time interval from a preset time series database; and selecting records in the candidate datasets that fully match the query attribute parameters as target records to be confirmed.

[0011] By employing the above technical solution, the system can quickly predict query attribute parameters using a Bloom filter, enabling early detection of non-existent query requests and avoiding unnecessary subsequent database accesses. For requests detected by the Bloom filter, the system performs range queries in the time-series database using a time-sharded query cursor, limiting data retrieval to candidate datasets within the valid time interval. The rapid judgment characteristic of the Bloom filter reduces invalid database accesses and lowers system load. Simultaneously, the time-sharding mechanism of the time-series database improves the efficiency of range queries. By performing full matching between candidate datasets meeting time conditions and the query attribute parameters, the accuracy of query results is improved, allowing the system to optimize query performance while ensuring data correctness.

[0012] In some implementations of the first aspect, after reading the unique serial number from the target record to be confirmed, the method further includes: reading the current transaction status identifier of the target record to be confirmed; and determining whether the current transaction status identifier is in a waiting confirmation state or a timeout state.

[0013] If so, based on the received ownership lookup request, it is determined that the business message has reached the receiving system; the transaction status identifier of the target record to be confirmed is updated to "completed", and a status change log is generated; the unlocking of inventory lock or posting and reconciliation associated with the target record to be confirmed is triggered; a confirmation receipt containing a unique serial number and the updated "completed" status identifier is generated and sent to the receiving system.

[0014] By adopting the above technical solution and adding a transaction status judgment and update mechanism after reading the unique serial number, the system achieves precise control over the transaction lifecycle. When the system confirms that the current transaction status is in a pending confirmation or timeout state, it determines that the business message has been delivered based on the received ownership lookup request, then updates the transaction status and generates a change log, enabling the system to more accurately track the transaction processing progress and reduce the occurrence of transaction status asynchrony. By triggering the release of inventory locks or accounting reconciliation operations, the system promptly releases business resources and improves resource utilization efficiency. Including the updated status identifier when generating confirmation receipts enhances the timeliness of system status synchronization and improves the reliability of the transaction processing process. This state-transition-based processing method improves the system's transaction integrity and enhances the ability to trace abnormal situations.

[0015] In conjunction with some implementation methods of the first aspect, in some implementation methods, triggering the unlocking of inventory locks or the posting and reconciliation of the target record to be confirmed specifically includes: constructing a transaction message containing a unique serial number and an unlocking operation instruction, and writing the transaction message to a local message table in the same database instance as the target record to be confirmed; updating the transaction status identifier to completed and writing to the local message table as an atomic transaction commit; if the atomic transaction commit is successful, starting an asynchronous task thread to scan the local message table and sending the transaction message to the inventory management subsystem or the accounting subsystem to perform the unlocking of inventory locks or the posting and reconciliation of accounts.

[0016] By adopting the above technical solution, writing transactional messages containing unique serial numbers and unlocking operation instructions to a local message table, and committing message writing and status update operations as atomic transactions, the consistency between status updates and message writing can be improved. After a successful atomic transaction commit, the asynchronous task thread scans the local message table and sends unlocking instructions to relevant subsystems, reducing the implementation complexity of distributed transactions. Since message writing and status updates are performed within the same database instance, cross-system distributed transactions are avoided, improving system processing performance. The asynchronous task thread performs message delivery by scanning the local message table; even if a network anomaly occurs during message delivery, it can still re-deliver after the network recovers, improving the reliability of the unlocking operation, reducing direct dependencies between systems, and enhancing system scalability. Simultaneously, the transactional message mechanism based on the local message table provides message persistence capabilities, enhancing the system's fault tolerance.

[0017] In conjunction with some implementations of the first aspect, in some implementations, after updating the transaction status identifier of the target record to be confirmed to "completed" and generating a status change log, the method further includes: generating a status verification instruction corresponding to the target record to be confirmed, wherein the status verification instruction contains an inference correction identifier; sending the status verification instruction to the receiving system and starting a verification waiting timer; if a persistent confirmation message is received from the receiving system before the verification waiting timer expires, the target record to be confirmed is marked as "finally completed"; if no persistent confirmation message is received after the verification waiting timer expires, or an abnormal message is received from the receiving system, the current transaction status identifier of the target record to be confirmed is rolled back to "abnormal pending processing" status.

[0018] By adopting the above technical solution, and sending a status verification command containing an inference correction flag to the receiving system, and starting a verification waiting timer for timeout monitoring, data persistence anomalies in the receiving system can be detected in a timely manner. The system dynamically adjusts the status of records to be confirmed based on the persistence confirmation message or anomaly message returned by the receiving end, making the record status more accurately reflect the actual business processing results. When the receiving end returns a persistence confirmation message normally, the record is marked as finally completed, improving the reliability of the data status. In case of anomalies, the system rolls back the record status to an abnormal pending state, enhancing the system's anomaly handling capabilities. Through status verification and timeout control mechanisms, the accuracy of system processing results is improved, data inconsistency problems caused by network latency or system anomalies are reduced, and the system's data consistency level is enhanced.

[0019] In conjunction with some implementation methods of the first aspect, in some implementation methods, a status verification instruction corresponding to the target record to be confirmed is generated, specifically including: obtaining the business credential information carried by the receiving system when sending the status reverse lookup request; determining whether the business data stored in the target record to be confirmed is consistent with the business credential information; if consistent, generating a status verification instruction; if inconsistent, generating a data conflict alarm and stopping sending information to the receiving system that the target record to be confirmed is in a successful transaction status.

[0020] By adopting the above technical solution, a data consistency verification mechanism is established by comparing the business data stored in the target confirmation record with the business credential information carried by the receiving system. When data inconsistency is detected, the system generates a data conflict alarm and stops sending transaction success status information, thus improving the accuracy of status data. Generating status verification instructions based on data comparison reduces the propagation of erroneous statuses caused by data inconsistency. Timely detection and handling of data conflicts reduces the likelihood of anomalies in subsequent business processing stages. This verification mechanism based on business credential information enhances the system's data quality control capabilities, improves the reliability of business processing, and the preventative data verification mechanism further enhances the system's error prevention capabilities.

[0021] Secondly, embodiments of this application provide a data rapid tracing system based on multi-source log analysis. The data rapid tracing system based on multi-source log analysis includes: one or more processors and a memory; the memory is coupled to one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and one or more processors call the computer instructions to cause the system to perform the method described in the first aspect and any possible implementation of the first aspect.

[0022] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a system, cause the system to perform the method described in the first aspect and any possible implementation thereof.

[0023] Fourthly, embodiments of this application provide a computer program product that, when run on a system, causes the system to execute the method described in any possible implementation of the first aspect.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0025] 1. This application provides a rapid data tracing method based on multi-source log analysis. By introducing a unique serial number into the original transaction documents and establishing its association with business attribute parameters, combined with timestamp records and calculation of valid time intervals, the system can quickly locate relevant transaction records upon receiving an ownership lookup request. After the system calculates the valid time interval based on the current time and a preset delay threshold, the query scope can be limited to a specific time window, reducing the overhead of full data scanning. Through matching and filtering business attribute parameters with the attribute parameters to be queried, the system can accurately identify the target record to be confirmed and extract a unique serial number to generate a confirmation receipt, reducing system resource consumption during data tracing, improving query efficiency, and enhancing the accuracy of data tracking. The use of a unique serial number as an index identifier improves the uniqueness guarantee of transaction records and reduces the possibility of data confusion in high-concurrency scenarios.

[0026] 2. This application provides a rapid data tracing method based on multi-source log analysis. By adding a transaction status judgment and update mechanism after reading the unique serial number, the system achieves precise control over the transaction lifecycle. When the system confirms that the current transaction status is in a pending confirmation or timeout state, it determines that the business message has been delivered based on the received ownership lookup request, then updates the transaction status and generates a change log, enabling the system to more accurately track the transaction processing progress and reduce the occurrence of transaction status asynchrony. By triggering the release of inventory locks or accounting reconciliation operations, the system promptly releases business resources and improves resource utilization efficiency. The updated status identifier is included when generating the confirmation receipt, enhancing the timeliness of system status synchronization and improving the reliability of the transaction processing process. This state transition-based processing method improves the transaction integrity of the system and enhances the tracing capability in abnormal situations.

[0027] 3. This application provides a rapid data tracing method based on multi-source log analysis. By sending a status verification command containing an inference correction flag to the receiving system and starting a verification waiting timer for timeout monitoring, data persistence anomalies in the receiving system can be detected in a timely manner. The system dynamically adjusts the status of records to be confirmed based on the persistence confirmation message or anomaly message returned by the receiving end, making the record status more accurately reflect the actual business processing results. When the receiving end returns a persistence confirmation message normally, the record is marked as finally completed, improving the reliability of the data status. In case of anomalies, the system rolls back the record status to an abnormal pending state, enhancing the system's anomaly handling capabilities. Through status verification and timeout control mechanisms, the accuracy of system processing results is improved, data inconsistency problems caused by network latency or system anomalies are reduced, and the system's data consistency level is improved. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a method for rapid data tracing based on multi-source log analysis in an embodiment of this application.

[0029] Figure 2 This is another flowchart illustrating a method for rapid data tracing based on multi-source log analysis in an embodiment of this application.

[0030] Figure 3 This is another flowchart illustrating a method for rapid data tracing based on multi-source log analysis in the embodiments of this application.

[0031] Figure 4 This is a schematic diagram of the physical device structure of a data rapid tracing system based on multi-source log analysis provided in an embodiment of this application. Detailed Implementation

[0032] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0033] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0034] The following example is used in conjunction with Figure 1 This application describes a method for rapid data tracing based on multi-source log analysis in its embodiments: Please refer to... Figure 1 This is a flowchart illustrating a method for rapid data tracing based on multi-source log analysis in an embodiment of this application.

[0035] S101. In response to the business initiation instruction, generate the original transaction document;

[0036] In response to a business initiation command, an original transaction document is generated. This document contains a unique serial number and business attribute parameters. A business initiation command is an electronic signal or command message triggered by an external user terminal, internal scheduled task system, or third-party interface, designed to initiate a specific business interaction or data processing flow. This command typically carries basic request metadata, such as the request source IP, request time, and request type identifier. The original transaction document is the first structured data object instantiated in memory or created in persistent storage based on the received business initiation command; it serves as the foundation for all subsequent processing logic. A unique serial number is a string or number sequence that is unique and non-repeatable across the entire system. It is used to accurately identify the transaction in a distributed environment, and its generation mechanism typically does not depend on the data content of the business itself, ensuring independence. Business attribute parameters are a set of key-value pairs describing the specific content of the transaction, including but not limited to transaction amount, transaction currency, payer account information, payee account information, product code, and transaction channel identifier. During the process of responding to instructions and generating documents, the system first verifies the legality and completeness of the instructions. If the verification passes, it calls an internal object factory or builder to transform the payload in the instructions into a standardized raw transaction document format. At this point, a unique serial number is automatically assigned and written to the document header, while business attribute parameters are filled into the document's data body. If multiple thresholds are involved, such as amount limits or frequency control thresholds, the system will perform a pre-judgment before generating the document to ensure that the business attribute parameters do not trigger the risk control circuit breaker.

[0037] One approach to responding to business initiation commands and generating original transaction documents is a synchronous blocking processing architecture. In this architecture, the system listens on HTTP / HTTPS ports through a web container deployed on the application server. When a business initiation command arrives as a POST request, the control layer component intercepts the request and parses the JSON-formatted request body. Subsequently, the document creation method in the business service layer is called. Internally, the service layer uses a UUID generation algorithm or an ID generator based on a database auto-incrementing sequence to generate a unique serial number. Next, a POJO object is instantiated as the original transaction document, and the parsed business attribute parameters are injected into this object via reflection or setter methods. Finally, this object is temporarily stored in the thread's context for use in subsequent steps. Another implementation approach is an asynchronous message-driven architecture. Business initiation commands are first sent to a specific topic in a high-throughput message queue. The backend consumer service subscribes to this topic, and when a new command message is retrieved, it triggers the consumption logic. The consumer service integrates a distributed ID generator based on the snowflake algorithm, which can quickly generate unique, non-repeating serial numbers locally, avoiding network I / O overhead. Subsequently, consumers use the builder pattern to map message content to original transaction document objects. This approach can effectively smooth out peaks and valleys, adapting to document generation needs in high-concurrency scenarios.

[0038] S102. Establish the association between the unique serial number and the business attribute parameters, generate the record to be confirmed and generate the timestamp;

[0039] Establishing a relationship refers to using the unique serial number generated in step S101 as the primary key or index key, and the business attribute parameters as the associated data or payload content, binding the two logically or physically. This allows a set of business attribute parameters to be uniquely identified through the unique serial number, and vice versa. A record awaiting confirmation is a temporary or persistent data representation of this relationship within the system, representing a transaction in an intermediate state of "initiated but not yet finalized." The generation timestamp is the precise time value provided by the system clock at the instant the system completes the above association operation and creates the record awaiting confirmation. It is typically accurate to milliseconds or microseconds and used for subsequent time window determination. This timestamp not only records the moment the data was generated but also serves as the basis for subsequent data lifecycle management. During this process, the system extracts the data from the original transaction documents and reassembles it into the data structure of the record awaiting confirmation. This structure contains three core fields: the ID field (stores the unique serial number), the Data field (stores business attribute parameters), and the Time field (stores the generation timestamp). The system must ensure that the data in these three fields is atomic when written to the storage medium, that is, either all are written successfully or all fail and are rolled back, to prevent data with only serial numbers but no parameters from appearing.

[0040] One way to establish relationships and generate records to be confirmed is using transactional insertion techniques in relational databases. The system configures a dedicated table for pending transactions with three main fields: `serial_no` (primary key, corresponding to a unique serial number), `business_data` (JSON text type, storing serialized business attribute parameters), and `create_time` (timestamp type). The system starts a database transaction, executes an INSERT statement, and writes the data in memory to this table. The database engine automatically populates the `create_time` field using the current server time, or the current time is passed in from the application layer. After the transaction is committed, the relationship is persistently established. Another approach is to utilize a high-performance key-value storage system. The system uses the unique serial number as the key and constructs a hash structure or Protobuf serialized object containing business attribute parameters and the current timestamp as the value. The storage system's SET command is then called to write this key-value pair to the in-memory database. To facilitate subsequent time-based queries, the system can also maintain an ordered set, using the generated timestamp as the score and the unique serial number as the member. This allows for the establishment of time-based index relationships even in a non-relational database, enabling fast writes under high concurrency.

[0041] S103. Send a service message to the receiving system;

[0042] The system sends business messages to the receiving system, which contain business attribute parameters. The receiving system refers to downstream business platforms, third-party payment gateways, regulatory data centers, or partner API interface services that interact with the current system. This system is responsible for receiving requests from the current system and performing subsequent business processing or ownership confirmation. A business message is a data packet encapsulated according to the communication protocol and data format standards specified by the receiving system. It contains not only core business attribute parameters but may also include signatures for authentication, encrypted sensitive information blocks, and protocol header information. The sending action refers to the process of transmitting the encapsulated business message to the receiving system via network transmission media. In this step, the system first needs to convert the internal raw transaction documents into an externally recognizable message format according to the interface documentation. Specifically, the business message must contain business attribute parameters so that the receiving system can initiate subsequent reverse lookup requests based on these parameters. The system typically establishes a network connection before sending and waits for an acknowledgment signal from the transport layer after sending, but not necessarily for the processing result from the business layer, to achieve non-blocking and efficient communication.

[0043] One way to send business messages to the receiving system is through RESTful API calls based on the HTTP / HTTPS protocol. The system integrates an HTTP client component, configuring the receiving system's URL, port, and timeout. The system serializes the business attribute parameters into a JSON string and sets the Content-Type in the HTTP header to application / json. For secure transmission, an SSL certificate is also required to establish a secure channel. After constructing the Request object, the POST method is executed to send the message. The system captures the HTTP status code; a 200 series status code indicates successful transmission. Another approach is to use long-connection socket communication or an RPC (Remote Procedure Call) framework based on the TCP / IP protocol. The system and the receiving system pre-establish and maintain a long TCP connection, or use stubs generated by frameworks such as gRPC or Dubbo. The system encodes the business attribute parameters according to binary serialization protocols such as Protobuf or Thrift to compress the data volume. The binary stream is directly pushed to the receiving end through the long-connection channel. This method reduces handshake overhead and is suitable for internal microservice calls or financial leased line transmissions with extremely high real-time requirements.

[0044] S104. Receive a title lookup request from the receiving system and determine the current system time when the title lookup request was received;

[0045] The system receives a title lookup request from the receiving system and determines the current system time at which the request was received. The title lookup request contains attribute parameters to be queried. A title lookup request is a callback instruction initiated by the receiving system after receiving a business message to the current system to verify the authenticity of the transaction, prevent fraud, or complete a two-way handshake. The core purpose of this request is to ask: "Was this transaction containing specific business attribute parameters indeed initiated by you?" The attribute parameters to be queried in the request are usually key fields extracted by the receiving system from previously received business messages, such as order number, amount, and account number. The current system time refers to the moment the current system receives the title lookup request, determined by the local clock of the server node receiving the request. Determining the current system time is the cornerstone for subsequent calculations of the valid time interval. This step requires the system to have the ability to listen for external requests and capture the time point with extremely low latency the instant the request arrives. The system needs to parse the request message, extract the attribute parameters to be queried, temporarily store them in memory, and add a receiving timestamp. This timestamp must be accurate because it directly determines the time window range for the callback query.

[0046] One way to implement receiving ownership lookup requests and determining the specific time is through a Servlet listener mechanism based on a web container. The system exposes a specific URL path as the lookup interface. When the receiving system initiates an HTTP POST request, the Servlet container's thread pool allocates a worker thread to handle the request. In the first line of the business logic code, `System.currentTimeMillis()` or `Instant.now()` is immediately called to obtain the current system time and assigned to a local variable. Subsequently, a JSON parsing library is used to deserialize the object containing the queried attribute parameters from the HTTP body. Another implementation is a message queue (MQ) subscription model. If the receiving system is decoupled from the current system via a message queue (MQ), the current system acts as a consumer subscribing to the lookup request body. When a message arrives at the MQ client's listener, the listener triggers a callback function. At the entry point of the callback function, the current server time is immediately recorded as the reception time. Then, the message body is parsed, and the queried attribute parameters are extracted. In this approach, the reception time is actually the time the message is consumed, making it suitable for asynchronous processing scenarios.

[0047] S105. Based on the current system time and the preset delay threshold, the effective time interval is calculated;

[0048] The preset latency threshold is a time difference constant set by the system administrator based on historical network conditions, system processing time, and business tolerance. It represents the maximum reasonable delay that may exist between the sending of a business message and the receiving end initiating a reverse lookup request, including network transmission latency, receiving end processing latency, and queue waiting latency. The effective time interval is a closed or semi-closed time period that defines the time search range for data tracing. The calculation logic for this interval is usually derived backward from the current system time determined in step S104. Specifically, the end time of the effective time interval is usually the current system time, while the start time is the current system time minus the preset latency threshold. Through this calculation, the system narrows the search range of all data to a highly targeted time window, significantly excluding irrelevant historical data generated outside this window. When multiple thresholds are involved, such as network jitter thresholds and "business processing thresholds," the preset latency threshold should be the sum of these sub-thresholds or the maximum value to ensure coverage of all reasonable latency scenarios.

[0049] The specific method for calculating the effective time interval based on the current system time and a preset latency threshold can be achieved using basic time arithmetic operations. Assume the current system time is T_{current}, and the preset latency threshold is ΔT (e.g., 3000 milliseconds). The system uses the date and time library provided by the programming language (such as the java.time package in Java 8) to perform a subtraction operation: T_{start} = T_{current} - ΔT. Simultaneously, T_{end} is set to T_{current}. This ultimately constructs a time interval object, Range. <time>=[T_{start}, T_{end}]. This object will be passed to subsequent database queries or memory retrieval modules. Another implementation is to use a dynamic configuration center combined with a sliding window algorithm. The preset latency threshold ΔT is not a hard-coded constant but is stored in the configuration center. The system dynamically retrieves this configuration at runtime.

[0050] S106. Filter out target records to be confirmed that have the same business attribute parameters as the attribute parameters to be queried and whose generated timestamps are within the valid time interval.

[0051] The process of selecting target records to be confirmed that match the business attribute parameters and the query attribute parameters, and whose generation timestamps are within the valid time interval, includes: extracting key business fields from the query attribute parameters and calculating the target hash index key using a preset hash algorithm; locating the target hash bucket in the hash bucket array of the local in-memory database using the target hash index key, and determining the target hash bucket corresponding to the key business field. Each target hash bucket stores multiple candidate record lists arranged in descending order of generation timestamp; traversing the candidate record lists in the target hash bucket to compare the overlap between the generation timestamps and the valid time intervals; and when the generation timestamp of any record in a candidate record list is within the valid time interval, and the corresponding business attribute parameter fully matches the query attribute parameter, the record corresponding to the generation timestamp is identified as the target record to be confirmed, and the traversal stops.

[0052] Preferably, the attribute parameters to be queried can also be input into a preset Bloom filter for mapping detection. The preset Bloom filter is a set of bit arrays constructed based on the business attribute parameters of historically generated records to be confirmed. If the output existence determination is negative, a query failure response is returned to the receiving system. If the output existence determination is positive, a time-sliced ​​query cursor is constructed based on the start and end times of the valid time interval. The time-sliced ​​query cursor is used to obtain all candidate datasets located within the valid time interval in the preset time series database. Records in the candidate dataset that fully match the attribute parameters to be queried are selected as target records to be confirmed.

[0053] The filtering process involves matching across two dimensions: time and content. A target record to be confirmed is a record that simultaneously meets the following two conditions: first, its generation timestamp falls within the valid time interval calculated by S105; second, its business attribute parameters are completely consistent with the query attribute parameters received by S104.

[0054] Option 1 (based on hash bucket array): Extract key business fields from the attribute parameters to be queried, and calculate the target hash index key using a preset hash algorithm. Use this key to locate the target hash bucket in the hash bucket array of the local in-memory database. This bucket stores multiple candidate records, which are arranged in a linked list in reverse order of their generation timestamps. The system traverses this linked list, first checking if the record's timestamp is within the valid time interval. If a record's timestamp is found to be earlier than the start time of the valid time interval, since the linked list is in reverse order, it means that subsequent records are earlier, so traversal can be stopped immediately. For records with matching timestamps, the system then compares whether all business attribute parameters match.

[0055] Option 2 (based on Bloom filters and a time series database): First, the attribute parameters to be queried are input into a preset Bloom filter. A Bloom filter is a set of bit arrays used to quickly determine whether an element "might exist." If the determination is "no," then it absolutely does not exist, and a failure is returned directly, avoiding subsequent expensive database queries. If the determination is "yes," a time-sliced ​​query cursor is constructed, and all candidate datasets are retrieved from a preset time series database (such as InfluxDB or OpenTSDB) according to the valid time interval. Finally, these candidate data are precisely compared one by one with the attribute parameters in memory.

[0056] One implementation of Scheme 1 (based on a hash bucket array) can employ a custom in-memory hash table structure. The system pre-allocates a large array in memory as the hash bucket array. When a new record is generated, its hash value is calculated and inserted at the head of the corresponding bucket's linked list. During queries, the system uses the same hash algorithm to locate the bucket index. To reduce lock contention, segmented locking techniques can be used, similar to the implementation principle of ConcurrentHashMap. When traversing the linked list, pointers are used for fast access to node data, directly comparing timestamps and attribute parameters in byte array form in the CPU cache, achieving microsecond-level lookups. Another implementation method combines Redis's Hash and List structures. The hash index key is used as the Redis Key, and the Value is stored in a List. Although Redis Lists do not natively support timestamp indexing, the application layer can maintain the List's order. During queries, the LRANGE command is used to retrieve the first N records from the List, which are then traversed and filtered in the application server's memory.

[0057] The second approach (based on a Bloom filter and a time series database) can be implemented using Redis's Bitmap data structure. Multiple different hash functions are used to map the query parameters, setting the corresponding positions in the Bitmap to 1. During the query, it is checked whether all these positions are 1. If the check passes, Elasticsearch is used as the storage engine for the time series data. Elasticsearch's Range Query feature is used to set GTE (greater than or equal to) and LTE (less than or equal to) conditions for the timestamp field, while simultaneously using business attribute parameters as Term Query conditions for a joint query. Elasticsearch's internal inverted index and BKD tree can quickly return a list of documents matching the time range and attributes. Another approach is to use the Guava library's BloomFilter class to build a filter in local memory, combined with the ClickHouse database. ClickHouse offers excellent performance in handling time-range queries of massive amounts of log data. The system first performs a preliminary screening using a local Guava filter. Once the selected logs pass the initial screening, it constructs an SQL statement: SELECT * FROM logs WHERE time>=? AND time<=? AND attributes=?, and sends it to the ClickHouse cluster for execution to obtain the final result.

[0058] S107. Read the unique serial number from the target pending confirmation record;

[0059] Reading refers to the system's operation of accessing storage media or memory addresses to obtain the value of a target data field. After successfully locking onto the unique target record to be confirmed in step S106, the record exists as a complete data object or data row in the system's current processing context. The unique serial number, as the primary key or core identifier of the record, is stored in a specific field location. This step, seemingly simple, is actually the ultimate goal of data tracing—that is, to retrieve the system's internal unique identifier through business attributes. The read operation needs to ensure data integrity, meaning the read serial number must be non-empty and in a valid format. If the target record to be confirmed is stored in a database, the read operation involves parsing the fields of the result set; if the record is in a memory object, it involves accessing the object's attributes. After reading, the serial number will be temporarily stored in a local variable, ready for use in constructing a receipt.

[0060] S108. Generate a confirmation receipt containing a unique serial number and send it to the receiving system.

[0061] A confirmation receipt is a formal response message issued by the system to the receiving system's ownership lookup request. This message not only informs the receiving system that the query was successful, but more importantly, it carries a unique serial number traced internally, serving as proof of transaction reconciliation between the two parties. The process of generating a confirmation receipt includes constructing the response message text, filling in the status code, writing the unique serial number, and any additional information (such as confirmation time and system signature). Sending it to the receiving system means transmitting the constructed receipt message back to the requesting party over the network. This step marks the closure of the entire rapid tracing process. The sending process should follow a request-response model, typically using the connection channel established in step S104 for synchronous return, or asynchronous notification via a callback URL. The system must ensure that the receipt transmission is reliable; failure to send it may lead the receiving system to mistakenly determine that the transaction is not owned.

[0062] One way to generate and send an acknowledgment receipt is by directly writing it into the HTTP response stream. In the web service framework, the return value of the processing method is designed as an entity object containing the response body and the HTTP status code. The system creates a Map or DTO object, storing key-value pairs {"code": 200, "msg": "OK", "serialNo": "extracted_value"}. The framework automatically serializes this object into JSON format using a message converter and writes it to the TCP buffer via the HttpServletResponse output stream, directly returning it to the waiting client. Another implementation is to generate an XML receipt with a digital signature. The system uses an XML builder (such as DOM or SAX) to assemble the message, storing a unique serial number... <serialno>In the label. To prevent the receipt from being tampered with, the system uses a private key to encrypt the digest of the receipt content, generates a digital signature, and places it in the label. <signature>The entire XML string is then sent back to the receiving end as the body of the SOAP protocol via a Web Service interface. This method is suitable for cross-institutional financial transactions with extremely high security requirements.

[0063] In the above embodiments, by introducing a unique serial number into the original transaction document and establishing its association with business attribute parameters, combined with timestamp records and calculations of valid time intervals, the system can quickly locate relevant transaction records upon receiving an ownership lookup request. Once the system calculates the valid time interval based on the current time and a preset delay threshold, the query scope can be limited to a specific time window, reducing the overhead of full data scanning. Through matching and filtering business attribute parameters with the attribute parameters to be queried, the system can accurately identify the target record to be confirmed and extract a unique serial number to generate a confirmation receipt, reducing system resource consumption during data tracing, improving query efficiency, and enhancing the accuracy of data tracking. Using a unique serial number as an index identifier improves the uniqueness guarantee of transaction records and reduces the possibility of data confusion in high-concurrency scenarios.

[0064] The above embodiments describe the basic data traceability process. To further improve the reliability and integrity of the system processing, another embodiment of this application adds a refined management scheme for transaction status after successfully locating the target record to be confirmed and reading the unique serial number. The following is a combination of... Figure 2 Another method for rapid data tracing based on multi-source log analysis is described in the embodiments of this application:

[0065] Please see Figure 2 This is another flowchart illustrating a method for rapid data tracing based on multi-source log analysis in an embodiment of this application.

[0066] S201. Read the current transaction status identifier of the target record to be confirmed;

[0067] The target record to be confirmed refers to a business data entry that has been initially located by the system during the data tracing process, but has not yet completed final status synchronization or confirmation. It can be stored in a relational database, NoSQL database, or distributed cache. The current transaction status identifier is a digital tag or enumeration value used to characterize the stage of the record in the business lifecycle, typically including but not limited to initialization, processing, awaiting confirmation, timeout, completed, and failed states. The read operation refers to the process by which the system retrieves this identifier value from the storage medium through a specific access protocol or interface. The target record to be confirmed here is not limited to a single database table row, but can also be an aggregated data object spanning multiple shards. The read operation can be performed quickly based on a unique serial number as an index key. When performing this step, the system aims to obtain the latest business flow status of the record, providing a data foundation for subsequent logical judgments. The system first constructs a query instruction based on the unique serial number obtained in the previous stage, accesses the storage layer through a database connection pool or cache client, accurately locates the status field corresponding to the record, and loads the status value into memory for parsing.

[0068] To retrieve the current transaction status identifier of a record awaiting confirmation, the system can employ a direct query technique based on a primary key index. Specifically, the system utilizes the database engine's efficient indexing mechanism, using a unique serial number as the retrieval key to directly locate a specific row in the data page, thereby extracting the status field value with extremely low time complexity. This approach is suitable for fast reads in high-concurrency scenarios, minimizing database I / O overhead. Another implementation method is a cache-first read strategy based on a read-write separation architecture. The system first attempts to query the record's status identifier from a distributed cache cluster such as Redis or Memcached. If the cache is hit, it returns the result directly; otherwise, it bypasses the cache and reads from the slave database, then populates the cache with the result. This approach effectively reduces the load on the primary database, especially in read-heavy, write-light business scenarios, significantly improving the overall system throughput.

[0069] S202. Determine whether the current transaction status indicator is in a waiting confirmation state or a timeout state.

[0070] The "Waiting for Confirmation" state refers to an intermediate state where a business request has been sent but a clear response has not yet been received from the receiving end. A preset time window is typically set. The "Send Timeout" state refers to an abnormal pending state automatically marked by the system after a business request has been sent but no response has been received within the preset maximum waiting time. The judgment operation refers to the process by which the system compares the status identifier value read in step S201 with a predefined enumerated constant. The set of states here is not limited to these two states and can be extended to similar states such as "Unknown State" or "Network Abnormal Suspension" depending on the business configuration. Regarding threshold relationships, if timeout judgment is involved, the send timeout threshold is usually greater than the average network round-trip latency plus the business processing time, but less than the system's overall transaction circuit breaker threshold. The core purpose of this step is to filter out transactions on the verge of "uncertainty"—transactions that have neither clearly failed nor clearly succeeded—and are precisely the objects that need to be "rescued" or "confirmed" through the ownership reverse lookup mechanism. The system determines whether the current record meets the conditions for remediation or confirmation through logical comparison operations.

[0071] One specific way to implement this judgment logic is to use a state mask matching technique based on bitwise operations. The system defines all transaction states as specific bits in a binary bitmap; for example, defining the "waiting for confirmation" state as 0001 and the "timeout" state as 0010. The system converts the read state identifier into binary form and performs a bitwise AND operation with a preset target state mask. If the result is non-zero, the judgment condition is met. This method is highly efficient when handling multi-state matching and is easily scalable. Another implementation method is a configuration-based matching technique based on a rule engine. The system introduces a lightweight rule engine such as Drools or Aviator, configuring "waiting for confirmation" and "timeout" as conditional expressions in the rule file. The system passes the read state object to the rule engine, which dynamically parses and executes the logical judgment based on the configuration. This method decouples business logic from the code, allowing operations personnel to dynamically adjust the types of states to be intercepted without restarting the service, enhancing the system's flexibility.

[0072] S203. If so, then based on the receipt of the ownership lookup request, it is determined that the service message has arrived at the receiving end system;

[0073] A reverse lookup request refers to a query instruction initiated by the receiving system to the initiating system during business processing, for some reason (such as timeout retry or inconsistent status), to confirm the ownership or status of a transaction. "Business message has arrived" means that the data packet sent by the initiating system has actually been successfully transmitted to the network boundary of the receiving system and has not been lost at the network transport layer. The judgment operation refers to the system's inference based on causal logic: since the receiving end can initiate a reverse lookup request for a specific serial number, it means that the receiving end must have obtained the serial number and related business information, thus proving that the previous business message was not lost; only the response receipt may have been lost or delayed on the return path. This step is the key turning point in the entire tracing logic, transforming "passively waiting for a response" into "actively utilizing reverse lookup signals," using the receiving end's behavior as a confirmation signal, and solving the classic "communication uncertainty" problem in distributed systems.

[0074] One specific way to implement this judgment logic is to use a request fingerprint-based association verification technology. When the system receives an ownership lookup request, it extracts key features from the request to generate a request fingerprint. The system compares this fingerprint with the metadata of the local record to be confirmed to confirm that they belong to the same transaction context. Once the fingerprint matches successfully and the local record is in the state described in S202, the system marks the transaction link as "physically connected" in memory, which logically determines that the message has been delivered. Another implementation method is based on the context association technology of a distributed link tracing system. The system integrates link tracing components such as SkyWalking or Zipkin. When a lookup request is received, it parses the TraceID in the request header. The system queries the link tracing server to view the call chain topology graph corresponding to the TraceID. If the topology graph shows that the call span from the initiator to the receiver is abnormal but there are indeed network interaction records, the system combines the facts of the lookup request to comprehensively determine that the message has arrived.

[0075] S204. Update the transaction status identifier of the target pending confirmation record to "completed" and generate a status change log;

[0076] The "Completed" status indicates the final state where the transaction process is closed, the business logic has been executed, and the result has been confirmed. The status change log is a detailed historical record of data transitioning from one state to another, including the change time, the state before the change, the state after the change, and the operation source (in this case, a reverse lookup trigger). An update operation refers to a write operation to a specific field in the database. Log generation typically involves appending records to a separate log table or log file system. This step signifies that the system has officially corrected the transaction status, eliminating data inconsistencies. When performing this operation, the system must not only modify the current record but also leave traceable audit trails to ensure that any state transition is verifiable. Update operations here should be idempotent, meaning that executing the same update command multiple times should produce consistent results and prevent data corruption.

[0077] One specific way to implement this step is to use database transactional update technology. The system starts a database transaction, first executing an UPDATE statement to modify the status field of the target record to "completed," and simultaneously executing an INSERT statement within the same transaction to insert a new log record into the status log table. The ACID properties of the database ensure that these two operations either succeed simultaneously or fail simultaneously and roll back, ensuring consistency between the status and the log. Another implementation method is to use an update technology based on event sourcing. The system does not directly modify the status of the current record, but instead generates a "transaction completed" event object, which contains a serial number and the new status. The system appends this event to the event store. When the query service reads the status, it calculates the current status as "completed" by replaying all events under that serial number. This method naturally generates an immutable change log and preserves a complete business history.

[0078] S205, triggering the release of inventory lock or posting verification associated with the target pending confirmation record;

[0079] Triggering the unlocking of inventory locks or posting of accounting records associated with the target record to be confirmed involves: constructing a transaction message containing a unique serial number and unlocking operation instructions, and writing the transaction message to a local message table in the same database instance as the target record to be confirmed; updating the transaction status to "completed" and writing to the local message table as an atomic transaction commit; if the atomic transaction commit is successful, starting an asynchronous task thread to scan the local message table and sending the transaction message to the inventory management subsystem or accounting subsystem to execute the unlocking of inventory locks or posting of accounting records.

[0080] S206. Generate a confirmation receipt containing a unique serial number and an updated completed status indicator, and send it to the receiving system.

[0081] A confirmation receipt is a formal response message sent by the initiating system to the receiving system after completing internal state corrections and resource processing. This message must include a unique serial number to ensure context matching, and a completion status indicator to inform the receiving end of the final result. The sending operation refers to the process of transmitting data packets to the receiving end via network protocols. This step is the final link in the entire tracing and confirmation process, aiming to notify the receiving end, "We have confirmed the transaction was successful; please synchronize your status as well." This forms a complete closed loop: receiving end initiates a reverse lookup -> initiating end confirms and processes -> initiating end informs the receiving end of the processing result. Through this step, the data states of both systems achieve final consistency.

[0082] One specific way to implement this step is to use synchronous RPC call technology. The system constructs a DTO (Data Transfer Object) containing a serial number and status code, and serializes it into JSON or Protobuf format. The system calls the client proxy of an RPC framework such as Dubbo or gRPC to directly initiate a synchronous call to the receipt interface exposed by the receiving end. If the call returns successfully, the process ends; if the call fails, the system can catch the exception and perform a limited number of immediate retries. This approach has strong real-time performance and is suitable for latency-sensitive business scenarios. Another implementation method is to use the publish-subscribe pattern of asynchronous message queues. The system encapsulates the acknowledgment receipt into a message and publishes it to the "Receipt Notification" Topic specified in the message middleware. The receiving system subscribes to this Topic, asynchronously consumes messages, and updates its own status. This approach decouples the initiating and receiving ends; even if the receiving end is temporarily unavailable, messages will not be lost and can be processed after it recovers, enhancing the system's fault tolerance.

[0083] In the above embodiments, by adding a transaction status judgment and update mechanism after reading the unique serial number, the system achieves precise control over the transaction lifecycle. When the system confirms that the current transaction status is in a pending confirmation or timeout state, it determines that the business message has been delivered based on the received ownership lookup request, then updates the transaction status and generates a change log, enabling the system to more accurately track the transaction processing progress and reduce the occurrence of asynchronous transaction statuses. By triggering the release of inventory locks or accounting reconciliation operations, the system promptly releases business resources, improving resource utilization efficiency. Including the updated status identifier when generating the confirmation receipt enhances the timeliness of system status synchronization and improves the reliability of the transaction processing process. This state transition-based processing method improves the system's transaction integrity and enhances the ability to trace abnormal situations.

[0084] The above embodiments achieve control over the transaction processing process through state management. However, in a distributed system environment, it is still necessary to further ensure the reliability of data processing at the receiving end system. Therefore, another embodiment of this application adds state verification and exception handling mechanisms to the transaction state update. The following is a detailed explanation... Figure 3 This application describes yet another method for rapid data tracing based on multi-source log analysis:

[0085] Please see Figure 3 This is another flowchart illustrating a method for rapid data tracing based on multi-source log analysis in this application.

[0086] S301. Generate a status verification instruction corresponding to the target record to be confirmed.

[0087] Generate a status verification instruction corresponding to the target record to be confirmed. The status verification instruction contains an inference correction identifier, which specifically includes: obtaining the business credential information carried by the receiving system when sending the status reverse lookup request; determining whether the business data stored in the target record to be confirmed is consistent with the business credential information; if consistent, generating a status verification instruction; if inconsistent, generating a data conflict alarm and stopping sending information to the receiving system that the target record to be confirmed is in a successful transaction status.

[0088] A status verification command is a command data packet containing specific control information, constructed by the system to verify the consistency of data at the receiving end. This command is not only used for querying but also carries correction suggestions generated based on inference logic. The inference correction identifier is a Boolean value or enumeration code embedded in the command, used to inform the receiving system that the initiating end infers the transaction should be in a successful state based on the current context and requests the receiving end to verify or correct accordingly. Business credential information refers to a set of data that uniquely proves the occurrence of a business transaction and its key attributes, typically including but not limited to core fields such as transaction hash value, amount, timestamp, user ID, and product SKU. A data conflict alarm is an abnormal notification signal triggered when the system detects a mismatch between the core business data held by both parties; this signal usually has high priority. In this step, the system first performs a refinement operation: the system parses the business credential information from the reverse lookup request message initiated by the receiving end, and then calls the local storage engine to read the original business data from the target record to be confirmed. The system performs a rigorous field-level comparison between these two sets of data. If the data is completely consistent, the system will construct a verification instruction containing an inference correction flag, aiming to push the transaction to its final execution; if data inconsistency is found, the system will determine that there is a serious data integrity risk, immediately generate a data conflict alarm, and forcibly suspend the subsequent confirmation process to prevent the spread of erroneous data.

[0089] The first approach is based on hash digest comparison and XML message construction. The system first uses the SHA-256 or MD5 algorithm to concatenate key fields in the received business credential information and calculate a digest value. Simultaneously, the system performs the same digest calculation on the corresponding target record in the local database. The system compares the two digest values; if they match, it uses an XML processing library such as DOM4J or JAXB to construct an XML-formatted status verification instruction conforming to a predefined schema. <correctionflag>Set the tag to True. If the digests do not match, the system calls the monitoring system's API interface to send an alarm event, records the "DATA_MISMATCH" error code in the local log table, and sets a blocking flag in memory to block subsequent sending operations targeting that record.

[0090] The second approach is based on JSON object deep differencing and Protobuf serialization. The system deserializes the received business credential information into a JSON object in memory and uses reflection to traverse the attributes of the local entity class. The system uses a deep differencing algorithm to compare key-value pairs one by one. If all fields match, the system uses Protocol Buffers to serialize the verification command into a binary stream to improve transmission efficiency and writes a correction flag in the extended field of the command header. If a discrepancy is found, the system generates a JSON object containing details of the discrepancy, pushes it to the operation and maintenance alarm message queue, and immediately terminates the current thread's processing logic for that record.

[0091] S302. Send the status verification command to the receiving system and start the verification waiting timer;

[0092] The receiving system refers to a remote server cluster that interacts with the initiating system, responsible for finalizing business data or executing subsequent business logic. The verification wait timer is a time management object instantiated within the system, used to measure the time span from issuing a command to receiving a response; it has countdown or timestamp comparison functions. The startup operation means that the system synchronously activates this timing task the instant the data packet is sent at the network layer. This step is crucial for ensuring the liveness of the distributed system. After the system sends the command generated by S301 through the network transmission protocol, it cannot remain in a blocked waiting state indefinitely, otherwise it will lead to the exhaustion of thread resources. Therefore, the system must set a clear time window. The length of this time window is usually set to the P99 value of the network round-trip time plus the average time required for the receiving end to process the business logic, with a certain buffer margin reserved. Through this step, the system transforms the uncertainty of asynchronous communication into a controllable time range, forcing the system to make a clear decision on the transaction status within a specified time.

[0093] One specific way to implement this step is to use a Netty-based time wheel mechanism. After the system writes the status verification command to the network buffer and refreshes it via the Netty channel, it immediately creates a scheduled task. The system submits this task to the time wheel instance, setting the delay time to a preset timeout threshold. The task's execution logic is to trigger a timeout handling process. If a response is received before the timeout, the system will actively cancel the scheduled task based on the task handle. This method can efficiently manage tens of thousands of timers in high-concurrency scenarios with extremely low resource consumption. Another implementation method is to utilize Redis's key expiration notification mechanism. While sending the command, the system writes a placeholder with a serial number as the key to the Redis cache cluster and sets its time-to-live (TTL) equal to the timeout threshold. The system subscribes to Redis's `__keyevent@0__:expired` channel. When the command is sent, if the receiving end does not respond within the TTL, Redis automatically deletes the key and publishes an expiration event. Upon receiving this event, the system considers the verification wait timer to have timed out. This approach leverages the capabilities of distributed caching, facilitating the sharing of timeout states among cluster nodes, making it suitable for stateless service architectures.

[0094] S303. If a persistent confirmation message is received from the receiving system before the verification waiting timer expires, the target record to be confirmed is marked as the final completed state.

[0095] A persistent acknowledgment message is a response message returned by the receiving system to the initiating system after successfully writing business data to non-volatile storage, explicitly indicating that the data has been securely stored. The final completion status signifies the end of the transaction's lifecycle, indicating that the record has achieved strong consistency between the two systems and cannot be rolled back or modified. The marking operation refers to the atomic action of updating the status field in the database. This step is a closed-loop process of "positive acknowledgment." The system continuously listens to the network port; once a response message from the receiving end is captured, it first verifies whether its sequence number matches the currently waiting record. Then, the system checks whether the response content contains "Persistence_ACK" or a similar success identifier. If the verification is successful and the current timestamp is less than the timer's expiration timestamp, the system determines that the verification is successful. The system then executes a database update operation, transitioning the record's status from "awaiting verification" to "final completion," and may trigger subsequent archiving or cleanup tasks. This process ensures that the initiating end only closes the transaction context if the receiving end explicitly commits to data security.

[0096] One specific way to implement this step is to use asynchronous processing technology based on callback functions. When the system sends a command, it registers a CompletableFuture object. When the network layer receives the persistent acknowledgment message, it finds the corresponding Future object through the message dispatcher and calls its complete() method to pass in the response data. In the thenAccept chained calls, the system executes the JDBC update statement: UPDATE records SET status='FINALIZED'WHERE id=?. This method processes the response non-blockingly, maximizing CPU utilization. Another implementation method is to use an event-driven architecture based on message queues. The receiving system sends the persistent acknowledgment message to a specified Topic in a message queue (such as RocketMQ). The initiating system subscribes to this Topic as a consumer. When the consumer thread pulls the acknowledgment message, it first checks the local cache to see if the transaction has timed out (i.e., checks the timer status). If it has not timed out, the system calls the ORM framework to perform a state update operation, modifying the entity object's state attribute to Complete and persisting it to the database. This approach achieves decoupling between systems and enhances the ability to smooth out peak and valley loads.

[0097] S304. If no persistent confirmation message is received after the verification waiting timer expires, or if an abnormal message is received from the receiving system, the current transaction status of the target record to be confirmed will be rolled back to the abnormal pending status.

[0098] An error message refers to an error message returned by the receiving system when it encounters a logical error, insufficient resources, or data verification failure during the processing of verification instructions. An abnormal pending state is an intermediate fault state, indicating that the transaction is on the verge of uncertainty or failure, requiring manual intervention or triggering an automatic compensation mechanism. A rollback operation refers to changing the recorded state from its current state (e.g., "awaiting verification") back to a state that identifies the problem, so that subsequent processes can identify and handle it. This step constitutes the system's "negative feedback" mechanism. When the timer runs out (timeout event triggered) or the system receives an explicit rejection signal (NACK), the system must consider the current verification attempt to have failed. To prevent data dangling or mistaken success, the system immediately performs a state change, marking the record as abnormal. This is not just a state change, but also a signal that the system enters defensive mode, which may subsequently trigger a retry queue, alarm notifications, or conflicting transactions. Through this step, the system ensures that no transaction that has not been explicitly acknowledged is missed, but is explicitly categorized into the abnormal bucket for subsequent management.

[0099] One specific way to implement this step is to use a timeout scanning and status update technique based on a scheduled task. The system deploys a Quartz task that scans the database at short intervals for records in the "awaiting verification" state whose creation time exceeds a threshold. For each scanned record, the system executes a batch update SQL statement, changing its status field to "EXCEPTION_PENDING" and appending a "Timeout" flag to the remarks field. In the event of an exception message, the system directly executes a single update statement in the catch block of the RPC call to roll back the status.

[0100] Another implementation approach is to use a precise rollback technique based on delayed queues. During the S302 phase, the system places the transaction ID into a RabbitMQ delayed exchange or a JDK DelayQueue. When an element is "expired" from the queue, the consumer checks the current status of that transaction ID in the database. If the status has not yet changed to "finally completed" (indicating no confirmation has been received), the consumer performs a database write operation, forcibly updating the status to "abnormal pending." For explicit exception messages, the system directly triggers a status update through regular message consumption logic.

[0101] In the above embodiments, by sending a status verification command containing an inferred correction flag to the receiving system and starting a verification waiting timer for timeout monitoring, data persistence anomalies in the receiving system can be detected in a timely manner. The system dynamically adjusts the status of the records to be confirmed based on the persistence confirmation message or anomaly message returned by the receiving end, making the record status more accurately reflect the actual business processing results. When the receiving end returns a persistence confirmation message normally, the record is marked as finally completed, improving the reliability of the data status. In case of anomalies, the system rolls back the record status to an abnormal pending state, enhancing the system's anomaly handling capabilities. Through status verification and timeout control mechanisms, the accuracy of system processing results is improved, data inconsistency problems caused by network latency or system anomalies are reduced, and the system's data consistency level is enhanced.

[0102] The system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference needed]. Figure 4 This is a schematic diagram of the physical device structure of a data rapid tracing system based on multi-source log analysis provided in an embodiment of this application.

[0103] It should be noted that, Figure 4 The structure of the system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0104] like Figure 4 As shown, the system includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 402 or a program loaded from storage portion 408 into Random Access Memory (RAM) 403, such as executing the methods described in the above embodiments. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0105] The following components are connected to I / O interface 405: input section 406 including a camera, infrared sensor, etc.; output section 407 including a liquid crystal display (LCD) and speakers, etc.; storage section 408 including a hard disk, etc.; and communication section 409 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. Communication section 409 performs communication processing via a network such as the Internet. Drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0106] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the various functions defined in the present invention.

[0107] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. The transmitted data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the system described in the above embodiments; or it may exist independently and not assembled into the system. The storage medium carries one or more computer programs that, when executed by a processor of a system, cause the system to implement the methods provided in the above embodiments.

[0110] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0111] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is detected" can be interpreted as "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0112] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.< / correctionflag> < / signature> < / serialno> < / time>

Claims

1. A method for fast data provenance based on multi-source log analysis, characterized in that, The method comprises the following steps: in response to a service initiation instruction, generating an original transaction document containing a unique serial number and service attribute parameters; establishing an association between the unique serial number and the service attribute parameters, generating a to-be-confirmed record and a generation timestamp; sending a service message containing the service attribute parameters to a receiving end system; receiving a homestead right reverse query request from the receiving end system, and determining the current system time at which the homestead right reverse query request is received, the homestead right reverse query request containing to-be-queried attribute parameters; based on the current system time and a preset time delay threshold, calculating an effective time interval; screening out a target to-be-confirmed record in which the service attribute parameters are consistent with the to-be-queried attribute parameters and the generation timestamp is located within the effective time interval, specifically comprising: extracting a key business field in the to-be-queried attribute parameters, and calculating the key business field using a preset hash algorithm to obtain a target hash index key; using the target hash index key to locate a target hash bucket in a hash bucket array of a local memory database, determining the target hash bucket corresponding to the key business field, and the target hash bucket storing a plurality of candidate record linked lists arranged in descending order of generation timestamp; traversing the candidate record linked lists in the target hash bucket to compare the overlapping relationship between the generation timestamp recorded in the candidate record linked lists and the effective time interval; when the generation timestamp recorded in any candidate record linked list is located within the effective time interval and the corresponding service attribute parameters are fully matched with the to-be-queried attribute parameters, determining the record corresponding to the generation timestamp as the target to-be-confirmed record and stopping the traversal; reading the unique serial number from the target to-be-confirmed record; generating a confirmation receipt containing the unique serial number and sending it to the receiving end system; reading the current transaction state identifier of the target to-be-confirmed record; determining whether the current transaction state identifier is in a waiting confirmation state or a sending timeout state; if yes, determining that the service message has arrived at the receiving end system based on the receipt of the homestead right reverse query request; updating the transaction state identifier of the target to-be-confirmed record to completed and generating a state change log; triggering the release of inventory locking or accounting cancellation associated with the target to-be-confirmed record; generating a confirmation receipt containing the unique serial number and the updated completed state identifier and sending it to the receiving end system.

2. The method of claim 1, wherein, The screening out of the target to-be-confirmed record in which the service attribute parameters are consistent with the to-be-queried attribute parameters and the generation timestamp is located within the effective time interval specifically comprises: inputting the to-be-queried attribute parameters into a preset Bloom filter for mapping detection, the preset Bloom filter being a bit array set constructed based on the service attribute parameters of the to-be-confirmed records generated in history; if the output existence determination is no, returning a query failure response to the receiving end system; if the output existence determination is yes, constructing a time slicing query cursor according to the start time point and the end time point of the effective time interval; acquire, by using the time-sliced query cursor, all candidate data sets located in the preset time-series database and within the valid time interval; match the records in the candidate data sets with the full quantity of the attribute parameter to be queried as target records to be confirmed.

3. The method of claim 1, wherein, The triggering of the de-inventory locking or accounting cancellation associated with the target records to be confirmed specifically includes: constructing a transaction message containing the unique serial number and an unlocking operation instruction, and writing the transaction message into a local message table in the same database instance as the target records to be confirmed; updating the transaction status identifier to the completed operation and writing the local message table as an atomic transaction submission; In the case where the atomic transaction submission is successful, an asynchronous task thread is started to scan the local message table, and the transaction message is sent to the inventory management subsystem or the accounting subsystem to perform the de-inventory locking or accounting cancellation operation.

4. The method of claim 1, wherein, After the transaction status identifier of the target records to be confirmed is updated to completed and the state change log is generated, the method further includes: generating a state verification instruction corresponding to the target records to be confirmed, the state verification instruction containing an inference correction identifier; sending the state verification instruction to the receiving end system and starting a verification waiting timer; if the persistent confirmation message returned by the receiving end system is received before the verification waiting timer expires, the target records to be confirmed are marked as a final completion state; if the persistent confirmation message is not received after the verification waiting timer expires, or an abnormal message returned by the receiving end system is received, the current transaction status identifier of the target records to be confirmed is rolled back to an abnormal to-be-processed state.

5. The method of claim 4, wherein, The generation of the state verification instruction corresponding to the target records to be confirmed specifically includes: obtaining the business credential information carried by the receiving end system when sending a state backtracking request; determining whether the business data stored in the target records to be confirmed is consistent with the business credential information; if consistent, the state verification instruction is generated; if inconsistent, a data conflict alarm is generated, and the sending of the information that the target records to be confirmed are in a transaction success state to the receiving end system is stopped.

6. A system for fast data provenance based on multi-source log analysis, characterized in that, The system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors invoke the computer instructions to make the system execute the method of any one of claims 1-5.

7. A computer-readable storage medium comprising instructions, characterized in that, When the instructions run on the system, the system executes the method of any one of claims 1-5.

8. A computer program product, characterised in that, When the computer program product runs on the system, the system executes the method of any one of claims 1-5.

Images