Invoicing data asynchronous concurrent processing method and system based on message queue

Abstract

The invention provides an invoicing data asynchronous concurrent processing method and system based on a message queue, and the method comprises the steps: receiving invoicing application data of a business system through a tax platform, carrying out the basic format verification of the invoicing application data, asynchronously pulling an invoicing application message from a message queue through a consumption cluster composed of one or more message consumers, and transmitting the invoicing application message to a server; judging whether all the preset service rules are met or not; if all the preset business rules pass the verification, pushing the verified invoicing application data to a tax center, and initiating an automatic invoicing request; generating a final result of invoicing processing; and persistently storing a final result of invoicing processing in a database or a distributed cache, and feeding back the final result to a service system through an asynchronous notification mechanism. Complete decoupling of requests and processing is realized, the throughput and concurrency capability of the system are effectively improved, the stability and fault tolerance of the system are enhanced, the maintainability and flexibility are improved, the impact on a downstream system is reduced, and elastic expansion and automatic operation and maintenance are supported.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer software engineering, and in particular to a message queue-based invoice data asynchronous concurrent processing method and system. BACKGROUND

[0002] In the modern enterprise financial digital system, as a core link of compliance and tax declaration, invoice management generally uses a "tax intermediate platform" as a unified access platform to centrally process invoice application from multiple business systems such as ERP, e-commerce platform, CRM, and POS system, and after completing rule verification, push the compliant data to the tax center (such as electronic invoice service platform, tax control server) for automatic invoicing.

[0003] In the prior art, the invoicing request usually adopts a synchronous blocking processing model: business system initiates an HTTP request → tax intermediate platform receives → synchronously performs time-consuming operations such as quota verification, taxpayer information verification, commodity code compliance check, and repeated invoicing detection → calls the tax center interface to invoice → returns the final result. This mode has the following technical defects:

[0004] High response delay: due to complex verification logic and dependence on external system calls (such as tax bureau interface query), the processing time of a single request can reach several seconds, leading to long waiting time for the front end in high concurrency scenarios, and poor user experience;

[0005] Severe system resource competition: a large number of requests simultaneously occupy the Web container thread pool and database connection pool, easily causing thread starvation and connection exhaustion, resulting in service unavailability;

[0006] Limited horizontal expansion capability: the synchronous processing model is difficult to achieve linear performance improvement by increasing server nodes;

[0007] Weak fault tolerance: if the push fails or the verification is abnormal, there is a lack of persistent message and reliable retry mechanism, which easily leads to request loss or state inconsistency;

[0008] Large impact on downstream systems: sudden traffic directly impacts the tax center interface, which may trigger flow limiting or service degradation. SUMMARY

[0009] The technical problem to be solved by the present application is to provide a message queue-based invoice data asynchronous concurrent processing method and system to solve at least one of the above problems existing in the prior art.

[0010] To achieve the above purpose, one or more embodiments of the present application provide a message queue-based invoice data asynchronous concurrent processing method and system, the method comprising:

[0011] The tax authority in Taiwan receives the invoicing application data of a business system, performs basic format verification on the invoicing application data, encapsulates the invoicing application data into a structured message after the verification, and publishes the structured message to a preset message queue, and then feeds back a lightweight confirmation response to the business system that initiates the request;

[0012] A consumer cluster composed of one or more message consumers asynchronously pulls the invoicing application message from the message queue, each consumer calls a rule engine to perform preset business rule verification on the invoicing application data, and judges whether all preset business rules are met;

[0013] If all preset business rule verifications are passed, the verified invoicing application data is pushed to the tax center to initiate an automatic invoicing request; if any preset business rule verification fails or fails to push to the tax center, the failure reason is recorded, and a preset exception handling process is started; a final result of invoicing processing is generated;

[0014] The final result of invoicing processing is stored in a database or a distributed cache, and the final result is fed back to the business system through an asynchronous notification mechanism.

[0015] Based on the above technical solutions of the present application, the following improvements can be made:

[0016] Optionally, the basic format verification includes field integrity, data type compliance and mandatory item check.

[0017] Optionally, the confirmation response is an HTTP 202 Accepted status code or an equivalent asynchronous acceptance identifier, indicating that the request has been received and entered the processing queue.

[0018] Optionally, the business rule verification includes invoice available quota verification, taxpayer identification number validity verification, commodity tax classification code matching, historical invoicing record comparison and duplicate invoicing detection.

[0019] Optionally, the final result includes invoicing status, invoice code, invoice number, processing timestamp and failure reason.

[0020] Optionally, the asynchronous notification mechanism includes callback interface call or message broadcast.

[0021] Optionally, the message queue is a distributed message middleware supporting high throughput and high availability, the distributed message middleware is Apache Kafka, RabbitMQ, RocketMQ or Apache Pulsar, and the message is set with a persistence attribute when published to prevent message loss caused by system failure.

[0022] Optionally, the exception handling process includes performing an exponential backoff retry mechanism on recoverable errors, with retry intervals exponentially increasing; for messages that fail after a preset number of retries, transferring them to a dead letter queue and generating an alarm log for manual intervention by an operator.

[0023] According to another aspect of the present application, a message queue-based billing data asynchronous concurrent processing system is provided, comprising:

[0024] A request access module is configured to receive a billing application from a business system and perform basic data format and field integrity checking.

[0025] A message publishing module is configured to encapsulate the checked billing data as a message and publish it to a message queue.

[0026] A fast response module is configured to return an HTTP 202 Accepted or equivalent status code to the business system, indicating that the request has been received.

[0027] A message queue is used as an asynchronous communication and buffering middleware, achieving spatiotemporal decoupling of producers and consumers, supporting message persistence and high-availability deployment.

[0028] A message consumption cluster contains multiple parallel running consumer instances, each of which integrates a message pulling unit, a rule checking unit, a condition pushing unit, and an exception handling unit; a rule engine is integrated in the rule checking unit, dynamically loading checking rule scripts through an external configuration center, supporting rule version management and rollback; the message consumption cluster is deployed on a container orchestration platform such as Kubernetes, supporting automatic scaling based on Prometheus monitoring indicators, achieving elastic scaling.

[0029] A result storage module is configured to persist billing processing results, supporting subsequent queries, reconciliation, and auditing.

[0030] A monitoring and alarm module is configured to collect message queue backlog, consumption rate, failure rate, and system resource usage in real time, and trigger an alarm when a preset threshold is exceeded.

[0031] Based on the above technical solutions of the present application, the following improvements can be made:

[0032] Optionally, the exception handling unit is configured to perform an exponential backoff retry on network timeout and service unreachable errors, and route the message to a dead letter queue after the retry fails, the dead letter queue and the main message queue being deployed in the same middleware cluster.

[0033] The beneficial effects of this invention are as follows: This invention provides an asynchronous concurrent processing method and system for invoice data based on message queues, achieving complete decoupling of requests and processing. By separating the reception and processing of invoice requests through message queues, the front-end response time is reduced from "processing time" to "queueing time," significantly improving system response performance; effectively improving system throughput and concurrency capabilities, the consumer cluster can be horizontally scaled, and processing capacity increases linearly with the number of instances, meeting the needs of high-concurrency businesses; enhancing system stability and fault tolerance, message persistence prevents data loss; dead-letter queues ensure traceability of abnormal messages; traffic shaping avoids system overload; improving maintainability and flexibility, the rule engine supports dynamic updates to adapt to constantly changing tax compliance requirements; reducing the impact on downstream systems: asynchronous push mode smooths traffic and protects the stability of the tax center interface; supporting elastic scaling and automated operation and maintenance, combined with container platforms such as Kubernetes, consumer instances can be dynamically scaled up or down according to queue backlog. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the system architecture according to an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0036] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in one or more embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0037] like Figure 1As shown in the figure, the overall system architecture of this invention includes: multiple business systems (such as e-commerce platforms, ERP, and CRM) submitting invoicing requests to the tax middle platform via API; the tax middle platform includes an API gateway, a data verification module, a message publishing module, and a rapid response module; a message queue (MQ) serves as a core decoupling component, connecting producers and consumers; a message consumption cluster consists of multiple Consumer instances, performing rule verification and pushing to the tax center; the tax center receives invoicing requests and returns results; a result database stores the final processing results; a dead-letter queue (DLQ) stores messages of failed retry attempts; and a monitoring system collects key indicators and issues alerts.

[0038] Specifically, one or more embodiments of this application describe an asynchronous concurrent processing method and system for invoicing data based on a message queue.

[0039] S1: Request reception and asynchronous queuing. The tax middle platform receives invoice application data from various business systems; performs basic format verification on the invoice application data; after the verification is passed, it encapsulates it into a structured message and publishes it to a preset message queue (MQ); then returns a lightweight confirmation response to the business system that initiated the request, indicating that "the request has been received and entered the processing queue".

[0040] S2: Asynchronous consumption and rule verification. A consumer cluster consisting of one or more message consumers asynchronously pulls invoice application messages from the message queue; each consumer instance independently performs the following operations:

[0041] The rule engine module is invoked to perform multi-dimensional business rule verification on the invoice data. The rules include, but are not limited to: invoice availability verification, taxpayer identification number validity verification, commodity tax classification code matching, historical invoice record comparison, and anti-duplicate invoice detection.

[0042] Determine whether all preset validation rules are met.

[0043] S3: Conditional push and exception handling. If all verification rules pass, the verified invoice data will be pushed to the tax center to initiate an automatic invoice request.

[0044] If any verification fails, or if the request to the tax center fails, the reason for the failure will be recorded, and the preset exception handling process will be triggered.

[0045] S4: Result persistence and feedback. The final result of the invoicing process (including invoicing success information, failure reason, processing timestamp, invoice code / number) is persisted in the database or distributed cache; the processing result is fed back to the relevant business system through asynchronous notification mechanisms (such as callback interface, message broadcast).

[0046] In addition, the message queue is a distributed message middleware that supports high throughput and high availability, selected from Apache Kafka, RabbitMQ, RocketMQ or Apache Pulsar.

[0047] The message consumers are deployed in a clustered, multi-instance manner, with each instance consuming messages in the message queue in parallel, thereby achieving horizontal scaling and load balancing of the verification and push logic.

[0048] The exception handling process includes:

[0049] For recoverable errors, implement an exponential backoff retry mechanism (e.g., retry up to 3 times with intervals of 10s, 30s, and 60s).

[0050] For messages that fail after a preset number of retries, they are transferred to the Dead Letter Queue (DLQ) and an alarm log is generated for manual intervention.

[0051] The method also includes traffic shaping and system protection mechanisms: utilizing the buffering capacity of message queues to smoothly handle sudden surges in invoicing requests and prevent the backend verification service and tax center interface from crashing due to instantaneous high load.

[0052] The rule engine supports dynamic configuration and hot reloading, allowing invoice verification rules to be updated without restarting the service, thus improving the flexibility of system operation and maintenance.

[0053] The following example, using a real-world deployment case of an electronic invoice processing system for a large retail enterprise, further illustrates the invention:

[0054] Deployment environment: MQ middleware, RocketMQ 4.9 cluster (3 masters and 3 slaves); tax middleware, Spring Boot + Docker + Kubernetes; consumer instances, initially deployed with 5 Pods, supporting HPA auto-scaling; rule engine, Drools + Nacos configuration center (supporting hot updates); database, MySQL 8.0 master-slave cluster; monitoring, Prometheus + Grafana + Alertmanager.

[0055] Example processing flow: E-commerce platform submits invoice request → Tax middle platform verifies and sends to MQ → Returns 202 response (<100ms); Consumer pulls message from MQ → Calls Drools to perform 4 verifications → All pass; Calls electronic invoice platform API to issue invoice → Successfully returns invoice code / number; Writes to MySQL result table → Notifies e-commerce system via callback.

[0056] Exception handling: If "insufficient quota", record the reason for failure, do not retry, and write directly to the database; if "network timeout", perform 3 index fallback attempts, and enter the DLQ after failure; the operation and maintenance personnel can view the DLQ message through the management backend and handle it manually.

[0057] Technical results: Average front-end response time: reduced from 3.5s to 90ms; Maximum concurrent processing capacity: increased from 80 TPS to 400 TPS; System availability: >99.95%; Invoicing failure rate: <0.1%.

[0058] In another embodiment, a message queue-based asynchronous concurrent processing system for invoicing data is provided, comprising:

[0059] The access request module is used to receive invoice applications from business systems and perform basic data format and field integrity checks.

[0060] The message publishing module encapsulates the verified invoicing data into a message and publishes it to the message queue;

[0061] The quick response module returns an HTTP 202 Accepted or equivalent status code to the business system, indicating that the request has been received;

[0062] Message queues, as middleware for asynchronous communication and buffering, enable the decoupling of producers and consumers in time and space, and support message persistence and high-availability deployment.

[0063] The message consumption cluster comprises multiple consumer instances running in parallel. Each instance integrates a message retrieval unit, a rule verification unit, a conditional push unit, and an exception handling unit. The rule engine is integrated into the rule verification unit, dynamically loading verification rule scripts through an external configuration center, and supporting rule version management and rollback. The message consumption cluster is deployed on container orchestration platforms such as Kubernetes, supporting automatic scaling based on Prometheus monitoring metrics to achieve elastic scaling.

[0064] The results storage module is used to persist the invoicing processing results and support subsequent queries, reconciliation and auditing.

[0065] The monitoring and alarm module collects real-time data on message queue backlog, consumption rate, failure rate, and system resource utilization, and triggers alarms when these data exceed preset thresholds.

[0066] The exception handling unit is configured to perform exponential backoff retries for network timeout and service unreachability errors, and route the message to the dead letter queue after the retry fails. The dead letter queue and the main message queue are deployed in the same middleware cluster.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0072] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for asynchronous concurrent processing of invoice data based on message queues, characterized in that, include: The tax middle platform receives invoice application data from the business system, performs basic format verification on the invoice application data, and after the verification is passed, encapsulates the invoice application data into a structured message and publishes it to a preset message queue, and then sends a lightweight confirmation response back to the business system that initiated the request. A consumer cluster consisting of one or more message consumers asynchronously pulls invoice application messages from the message queue. Each consumer calls the rule engine to perform preset business rule verification on the invoice application data and determines whether all preset business rules are met. If all preset business rules pass the verification, the verified invoice application data will be pushed to the tax center to initiate an automatic invoice request; if any preset business rule fails the verification or the push to the tax center fails, the reason for the failure will be recorded and the preset exception handling process will be triggered. Generate the final result of the invoicing process; The final result of the invoicing process is persisted in a database or distributed cache, and the final result is fed back to the business system through an asynchronous notification mechanism.

2. The asynchronous concurrent processing method for invoicing data based on message queues according to claim 1, characterized in that, Basic format validation includes checking field integrity, data type compliance, and required fields.

3. The asynchronous concurrent processing method for invoicing data based on message queues according to claim 1, characterized in that, A response that confirms the HTTP 202 Accepted status code or an equivalent asynchronous accept flag indicates that the request has been received and entered the processing queue.

4. The asynchronous concurrent processing method for invoicing data based on message queues according to claim 1, characterized in that, Business rule verification includes invoice availability verification, taxpayer identification number validity verification, commodity tax classification code matching, historical invoicing record comparison, and anti-duplicate invoicing detection.

5. The asynchronous concurrent processing method for invoicing data based on message queues according to claim 1, characterized in that, The final result includes the invoice status, invoice code, invoice number, processing timestamp, and reason for failure.

6. The asynchronous concurrent processing method for invoicing data based on message queues according to claim 1, characterized in that, Asynchronous notification mechanisms include callback API calls or message broadcasts.

7. The asynchronous concurrent processing method for invoicing data based on message queues according to claim 1, characterized in that, The message queue is a distributed message middleware that supports high throughput and high availability. The distributed message middleware is Apache Kafka, RabbitMQ, RocketMQ or Apache Pulsar, and the messages are set with persistence attributes when published to prevent message loss due to system failure.

8. The asynchronous concurrent processing method for invoicing data based on message queues according to claim 1, characterized in that, The exception handling process includes an exponential backoff retry mechanism for recoverable errors, with the retry interval increasing exponentially; for messages that still fail after a preset number of retries, they are transferred to a dead-letter queue and an alarm log is generated for manual intervention by operations and maintenance personnel.

9. A message queue-based asynchronous concurrent processing system for invoicing data, characterized in that, include: The access request module is used to receive invoice applications from business systems and perform basic data format and field integrity checks. The message publishing module encapsulates the verified invoicing data into a message and publishes it to the message queue; The quick response module returns an HTTP 202 Accepted or equivalent status code to the business system, indicating that the request has been received; Message queues, as middleware for asynchronous communication and buffering, enable the decoupling of producers and consumers in time and space, and support message persistence and high-availability deployment. The message consumption cluster comprises multiple consumer instances running in parallel. Each instance integrates a message retrieval unit, a rule verification unit, a conditional push unit, and an exception handling unit. The rule engine is integrated into the rule verification unit, dynamically loading verification rule scripts through an external configuration center, and supporting rule version management and rollback. The message consumption cluster is deployed on container orchestration platforms such as Kubernetes, supporting automatic scaling based on Prometheus monitoring metrics to achieve elastic scaling. The results storage module is used to persist the invoicing processing results and support subsequent queries, reconciliation and auditing. The monitoring and alarm module collects real-time data on message queue backlog, consumption rate, failure rate, and system resource utilization, and triggers alarms when these data exceed preset thresholds.

10. The asynchronous concurrent processing system for invoicing data based on a message queue according to claim 9, characterized in that, The exception handling unit is configured to perform exponential backoff retries for network timeout and service unreachability errors, and to route the message to the dead letter queue after the retry fails. The dead letter queue and the main message queue are deployed in the same middleware cluster.

Images