Railway material purchasing asynchronous task processing method, device and system

Abstract

The invention relates to a railway material purchasing asynchronous task processing method, device and system, and the method comprises the steps: constructing a six-layer message channel architecture of a purchasing business process, and executing an instruction of deploying a message agent node in an active-active mode for each channel based on the core message middleware of a Kafka cluster; the front-end gateway is instructed to package the purchase event into a binary message body with a transaction ID and a timestamp, and the binary message body is written into a persistent queue after priority labeling is carried out according to a task type label; processing the concurrent tasks by adopting a multi-stage feedback queue scheduling algorithm, and dynamically allocating computing resources based on a thread pool; a maximum retry threshold value is set according to an exponential backoff retry strategy, so that the failed task is automatically transferred into a dead message queue; and dynamically calculating an optimal message routing path based on the supplier portrait and the material classification model, and extracting the order to an exclusive processing queue of the optimal supplier according to Topic dynamic switching of the message queue. According to the invention, the coupling degree is reduced, the system performance is improved, and the fault tolerance is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of railway material procurement, and in particular to a method, device and system for processing asynchronous tasks in railway material procurement. Background Art

[0002] Railway material procurement is a crucial component of national railway operations, involving a large number of complex business processes, including material demand planning, supplier management, procurement execution, and logistics distribution. With the rapid expansion of the railway network and the popularization of digital technology, the traditional material procurement model has gradually exposed the following problems:

[0003] 1. System performance bottleneck: In high-concurrency scenarios, traditional synchronous processing methods easily lead to task backlogs, resulting in a decrease in system response speed. For example, when multiple users submit purchase requests simultaneously, the system needs to process each task sequentially, resulting in long wait times for subsequent tasks and affecting the user experience.

[0004] 2. High system coupling: Direct calls between business modules result in poor system scalability and maintainability. For example, the order generation module directly calls the logistics tracking module. If the logistics tracking module changes, the order generation module also needs to be adjusted accordingly, increasing system maintenance costs.

[0005] 3. Insufficient fault tolerance: A failure in a subsystem could disrupt the entire procurement process, and there is a lack of effective fault recovery mechanisms. For example, if the supplier matching module fails, the entire procurement process will be unable to continue, affecting the timely supply of materials. Summary of the Invention

[0006] Based on this, it is necessary to provide a railway material procurement asynchronous task processing method, device and system to address the problems of system performance bottleneck, high system coupling and insufficient fault tolerance in the traditional material procurement model.

[0007] The present invention provides a method for processing asynchronous tasks in railway material procurement, the method comprising:

[0008] Build a six-layer message channel architecture for the procurement business process, including demand submission, approval process, supplier matching, order generation, logistics tracking, and settlement and payment. Based on the core message middleware of the Kafka cluster, execute instructions to deploy active-active message broker nodes for each channel.

[0009] Based on the atomic microservice components and event-driven service architecture encapsulated by the Spring Cloud Stream framework, the front-end gateway is instructed to encapsulate procurement events into binary message bodies with transaction IDs and timestamps, annotate them with priorities based on task type tags, and write them to a persistent queue.

[0010] A multi-level feedback queue scheduling algorithm is used to handle concurrent tasks, and computing resources are dynamically allocated based on the thread pool;

[0011] Set the maximum retry threshold based on the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue;

[0012] Based on the supplier profile and material classification model, the optimal message routing path is dynamically calculated, and the order is extracted to the exclusive processing queue of the optimal supplier according to the dynamic switching of the message queue topic.

[0013] In one embodiment, the multi-level feedback queue scheduling algorithm is used to process concurrent tasks and dynamically allocate computing resources based on a thread pool, including:

[0014] Get request information for executing a new task;

[0015] Assign new tasks to a level in a multi-level queue based on estimated execution time, task type, and historical execution data.

[0016] Arrange tasks from high to low priority to obtain immediate execution rights for high-priority queue tasks;

[0017] Obtain monitoring information on task execution status and perform queue downgrade operations on unfinished tasks;

[0018] Periodically reset all unfinished tasks to the highest priority queue.

[0019] In one embodiment, arranging the tasks from high to low priority to obtain the immediate execution right of the high-priority queue task includes:

[0020] Arrange tasks from high to low priority. The high-priority queue is allocated 10ms time slices to process real-time short tasks, and the low-priority queue is allocated 100ms time slices to process batch long tasks.

[0021] In response to a new task being generated by a high-priority queue, the execution of the current low-priority task is immediately interrupted, and the task is suspended after retaining its execution context, and computing resources are released to prioritize processing of high-priority queue tasks.

[0022] In one embodiment, obtaining monitoring information of task execution status and performing queue demotion operations on unfinished tasks includes:

[0023] In response to the task completing within the allocated time slice, the thread directly processes the next task;

[0024] In response to a task not being completed within the allocated time slice, the task is extracted to the next level queue and assigned a time slice length of 1.5 to 2 times the original time slice.

[0025] In one embodiment, setting a maximum retry threshold according to an exponential backoff retry strategy so that failed tasks are automatically transferred to a dead letter queue includes:

[0026] In response to a task failure, wait for a gradually increasing delay before retrying, with the retry wait time increasing exponentially;

[0027] In response to a task failing after exceeding the retry limit, the task is extracted to a dead letter queue. The retry limit parameters are the maximum number of retries and the maximum delay time.

[0028] In one embodiment, the method dynamically calculates the optimal message routing path based on the supplier profile and material classification model, and dynamically switches the topic of the message queue to extract the order to the exclusive processing queue of the optimal supplier, including:

[0029] Obtain supplier data including credit ratings, historical fulfillment rates, logistics timeliness indicators, and risk penalty records, and build a multi-dimensional supplier profile library;

[0030] Establish a material classification knowledge graph based on hierarchical feature vectors of urgency, material type, and quality standards;

[0031] Calculate the matching scores between supplier capability values ​​and order requirements in real time to develop a dynamic routing decision matrix;

[0032] Establish a temporary routing channel between the optimal supplier's exclusive processing queue and the current order message to implement the Topic dynamic binding mechanism.

[0033] The present invention also provides a railway material procurement asynchronous task processing device, comprising:

[0034] The construction module is used to build a six-layer message channel architecture for the procurement business process, including demand submission, approval process, supplier matching, order generation, logistics tracking, and settlement and payment. Based on the core message middleware of the Kafka cluster, it executes instructions to deploy active-active message broker nodes for each channel.

[0035] The instruction module is used to instruct the front-end gateway to encapsulate procurement events into binary messages with transaction IDs and timestamps, annotate them with priorities based on task type tags, and write them to a persistent queue based on the atomic microservice components and event-driven service architecture encapsulated using the Spring Cloud Stream framework.

[0036] The processing module is used to process concurrent tasks using a multi-level feedback queue scheduling algorithm and dynamically allocate computing resources based on a thread pool;

[0037] The threshold module is used to set the maximum retry threshold based on the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue;

[0038] The calculation module is used to dynamically calculate the optimal message routing path based on the supplier profile and material classification model, and dynamically switch the topic of the message queue to extract the order to the exclusive processing queue of the optimal supplier.

[0039] The present invention also provides a railway material procurement asynchronous task processing system, comprising:

[0040] The message queue layer includes a Kafka cluster with active-active deployment and mirrored queues, a multi-level feedback queue scheduler, a Redis distributed lock service, an intelligent routing decision engine, and a backoff and retry controller. The multi-level feedback queue scheduler has built-in multi-level priority queues and associated thread pool resource groups, and is used to perform time-slice round-robin scheduling with a preemptive mechanism. The Redis distributed lock service implements cross-node resource conflict arbitration through the SETNX command. The intelligent routing decision engine is used to dynamically select the optimal message channel based on a scoring model built based on the supplier's historical performance data. The backoff and retry controller is used to calculate the retry interval using an exponentially increasing algorithm. Failed tasks are automatically dumped to a dead letter queue after multiple retries.

[0041] The microservice layer is connected to the message queue layer and includes demand processing microservices, approval microservices, supplier microservices, and other domain microservices. It is also used to connect to the message queue layer to publish domain events.

[0042] The gateway layer is connected to the microservice layer and integrates the Spring Cloud Gateway service gateway. It is used to perform protocol conversion and message normalization on inbound requests, implement dynamic routing distribution based on request path characteristics, and achieve load balancing of microservice instances through a weighted polling algorithm;

[0043] The external access layer is connected to the gateway layer and is used to receive procurement business requests initiated by multi-source heterogeneous terminals, and includes front-end gateways, mobile terminals and third-party systems.

[0044] The present invention also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any of the above-described methods for processing asynchronous tasks in railway material procurement.

[0045] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described methods for processing asynchronous tasks in railway material procurement.

[0046] The above-mentioned railway material procurement asynchronous task processing method, device and system decouple the subsystems through the message queue based on the six-layer message channel architecture, improve scalability and maintainability, reduce coupling, and process concurrent tasks by adopting a multi-level feedback queue scheduling algorithm and dynamically allocate computing resources based on the thread pool, so as to facilitate dynamic adjustment of task priority according to task type and urgency, meet the diversified needs of railway material procurement, improve concurrent processing capabilities, reduce task accumulation, and improve response speed. When messages are lost or failures occur, exponential backoff retry strategies are used to compensate for them in time, so as to ensure normal operation in the event of failures and enhance fault tolerance. By dynamically switching according to the Topic of the message queue, orders are extracted to the exclusive processing queue of the optimal supplier, reducing task processing delays and improving real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 A schematic flow chart of a method for processing asynchronous tasks for railway material procurement in one embodiment;

[0049] Figure 2 A schematic diagram of a process for processing concurrent tasks using a multi-level feedback queue scheduling algorithm in one embodiment;

[0050] Figure 3 A schematic diagram of a process for arranging tasks from high to low priority in one embodiment;

[0051] Figure 4 A schematic diagram of a process for obtaining monitoring information of task execution status in one embodiment;

[0052] Figure 5 Schematic diagram of the exponential backoff retry strategy process in one embodiment;

[0053] Figure 6 A schematic diagram of the process of dynamically calculating the optimal message routing path in one embodiment;

[0054] Figure 7 This is a schematic diagram of the structure of an asynchronous task processing device for railway material procurement in one embodiment;

[0055] Figure 8 This is a schematic diagram of the structure of an asynchronous task processing system for railway material procurement in one embodiment;

[0056] Figure 9FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment.

[0057] Reference numerals:

[0058] 710, construction module; 720, instruction module; 730, processing module; 740, threshold module; 750, computing module; 810, message queue layer; 811, Kafka cluster; 812, multi-level feedback queue scheduler; 813, Redis distributed lock service; 814, intelligent routing decision engine; 815, backoff retry controller; 820, microservice layer; 830, gateway layer; 840, external access layer. DETAILED DESCRIPTION

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0060] The following combination Figures 1-9 The present invention describes the railway material procurement asynchronous task processing method, device and system.

[0061] like Figure 1 As shown, in one embodiment, a method for processing asynchronous tasks for railway material procurement includes the following steps:

[0062] Step S100: Build a six-layer message channel architecture for the procurement business process, including demand submission, approval flow, supplier matching, order generation, logistics tracking, and settlement and payment services. Based on the core message middleware of the Kafka cluster, execute the instructions to deploy a message proxy node in active-active mode for each channel.

[0063] By adopting Kafka cluster as the core message middleware and realizing cross-platform communication based on AMQP protocol, we designed dedicated message channels for six major business domains: procurement demand submission, approval process, supplier matching, order generation, logistics tracking, and settlement and payment. We also deployed active-active message broker nodes for each channel to ensure high availability.

[0064] Step S200, based on the atomic microservice components and event-driven service architecture encapsulated by the Spring Cloud Stream framework, instructs the front-end gateway to encapsulate the procurement event into a binary message body with a transaction ID and a timestamp, and writes it into a persistent queue after prioritizing it according to the task type tag.

[0065] By sending task messages to the message queue, the task processing module obtains the task messages from the message queue and processes them, avoiding direct calls between subsystems, reducing system coupling, and improving the scalability and maintainability of the system.

[0066] Step S300: Use a multi-level feedback queue scheduling algorithm to process concurrent tasks and dynamically allocate computing resources based on a thread pool.

[0067] Dynamically adjust task priorities based on task type and urgency to ensure that high-priority tasks are processed first. For example, urgent material procurement tasks can be set to high priority and processed first, thus meeting the diverse needs of railway material procurement and improving task processing efficiency.

[0068] Step S400: Setting a maximum retry threshold according to an exponential backoff retry strategy so that failed tasks are automatically transferred to a dead letter queue.

[0069] Task retry and compensation mechanisms ensure normal operation in the event of failures, enhancing fault tolerance.

[0070] Step S500, based on the supplier profile and material classification model, dynamically calculate the optimal message routing path, and dynamically switch the topic of the message queue to extract the order to the exclusive processing queue of the optimal supplier.

[0071] By optimizing the message queue distribution mechanism, task processing delays can be reduced and real-time performance can be improved.

[0072] This railway material procurement asynchronous task processing method decouples each subsystem through a message queue based on a six-layer message channel architecture, improves scalability and maintainability, and reduces coupling. It processes concurrent tasks by adopting a multi-level feedback queue scheduling algorithm and dynamically allocates computing resources based on a thread pool, making it convenient to dynamically adjust task priorities according to task type and urgency, meeting the diverse needs of railway material procurement, improving concurrent processing capabilities, reducing task accumulation, and increasing response speed. It uses an exponential backoff retry strategy to perform timely compensation processing when messages are lost or failures occur, ensuring normal operation in the event of failures and enhancing fault tolerance. It extracts orders to the exclusive processing queue of the optimal supplier by dynamically switching according to the topic of the message queue, reducing task processing delays and improving real-time performance.

[0073] In this embodiment, see Figure 2 , uses a multi-level feedback queue scheduling algorithm to handle concurrent tasks and dynamically allocates computing resources based on the thread pool, including the following steps:

[0074] Step S310: Obtain request information for executing a new task.

[0075] Step S320 , assigning the new task to a level in the multi-level queue based on the estimated execution time, task type, and characteristics of historical execution data.

[0076] In the actual application of the multi-level feedback queue scheduling algorithm, when a new task is received, it is first assigned to a level in the multi-level queue according to the characteristics of the task (such as estimated execution time, task type, historical data, etc.). Multiple independent task queues are maintained and arranged in descending order of priority (such as Q0 is the highest, Q n lowest).

[0077] Step S330 , arranging the tasks from high to low priority to obtain the immediate execution right of the high priority queue task.

[0078] By dynamically adjusting priorities, it is convenient to meet the diverse needs of railway material procurement.

[0079] Step S340: Acquire monitoring information of the task execution status and perform a queue demotion operation on unfinished tasks.

[0080] Build a full-link procurement monitoring system, collect task status change events through tracking points, and use the Elastic Stack technology stack to implement a Kibana-based visual monitoring screen, real-time tracking charts of task execution paths, automatic clustering analysis of abnormal events, and a multi-dimensional scoring dashboard for system health.

[0081] By monitoring the task execution status in real time and providing timely feedback of task processing results to users or external systems, users can be ensured to understand the task processing progress in real time, thus improving the real-time performance and user experience of the system.

[0082] Step S350: Periodically reset all unfinished tasks to the highest priority queue.

[0083] Set the priority reset period T. Every T time, scan all tasks in the queue whose waiting time exceeds the threshold K, reinsert them into the head of the Q0 queue and reset the priority label.

[0084] In this embodiment, see Figure 3 , sort the tasks from high to low priority to obtain the immediate execution right of the high-priority queue task, including the following steps:

[0085] Step S331 , sorting tasks from high to low priority, allocating 10ms time slices to the high priority queue to process real-time short tasks, and allocating 100ms time slices to the low priority queue to process batch long tasks.

[0086] Q0 high-priority queue: allocates smaller time slices (such as 10ms) when processing short or interactive tasks such as real-time requests and cache updates.

[0087] Q n Low-priority queue: When processing long-time tasks such as batch data processing and report exporting, a larger time slice (such as 100ms) is allocated.

[0088] Step S332: in response to the high priority queue generating a new task, immediately interrupt the execution of the current low priority task, retain its execution context and suspend the task, and release computing resources to prioritize processing the high priority queue task.

[0089] As long as there are tasks in the high-priority queue, the tasks in the low-priority queue must wait. For example, if Q0 suddenly receives a real-time task, the task currently executing in Q1 will be suspended and the thread will immediately switch to the Q0 task.

[0090] In this embodiment, see Figure 4 , obtain the monitoring information of the task execution status and perform queue downgrade operations on unfinished tasks, including the following steps:

[0091] Step S341 : In response to the task being completed within the allocated time slice, the thread directly processes the next task.

[0092] Step S342 : In response to the task not being completed within the allocated time slice, the task is extracted to the next level queue and assigned a time slice length of 1.5 to 2 times the original time slice.

[0093] The time slice exhaustion feedback mechanism can prevent long tasks from occupying high-priority resources for a long time.

[0094] In this embodiment, see Figure 5 , set the maximum retry threshold based on the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue, including the following steps:

[0095] Step S410 , in response to a task execution failure, retrying after waiting for a gradually increasing delay time, with the retry waiting time increasing exponentially.

[0096] When a task fails, the procurement system doesn't retry immediately. Instead, it waits for a gradually increasing delay before retrying. This prevents service avalanches and resource congestion caused by frequent retries. The wait time for each retry increases exponentially, giving the procurement system backend ample time to recover.

[0097] In step S420 , in response to the task still failing after exceeding the retry limit, the task is extracted to a dead letter queue, where the retry limit parameters are the maximum number of retries and the maximum delay time.

[0098] By setting the retry limit parameter, you can avoid task backlogs or long-term blocking caused by infinite retries.

[0099] In this embodiment, see Figure 6 Based on the supplier profile and material classification model, the optimal message routing path is dynamically calculated, and the order is extracted to the exclusive processing queue of the optimal supplier according to the dynamic switching of the message queue topic. The following steps are included:

[0100] Step S510, obtain supplier data including credit rating, historical fulfillment rate, logistics timeliness index, risk penalty record, and build a supplier multi-dimensional portrait library.

[0101] Credit ratings include bank credit scores, third-party credit reporting agency ratings, and historical payment default records. Historical fulfillment rates include on-time order delivery rates, contract terms fulfillment rates, quality compliance rates, and the number of returns or claims. Logistics timeliness indicators include average delivery cycles, emergency order response speeds, and transportation anomalies (delays, damage frequency). Risk penalty records include compliance violations, legal proceedings, blacklist history, and supplier audit failures.

[0102] Step S520: Establish a material classification knowledge graph based on the hierarchical feature vectors of urgency, material type, and quality standard.

[0103] By converting business experience into computable models, it is easier to realize intelligent railway material procurement.

[0104] Step S530 , calculating the matching scores between supplier capability values ​​and order requirements in real time to develop a dynamic routing decision matrix.

[0105] By developing a dynamic routing decision matrix, it is convenient to realize the intelligent scheduling center of the procurement system, forming a complete technical closed loop with multi-level feedback queues and portrait libraries, thereby improving response speed.

[0106] Step S540: Establish a temporary routing channel between the exclusive processing queue of the optimal supplier and the current order message to implement the Topic dynamic binding mechanism.

[0107] For example: When a batch of urgent bulk purchase orders is received, the supplier profile (such as credit rating, penalty record, logistics status) and material classification model (urgency, material type) will be combined, and the order will be delivered to the exclusive processing queue of the optimal supplier through dynamic switching of the message queue's topic, thereby completing intelligent distribution based on real-time risk and efficiency assessment in a very short time. Compared with traditional fixed routing methods, it shortens decision-making time and reduces fulfillment risks.

[0108] The railway material procurement asynchronous task processing device provided by the present invention is described below. The railway material procurement asynchronous task processing device described below and the railway material procurement asynchronous task processing method described above can be referenced to each other.

[0109] like Figure 7 As shown, in one embodiment, a railway material procurement asynchronous task processing device includes a construction module 710, an instruction module 720, a processing module 730, a threshold module 740 and a calculation module 750.

[0110] Construction module 710 is used to build a six-layer message channel architecture for the procurement business process, including demand submission, approval flow, supplier matching, order generation, logistics tracking, and settlement and payment services, and based on the core message middleware of the Kafka cluster, executes instructions for deploying active-active message proxy nodes for each channel.

[0111] The instruction module 720 is used to instruct the front-end gateway to encapsulate the procurement event into a binary message body with a transaction ID and a timestamp based on the atomic microservice components and event-driven service architecture encapsulated by the Spring Cloud Stream framework, and write it into a persistent queue after prioritizing it according to the task type tag.

[0112] The processing module 730 is used to process concurrent tasks using a multi-level feedback queue scheduling algorithm and dynamically allocate computing resources based on a thread pool.

[0113] The threshold module 740 is used to set a maximum retry threshold according to the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue.

[0114] The calculation module 750 is used to dynamically calculate the optimal message routing path based on the supplier profile and material classification model, and dynamically switch the topic of the message queue to extract the order to the exclusive processing queue of the optimal supplier.

[0115] In this embodiment, the processing module 730 is specifically configured to:

[0116] Get request information for executing a new task;

[0117] Assign new tasks to a level in a multi-level queue based on estimated execution time, task type, and historical execution data.

[0118] Arrange tasks from high to low priority to obtain immediate execution rights for high-priority queue tasks;

[0119] Obtain monitoring information on task execution status and perform queue downgrade operations on unfinished tasks;

[0120] Periodically reset all unfinished tasks to the highest priority queue.

[0121] In this embodiment, tasks are arranged from high to low priority to obtain immediate execution rights for high-priority queue tasks, specifically for:

[0122] Arrange tasks from high to low priority. The high-priority queue is allocated 10ms time slices to process real-time short tasks, and the low-priority queue is allocated 100ms time slices to process batch long tasks.

[0123] In response to a new task being generated by a high-priority queue, the execution of the current low-priority task is immediately interrupted, and the task is suspended after retaining its execution context, and computing resources are released to prioritize processing of high-priority queue tasks.

[0124] In this embodiment, monitoring information of the task execution status is obtained, and a queue downgrade operation is performed on unfinished tasks, specifically for:

[0125] In response to the task completing within the allocated time slice, the thread directly processes the next task;

[0126] In response to a task not being completed within the allocated time slice, the task is extracted to the next level queue and assigned a time slice length of 1.5 to 2 times the original time slice.

[0127] In this embodiment, the threshold module 740 is specifically configured to:

[0128] In response to a task failure, wait for a gradually increasing delay before retrying, with the retry wait time increasing exponentially;

[0129] In response to a task failing after exceeding the retry limit, the task is extracted to a dead letter queue. The retry limit parameters are the maximum number of retries and the maximum delay time.

[0130] In this embodiment, the calculation module 750 is specifically configured to:

[0131] Obtain supplier data including credit ratings, historical fulfillment rates, logistics timeliness indicators, and risk penalty records, and build a multi-dimensional supplier profile library;

[0132] Establish a material classification knowledge graph based on hierarchical feature vectors of urgency, material type, and quality standards;

[0133] Calculate the matching scores between supplier capability values ​​and order requirements in real time to develop a dynamic routing decision matrix;

[0134] Establish a temporary routing channel between the optimal supplier's exclusive processing queue and the current order message to implement the Topic dynamic binding mechanism.

[0135] The railway material procurement asynchronous task processing device decouples each subsystem through a message queue based on a six-layer message channel architecture, improves scalability and maintainability, and reduces coupling. It processes concurrent tasks by adopting a multi-level feedback queue scheduling algorithm and dynamically allocates computing resources based on a thread pool, making it convenient to dynamically adjust task priorities according to task type and urgency, meet the diverse needs of railway material procurement, improve concurrent processing capabilities, reduce task accumulation, and increase response speed. It uses an exponential backoff retry strategy to perform timely compensation processing when messages are lost or failures occur, making it convenient to ensure normal operation in the event of failures and enhance fault tolerance. By dynamically switching according to the Topic of the message queue, orders are extracted to the exclusive processing queue of the optimal supplier, reducing task processing delays and improving real-time performance.

[0136] like Figure 8 As shown, in one embodiment, a railway material procurement asynchronous task processing system includes a message queue layer 810, a microservice layer 820, a gateway layer 830 and an external access layer 840.

[0137] The message queue layer 810 includes a Kafka cluster 811 with active-active deployment and mirror queues, a multi-level feedback queue scheduler 812, a Redis distributed lock service 813, an intelligent routing decision engine 814 and a backoff retry controller 815. The multi-level feedback queue scheduler 812 has built-in multi-level priority queues and associated thread pool resource groups, and is used to perform time slice round-robin scheduling with a preemptive mechanism. The Redis distributed lock service 813 implements cross-node resource conflict arbitration through the SETNX command. The intelligent routing decision engine 814 is used to dynamically select the optimal message channel based on a scoring model built based on the supplier's historical performance data. The backoff retry controller 815 is used to calculate the retry interval according to an exponentially increasing algorithm. Failed tasks are automatically dumped to the dead letter queue after multiple retries.

[0138] The microservice layer 820 is connected to the message queue layer 810. The microservice layer 820 includes demand processing microservices, approval microservices, supplier microservices, and other domain microservices. The microservice layer 820 is used to connect to the message queue layer to publish domain events.

[0139] The gateway layer 830 is connected to the microservice layer 820. The gateway layer 830 integrates the Spring Cloud Gateway service gateway. The gateway layer 830 is used to perform protocol conversion and message normalization on inbound requests, implement dynamic routing distribution based on request path characteristics, and achieve load balancing of microservice instances through a weighted polling algorithm.

[0140] The external access layer 840 is connected to the gateway layer 830 and is used to receive procurement service requests initiated by multi-source heterogeneous terminals. The external access layer 840 includes a front-end gateway, a mobile terminal and a third-party system.

[0141] It should be noted that the railway material procurement asynchronous task processing system supports horizontal expansion and realizes containerized elastic deployment through Kubernetes. After stress testing, a single cluster can handle procurement transaction processing under high concurrency, significantly improving the collaborative efficiency and system robustness of railway material procurement.

[0142] Configure automatic scaling rules:

[0143] Automatically expand the queue capacity when the message backlog exceeds the queue capacity threshold

[0144] Automatically scale down when CPU utilization is lower than the set value and message delay meets the standard

[0145] Set the minimum number of surviving instances to ensure basic service capabilities.

[0146] The railway material procurement asynchronous task processing system decouples the subsystems through message queues, improves the system's scalability and maintainability, and reduces system coupling. The asynchronous processing method significantly improves the system's concurrent processing capabilities, reduces task accumulation, and improves system performance. The task retry and compensation mechanism ensures that the system can still operate normally in the event of a failure, enhances fault tolerance, supports task priority, and a dynamic priority adjustment mechanism to meet the diverse needs of railway material procurement. By optimizing the message queue distribution mechanism, it reduces task processing delays and achieves stronger real-time performance.

[0147] Figure 9 The following is a schematic diagram of the physical structure of an electronic device. The electronic device may be a smart terminal, and its internal structure diagram may be as follows: Figure 9 As shown. The electronic device includes a processor, a memory, and a network interface connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for processing asynchronous tasks for railway material procurement is implemented, which includes:

[0148] Build a six-layer message channel architecture for the procurement business process, including demand submission, approval process, supplier matching, order generation, logistics tracking, and settlement and payment. Based on the core message middleware of the Kafka cluster, execute instructions to deploy active-active message broker nodes for each channel.

[0149] Based on the atomic microservice components and event-driven service architecture encapsulated by the Spring Cloud Stream framework, the front-end gateway is instructed to encapsulate procurement events into binary message bodies with transaction IDs and timestamps, annotate them with priorities based on task type tags, and write them to a persistent queue.

[0150] A multi-level feedback queue scheduling algorithm is used to handle concurrent tasks, and computing resources are dynamically allocated based on the thread pool;

[0151] Set the maximum retry threshold based on the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue;

[0152] Based on the supplier profile and material classification model, the optimal message routing path is dynamically calculated, and the order is extracted to the exclusive processing queue of the optimal supplier according to the dynamic switching of the message queue topic.

[0153] Those skilled in the art will understand that Figure 9 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the electronic device to which the solution of the present invention is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0154] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements a method for processing asynchronous tasks for railway material procurement, the method comprising:

[0155] Build a six-layer message channel architecture for the procurement business process, including demand submission, approval process, supplier matching, order generation, logistics tracking, and settlement and payment. Based on the core message middleware of the Kafka cluster, execute instructions to deploy active-active message broker nodes for each channel.

[0156] Based on the atomic microservice components and event-driven service architecture encapsulated by the Spring Cloud Stream framework, the front-end gateway is instructed to encapsulate procurement events into binary message bodies with transaction IDs and timestamps, annotate them with priorities based on task type tags, and write them to a persistent queue.

[0157] A multi-level feedback queue scheduling algorithm is used to handle concurrent tasks, and computing resources are dynamically allocated based on the thread pool;

[0158] Set the maximum retry threshold based on the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue;

[0159] Based on the supplier profile and material classification model, the optimal message routing path is dynamically calculated, and the order is extracted to the exclusive processing queue of the optimal supplier according to the dynamic switching of the message queue topic.

[0160] In another aspect, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, implements a method for processing asynchronous tasks for railway material procurement, the method comprising:

[0161] Build a six-layer message channel architecture for the procurement business process, including demand submission, approval process, supplier matching, order generation, logistics tracking, and settlement and payment. Based on the core message middleware of the Kafka cluster, execute instructions to deploy active-active message broker nodes for each channel.

[0162] Based on the atomic microservice components and event-driven service architecture encapsulated by the Spring Cloud Stream framework, the front-end gateway is instructed to encapsulate procurement events into binary message bodies with transaction IDs and timestamps, annotate them with priorities based on task type tags, and write them to a persistent queue.

[0163] A multi-level feedback queue scheduling algorithm is used to handle concurrent tasks, and computing resources are dynamically allocated based on the thread pool;

[0164] Set the maximum retry threshold based on the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue;

[0165] Based on the supplier profile and material classification model, the optimal message routing path is dynamically calculated, and the order is extracted to the exclusive processing queue of the optimal supplier according to the dynamic switching of the message queue topic.

[0166] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0167] By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0168] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The above-described embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, and these modifications and improvements fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A railway material procurement asynchronous task processing method, characterized in that: The method comprises: Build a six-layer message channel architecture for the procurement business process, including demand submission, approval process, supplier matching, order generation, logistics tracking, and settlement and payment. Based on the core message middleware of the Kafka cluster, execute instructions to deploy active-active message broker nodes for each channel. Based on the atomic microservice components and event-driven service architecture encapsulated by the Spring Cloud Stream framework, the front-end gateway is instructed to encapsulate procurement events into binary message bodies with transaction IDs and timestamps, annotate them with priorities based on task type tags, and write them to a persistent queue. A multi-level feedback queue scheduling algorithm is used to handle concurrent tasks, and computing resources are dynamically allocated based on the thread pool; Set the maximum retry threshold based on the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue; Based on the supplier profile and material classification model, the optimal message routing path is dynamically calculated, and the order is extracted to the exclusive processing queue of the optimal supplier according to the dynamic switching of the message queue topic.

2. The railway material procurement asynchronous task processing method according to claim 1, characterized in that: The multi-level feedback queue scheduling algorithm is used to process concurrent tasks and dynamically allocate computing resources based on the thread pool, including: Get request information for executing a new task; Assign new tasks to a level in a multi-level queue based on estimated execution time, task type, and historical execution data. Arrange tasks from high to low priority to obtain immediate execution rights for high-priority queue tasks; Obtain monitoring information on task execution status and perform queue downgrade operations on unfinished tasks; Periodically reset all unfinished tasks to the highest priority queue.

3. The railway material procurement asynchronous task processing method according to claim 2, characterized in that: The tasks are arranged in descending order of priority to obtain the immediate execution right of the high-priority queue tasks, including: Arrange tasks from high to low priority. The high-priority queue is allocated 10ms time slices to process real-time short tasks, and the low-priority queue is allocated 100ms time slices to process batch long tasks. In response to a new task being generated by a high-priority queue, the execution of the current low-priority task is immediately interrupted, and the task is suspended after retaining its execution context, and computing resources are released to prioritize processing of high-priority queue tasks.

4. The railway material procurement asynchronous task processing method according to claim 3 is characterized in that: The acquisition of monitoring information of the task execution status and the execution of queue downgrade operations on unfinished tasks include: In response to the task completing within the allocated time slice, the thread directly processes the next task; In response to a task not being completed within the allocated time slice, the task is extracted to the next level queue and assigned a time slice length of 1.5 to 2 times the original time slice.

5. The railway material procurement asynchronous task processing method according to claim 4 is characterized in that: The exponential backoff retry strategy is used to set the maximum retry threshold so that failed tasks are automatically transferred to the dead letter queue, including: In response to a task failure, wait for a gradually increasing delay before retrying, with the retry wait time increasing exponentially; In response to a task failing after exceeding the retry limit, the task is extracted to a dead letter queue. The retry limit parameters are the maximum number of retries and the maximum delay time.

6. The railway material procurement asynchronous task processing method according to claim 5, characterized in that: The optimal message routing path is dynamically calculated based on the supplier profile and material classification model, and the order is extracted to the exclusive processing queue of the optimal supplier according to the dynamic switching of the message queue topic, including: Obtain supplier data including credit ratings, historical fulfillment rates, logistics timeliness indicators, and risk penalty records, and build a multi-dimensional supplier profile library; Establish a material classification knowledge graph based on hierarchical feature vectors of urgency, material type, and quality standards; Calculate the matching scores between supplier capability values ​​and order requirements in real time to develop a dynamic routing decision matrix; Establish a temporary routing channel between the optimal supplier's exclusive processing queue and the current order message to implement the Topic dynamic binding mechanism.

7. A railway material procurement asynchronous task processing device, characterized in that: include: The construction module is used to build a six-layer message channel architecture for the procurement business process, including demand submission, approval process, supplier matching, order generation, logistics tracking, and settlement and payment. Based on the core message middleware of the Kafka cluster, it executes instructions to deploy active-active message broker nodes for each channel. The instruction module is used to instruct the front-end gateway to encapsulate procurement events into binary messages with transaction IDs and timestamps, annotate them with priorities based on task type tags, and write them to a persistent queue based on the atomic microservice components and event-driven service architecture encapsulated using the Spring Cloud Stream framework. The processing module is used to process concurrent tasks using a multi-level feedback queue scheduling algorithm and dynamically allocate computing resources based on a thread pool; The threshold module is used to set the maximum retry threshold based on the exponential backoff retry strategy so that failed tasks are automatically transferred to the dead letter queue; The calculation module is used to dynamically calculate the optimal message routing path based on the supplier profile and material classification model, and dynamically switch the topic of the message queue to extract the order to the exclusive processing queue of the optimal supplier.

8. A railway material procurement asynchronous task processing system, characterized in that: include: The message queue layer includes a Kafka cluster with active-active deployment and mirrored queues, a multi-level feedback queue scheduler, a Redis distributed lock service, an intelligent routing decision engine, and a backoff and retry controller. The multi-level feedback queue scheduler has built-in multi-level priority queues and associated thread pool resource groups, and is used to perform time-slice round-robin scheduling with a preemptive mechanism. The Redis distributed lock service implements cross-node resource conflict arbitration through the SETNX command. The intelligent routing decision engine is used to dynamically select the optimal message channel based on a scoring model built based on the supplier's historical performance data. The backoff and retry controller is used to calculate the retry interval using an exponentially increasing algorithm. Failed tasks are automatically dumped to a dead letter queue after multiple retries. The microservice layer is connected to the message queue layer and includes demand processing microservices, approval microservices, supplier microservices, and other domain microservices. It is also used to connect to the message queue layer to publish domain events. The gateway layer is connected to the microservice layer and integrates the Spring Cloud Gateway service gateway. It is used to perform protocol conversion and message normalization on inbound requests, implement dynamic routing distribution based on request path characteristics, and achieve load balancing of microservice instances through a weighted polling algorithm; The external access layer is connected to the gateway layer and is used to receive procurement business requests initiated by multi-source heterogeneous terminals, and includes front-end gateways, mobile terminals and third-party systems.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images