An automated testing method and apparatus for complex scenarios of call center queuing systems

By using a scenario orchestration engine and a virtual terminal simulator, combined with a global event timeline and a freeze-time state machine model, automated testing of the call center queuing system was achieved. This solved the problems of low test coverage and fairness verification in existing technologies, and improved testing efficiency and accuracy.

CN123123500APending Publication Date: 2026-11-17FUJIAN FUJITSU COMM SOFTWARE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611049355.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-15
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing call center queuing system testing struggles to accurately simulate scenarios with multiple concurrent customers and complex time-series interactions. Manual testing has low coverage and cannot verify queuing fairness and exception handling logic. Regression testing is inefficient and prone to missing defects.

Method used

Test scenarios are defined using a scenario orchestration engine, concurrent calls are generated through a virtual terminal simulator, the sequence verification engine is used to accurately verify the order of events, and automated verification is performed by combining a global event timeline and a frozen state machine model to generate a multi-dimensional quantitative difference report.

Benefits of technology

It achieves full-link logic automation testing of the queuing system, improves coverage and regression testing efficiency, ensures fairness and stability, provides objective parameter tuning suggestions, and supports continuous monitoring of the production environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123500A_ABST
    Figure CN123123500A_ABST
Patent Text Reader

Abstract

This invention discloses an automated testing method and apparatus for complex scenarios in call center queuing systems. The method generates test cases by parsing scenario description files; drives call interactions through SIP virtual clients and agent simulators, recording event timelines; performs FIFO integrity verification to detect whether there are earlier queued but unconnected clients during the connection event, and traces back the freeze period to diagnose root causes; automatically verifies the accuracy of agent behaviors such as ringing stop, pause assignment, key resumption, and timeout hang-up based on assertions injected into the freeze period state machine; and performs multi-dimensional quantitative comparison of the behavior of the system under test with a reference model. This invention can automatically generate a test matrix covering concurrent and abnormal scenarios, and complete full-dimensional verification of the freeze period in a single execution, significantly improving regression testing efficiency and coverage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of communication testing technology, and in particular to an automated testing method and apparatus for complex scenarios of call center queuing systems. Background Technology

[0002] In existing call center queuing systems, the queuing and agent allocation process after a customer calls in involves multiple timeout mechanisms, multiple routing strategies, and complex exception handling logic. A typical process includes: customer entering the queue, Automatic Call Distribution (ACD) selecting an agent according to the routing strategy, agent ringing (with configurable timeout), agent entering a freeze period after call rejection (requiring customer key confirmation), retrying until the call is connected or disconnected, etc.

[0003] Currently, testing such queuing systems faces the following technical challenges: 1) In real production environments, scenarios such as multiple customers calling in almost simultaneously, agents responding / rejecting / timeout at different times, and allocation order under different routing strategies are intertwined. Manual testing cannot accurately simulate and reproduce these complex timing interactions. 2) During the freeze period, the system does not allocate the current customer, and subsequent customers may be connected first, leading to the violation of the first-in, first-out (FIFO) principle. Traditional black-box testing methods only focus on the final connection rate and cannot accurately verify queuing fairness from the perspective of event timing. 3) The system involves multiple layers of timeouts, such as agent ringing timeout (e.g., configurable to 20 seconds), freeze period timeout (e.g., 10-second button wait), and retry limit. The interaction combinations between each layer form an exponential number of test scenarios, resulting in low coverage for manual testing. 4) After each adjustment of the ACD routing strategy or timeout parameters, a large amount of manual regression testing is required to verify that the changes have not introduced new queuing logic defects, which is inefficient and prone to omissions. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing testing technologies and provide an automated testing method and device for complex scenarios of call center queuing systems. By defining test scenarios through a scenario orchestration engine, generating concurrent calls and agent behaviors through a virtual terminal simulator, accurately verifying the event sequence through a timing verification engine, and quantifying the differences between the system and industry standards through a comparative analysis engine, this invention achieves automated, high-coverage, and reproducible verification of the entire queuing system logic. It aims to ensure the fairness, stability, and user experience of the queuing system from a testing perspective.

[0005] The technical solution adopted in this invention is:

[0006] An automated testing method for complex scenarios of a call center queuing system includes the following steps:

[0007] Obtain the pre-configured test scenario description file for the business process based on the queuing system, and parse the test parameters in the test scenario description file to generate a test case matrix; the test parameters include at least virtual customer parameters, virtual agent parameters, routing strategy, ringing timeout time, freeze period duration, and number of retries;

[0008] The virtual client simulator and virtual agent simulator are created and driven based on the Session Initiation Protocol (SIP). The call interaction is executed according to the test case matrix, and the precise timestamp and event type of each call event are recorded.

[0009] All call events are collected based on a global clock, and a global event timeline is constructed based on the timestamps of the call events.

[0010] Automatic verification of FIFO integrity is performed on the global event timeline: Whenever a connection event is detected, it is checked whether there are other virtual customers with earlier queuing times who have not yet been connected; if so, a FIFO violation event is determined, and the virtual customers with earlier queuing times are checked back to see if they are in the freeze period at the current connection time, and a diagnostic conclusion containing the root cause of the violation is generated.

[0011] Multi-dimensional automatic verification is performed on the freeze period triggered by Ring No Answer (RNA): Based on the preset freeze period state machine model, assertion probes are injected into the state transition nodes of the freeze period to automatically verify the preset set of behaviors during the freeze period and verify the time deviation of the state transition; the preset set of behaviors includes agent stop ringing, system pauses allocation, resumes allocation after receiving customer key press, and automatically hangs up after freeze period timeout;

[0012] The test case matrix is ​​executed, and the actual call event behavior of the queuing system under test is compared with the expected behavior defined by the preset reference model through the comparative analysis engine, generating a multi-dimensional quantitative difference report.

[0013] Furthermore, the implementation of automatic FIFO integrity verification for the global event timeline includes:

[0014] Sort all events in the global event timeline in ascending order by timestamp;

[0015] Maintain a set of unconnected customers sorted by their enqueue time;

[0016] Iterate through the sorted events, and for each connection event, obtain the queuing time of the currently connected customer;

[0017] If there is another customer in the set of unconnected customers whose queuing time is earlier than that of the currently connected customer, a FIFO violation event is determined to have occurred. The system will then backtrack to check whether the virtual customer with the earlier queuing time was in a frozen period at the time of the current connection and generate a diagnostic conclusion containing the root cause of the violation. Otherwise, the currently connected customer will be removed from the set of unconnected customers.

[0018] The implementation of the diagnostic conclusion that includes the root cause violation includes: checking whether another customer with an earlier queuing time was in a frozen state when the connection event occurred; if so, outputting that the root cause is caused by the frozen state mechanism; if not, outputting that the root cause is an error in the routing policy execution.

[0019] Furthermore, based on the preset state machine model for the freeze period, assertion probes are injected into the state transition nodes during the freeze period to automatically verify the preset set of behaviors within the freeze period, specifically including:

[0020] Define the states of the frozen state machine model, including: frozen state, recovery state, and timeout disconnected state;

[0021] When a ringing no-response event is detected, triggering the start of a freeze, the system enters a frozen state and triggers the first assertion probe to verify whether the agent has stopped ringing and whether the system has paused allocation.

[0022] If a key signal sent by a customer is detected before the freeze timer expires, the system transitions from the frozen state to the recovery state and triggers the second assertion probe to verify whether the system has reassigned the agent and resumed ringing.

[0023] When the freeze timer expires and no key signal is detected, the system transitions from the frozen state to the timeout hang-up state and triggers the third assertion probe to verify whether the hang-up command was issued and whether the reason code is correct.

[0024] The implementation of verifying the time deviation of state transition is as follows: verify whether the deviation between the actual trigger duration of the freeze period timer and the pre-configured freeze period duration is within the preset threshold range.

[0025] Furthermore, the specific implementation of generating multi-dimensional quantitative difference reports includes:

[0026] Simulations of the queuing system under test and the reference model are run in parallel under the same test scenario.

[0027] The event timeline generated by the queuing system under test is normalized and time-aligned with the event timeline generated by the reference model.

[0028] Automatically calculate and generate multi-dimensional quantitative difference indicators, which include at least the difference in the number of FIFO violations, the difference in the system's proactive hang-up rate, the difference in the average number of customer operations, the difference in the connection rate, and the difference in the quantile of waiting time.

[0029] Furthermore, the reference models include built-in industry-standard reference models that conform to the ITIL (Information Technology Infrastructure Library) specification, and / or custom-injected expected behavior models based on testing requirements.

[0030] Furthermore, the test scenario description file uses YAML or JSON format to define the test intent in a declarative manner; when generating the test case matrix, the test parameters are expanded by combining all factors to automatically generate a set of test cases covering multiple routing strategies, multiple agent behaviors, multiple client concurrency, and multiple exception handling paths.

[0031] Furthermore, the method of the present invention also includes:

[0032] Based on multi-dimensional quantitative difference reports, assess the impact of queuing parameter changes on system performance indicators and generate parameter tuning recommendations.

[0033] Furthermore, the present invention also discloses an automated testing device for complex scenarios of a call center queuing system, which applies the aforementioned automated testing method for complex scenarios of a call center queuing system. The device includes:

[0034] The test scenario orchestration engine is used to acquire and parse pre-configured test scenario description files and generate a test case matrix from the test parameters;

[0035] The virtual terminal simulation layer, including a virtual client simulator and a virtual agent simulator, is used to execute call interactions in the test case matrix based on the Session Initiation Protocol (SIP) and record the precise timestamp and event type of each call event generated.

[0036] The timing verification engine collects all call events based on a global clock, constructs a global event timeline based on the timestamps of the call events, and performs automatic verification of FIFO integrity and multi-dimensional automatic verification of the freeze period. Automatic FIFO integrity verification includes checking for other virtual customers with earlier queuing times but not yet connected whenever a connection event is detected. If found, a FIFO violation is determined, and the system backtracks to check whether the customer was in the freeze period at the time of connection to output a diagnostic conclusion. Multi-dimensional automatic verification of the freeze period includes injecting assertion probes into the state transition nodes of the freeze period based on the freeze period state machine model to verify behaviors such as agent stopping ringing, system pausing allocation, resuming allocation after key press, and automatic hang-up after timeout, as well as the time deviation of state transitions.

[0037] The comparative analysis engine is used to compare the actual call event behavior of the queuing system under test with the expected behavior defined by the preset reference model, and generate a multi-dimensional quantitative difference report.

[0038] Furthermore, when the timing verification engine performs automatic FIFO integrity verification, it is used for:

[0039] Sort all events in ascending order of timestamp, and maintain a set of unconnected customers sorted by enqueue time;

[0040] For each connection event, if there is another customer in the set of unconnected customers who joined the queue earlier, it is determined to be a FIFO violation, and further identification is made to determine whether the other customer is in a frozen period in order to output a diagnostic conclusion containing the root cause.

[0041] Furthermore, the comparative analysis engine is used for:

[0042] In the same test scenario, the virtual terminal simulation layer is driven in parallel to perform simulations of the system under test and the reference model.

[0043] After normalizing and aligning the event timelines generated by both, multi-dimensional quantitative difference indicators are calculated and generated. These quantitative difference indicators include at least one of the following: FIFO violation count difference, system-initiated hang-up rate difference, average number of customer operations difference, and connection rate difference.

[0044] The present invention adopts the above technical solution and has the following beneficial effects compared with the prior art:

[0045] (1) Through the scenario orchestration engine and the virtual terminal simulation layer, routing strategies, agent behaviors, timeout parameters, etc. can be automatically combined to generate exponential test scenarios, realizing accurate reproduction of multi-customer concurrency and abnormal timing interactions, completely solving the problems of low coverage and inability to simulate concurrent scenarios in manual testing, and improving regression testing efficiency by tens of times. (2) The FIFO integrity automatic verification algorithm based on the global event timeline can accurately detect the hidden fairness defects of "last-in, first-out" from the event sequence, and automatically trace back the frozen period state to distinguish the root cause, fundamentally ensuring the fairness verification of the queuing system. (3) This invention utilizes a state machine assertion framework dedicated to the frozen period, which can automatically complete multi-dimensional verification such as agent ringing stop, pause allocation, key recovery, timeout hang-up and time accuracy in a single execution, ensuring that the abnormal handling logic is completely complete. (4) By comparing the actual behavior with the industry standard reference model in multiple dimensions, subjective experience is transformed into objective indicators such as FIFO violation rate and active hang-up rate, providing data decision support for parameter optimization and product iteration, and supporting continuous monitoring of the production environment, thus realizing the upgrade from passive complaints to active discovery. Attached Figure Description

[0046] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0047] Figure 1 This is a schematic diagram of the principle architecture of an automated testing device for complex scenarios of a call center queuing system according to the present invention;

[0048] Figure 2 This is a flowchart illustrating the event timeline analysis for FIFO integrity verification in this invention.

[0049] Figure 3 This is a schematic diagram illustrating the principle and flow of the frozen period verification state machine model of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0051] like Figures 1 to 3 As shown, this invention discloses an automated testing method for complex scenarios of a call center queuing system, comprising the following steps:

[0052] Obtain the pre-configured test scenario description file for the business process based on the queuing system, and parse the test parameters in the test scenario description file to generate a test case matrix; the test parameters include at least virtual customer parameters, virtual agent parameters, routing strategy, ringing timeout time, freeze period duration, and number of retries;

[0053] The virtual client simulator and virtual agent simulator are created and driven based on the Session Initiation Protocol (SIP). The call interaction is executed according to the test case matrix, and the precise timestamp and event type of each call event are recorded.

[0054] All call events are collected based on a global clock, and a global event timeline is constructed based on the timestamps of the call events.

[0055] Automatic verification of FIFO integrity is performed on the global event timeline: Whenever a connection event is detected, it is checked whether there are other virtual customers with earlier queuing times who have not yet been connected; if so, a FIFO violation event is determined, and the virtual customers with earlier queuing times are checked back to see if they are in the freeze period at the current connection time, and a diagnostic conclusion containing the root cause of the violation is generated.

[0056] Multi-dimensional automatic verification is performed on the freeze period triggered by Ring No Answer (RNA): Based on the preset freeze period state machine model, assertion probes are injected into the state transition nodes of the freeze period to automatically verify the preset set of behaviors during the freeze period and verify the time deviation of the state transition; the preset set of behaviors includes agent stop ringing, system pauses allocation, resumes allocation after receiving customer key press, and automatically hangs up after freeze period timeout;

[0057] The test case matrix is ​​executed, and the actual call event behavior of the queuing system under test is compared with the expected behavior defined by the preset reference model through the comparative analysis engine, generating a multi-dimensional quantitative difference report.

[0058] Specifically, the automated testing method of the present invention can be divided into the following four stages:

[0059] Phase 1: Test Scenario Preparation: Testers write scenario description files (YAML / JSON) to define the connection parameters of the system under test (SIP proxy address, port), virtual client parameters (number, caller ID range), virtual agent parameters (number, extension number range, skill group), routing policy configuration, timeout parameter configuration (ringing timeout T1, freeze period T2, number of retries R), and behavior scripts for each virtual agent (such as "respond on the first 2 calls, answer on the 3rd" or "always reject calls").

[0060] Phase Two: Test Environment Initialization: The scenario orchestration engine parses the description file and automatically expands the parameter combinations to generate a test case matrix. An independent virtual terminal instance pool is created for each test case, SIP registration is initialized (registering all virtual agents to the ACD system under test), and an event acquisition channel is established (through SIP message monitoring and RTP stream analysis interfaces).

[0061] Phase 3: Test Execution and Real-time Monitoring: Following the defined time intervals, the virtual client simulator sequentially initiates SIP INVITE calls. A global clock records the precise queuing timestamp for each call. The virtual agent simulator, according to a preset behavior script, triggers a response (200 OK for answering or 603 Decline for rejecting) when the ringing countdown precisely expires, or automatically triggers an RNA (RNAi) when the timeout threshold is reached. After detecting the RNA, the system records the start timestamp of the freeze period and monitors the client-side IVR playback event ("Press 1 to continue waiting" prompt tone injection). For key press scenarios, the virtual client sends a DTMF signal (press 1) during the freeze period; the system records the key press timestamp and verifies the reallocation behavior. For timeout scenarios, the virtual client remains silent until the freeze period timer expires; the system verifies the automatic hang-up event.

[0062] Phase 4: Result Verification and Report Generation: The timing verification engine collects all event records and executes verification rules one by one: FIFO rule verification (for any two customers A and B, if A's queuing time is earlier than B's, then A's connection time should be earlier than B's, unless A is in a freeze period and has not pressed a key), freeze period behavior verification (assertions for agents not ringing, system not assigning, customer key press recovery, and timeout hang-up), timeout accuracy verification (actual ringing duration vs. configured 20s, actual freeze period vs. configured 10s, with an allowable error of + / - 500ms), and routing policy execution verification (whether all idle agents ring simultaneously in full ringing mode, and whether they are assigned in sequence in polling mode). After verification, the comparative analysis report engine generates a test report, marking all failed assertions and their event timeline evidence.

[0063] Furthermore, the implementation of automatic FIFO integrity verification for the global event timeline includes:

[0064] Sort all events in the global event timeline in ascending order by timestamp;

[0065] Maintain a set of unconnected customers sorted by their enqueue time;

[0066] Iterate through the sorted events, and for each connection event, obtain the queuing time of the currently connected customer;

[0067] If there is another customer in the set of unconnected customers whose queuing time is earlier than that of the currently connected customer, a FIFO violation event is determined to have occurred. The system will then backtrack to check whether the virtual customer with the earlier queuing time was in a frozen period at the time of the current connection and generate a diagnostic conclusion containing the root cause of the violation. Otherwise, the currently connected customer will be removed from the set of unconnected customers.

[0068] The implementation of the diagnostic conclusion that includes the root cause violation includes: checking whether another customer with an earlier queuing time was in a frozen state when the connection event occurred; if so, outputting that the root cause is caused by the frozen state mechanism; if not, outputting that the root cause is an error in the routing policy execution.

[0069] Specifically, such as Figure 2 The example shown illustrates a multi-seat FIFO violation detection instance. Test scenario configuration: Number of agents = 2 (Agent 1, Agent 2), routing strategy = round-robin, ring timeout = 20s, freeze period = 10s, retries = 2. Customer A calls in first (t=0s), Customer B calls in at t=2s, and Customer C calls in at t=5s. Agent 1's behavior: Always RNA (do not answer). Agent 2's behavior: Answer Customer B at t=10s.

[0070] Current system behavior: When seat 2 becomes idle, it skips the frozen A and directly serves B / C who have just joined the queue, causing a FIFO violation.

[0071] Verification Algorithm: Traverse the event timeline and check each customer pair (x, y) – if x.queuing time < y.queuing time but x.connection time > y.connection time, mark it as a FIFO violation. For the marked case, further check whether x was in a freeze period when y was connected. If so, confirm that the freeze period is the root cause of the violation and output the diagnosis conclusion "Freeze period caused FIFO violation".

[0072] Furthermore, based on the preset state machine model for the freeze period, assertion probes are injected into the state transition nodes during the freeze period to automatically verify the preset set of behaviors within the freeze period, specifically including:

[0073] Define the states of the frozen state machine model, including: frozen state, recovery state, and timeout disconnected state;

[0074] When a ringing no-response event is detected, triggering the start of a freeze, the system enters a frozen state and triggers the first assertion probe to verify whether the agent has stopped ringing and whether the system has paused allocation.

[0075] If a key signal sent by a customer is detected before the freeze timer expires, the system transitions from the frozen state to the recovery state and triggers the second assertion probe to verify whether the system has reassigned the agent and resumed ringing.

[0076] When the freeze timer expires and no key signal is detected, the system transitions from the frozen state to the timeout hang-up state and triggers the third assertion probe to verify whether the hang-up command was issued and whether the reason code is correct.

[0077] The implementation of verifying the time deviation of state transition is as follows: verify whether the deviation between the actual trigger duration of the freeze period timer and the pre-configured freeze period duration is within the preset threshold range.

[0078] Specifically, such as Figure 3 The complete state machine verification model for the freeze period is shown. The test scenario configuration for the freeze period timeout boundary test is as follows: number of agents = 1, routing policy = full ringing, ringing timeout = 20s, freeze period = 10s, number of retries = 1. The customer calls in at t=0s. Agent behavior: first ringing RNA (20s timeout).

[0079] Sub-scenario 1 (Key Resumption): A virtual customer sends a DTMF key press 1 at the 5th second of the freeze period (t=25s). Verification point: The system should immediately unfreeze the system after the key press, re-add the customer to the allocation queue, and resume ringing for the agent.

[0080] Sub-scenario 2 (Key Timeout Boundary): A virtual client sends DTMF key 1 at 9.9s during the freeze period (t=29.9s). Verification point: The system should accept the key press (within the 10s threshold) and resume allocation normally.

[0081] Sub-scenario 3 (Timeout Hang-up Boundary): The virtual client sends DTMF button 1 at 10.1s (t=30.1s) during the freeze period. Verification point: If the 10s timer has expired precisely, the button should be ineffective, and the system should have issued a hang-up command. This test point is used to verify the strictness of the timeout accuracy.

[0082] Sub-scenario 4 (Retry Exhaustion): A virtual customer presses button 1 to resume during the first freeze period, and the second ring triggers another freeze period. The number of retries has been exhausted (1), and the system should automatically disconnect after 10 seconds instead of continuing to wait. Verification points: Is the timestamp of the disconnection event correct? Is the disconnection reason code "Retry Exhaustion" instead of "Customer did not press a button"?

[0083] Furthermore, this invention also includes routing policy consistency verification. The testing method involves executing the same client / agent behavior script under three routing policies: full ringing, round-robin, and random. This verifies whether the allocation behavior under each policy strictly conforms to its definition.

[0084] Full ringing strategy verification: Check whether INVITE ringing requests are sent to all N seats simultaneously when there are N available seats; and whether the ringing of the remaining seats is canceled by CANCEL after any seat answers.

[0085] Polling strategy verification: Maintain an expected allocation sequence and update the pointer after each allocation. Check whether the actual allocated agent number matches the expected sequence (skipping due to busy agents is allowed, but the overall order should not be skipped).

[0086] Random strategy validation: By repeatedly executing the strategy (>=100 times, ensuring that the expected selection frequency of each agent is >=5, satisfying the preconditions of the chi-square test), the frequency distribution of each agent's selection is statistically analyzed. The chi-square goodness-of-fit test is used to verify whether the distribution is uniform (p>0.05), eliminating pseudo-randomness or bias. When the number of agents M is large, the number of executions should be appropriately increased (ensuring N / M >= 5).

[0087] Furthermore, the specific implementation of generating multi-dimensional quantitative difference reports includes:

[0088] Simulations of the queuing system under test and the reference model are run in parallel under the same test scenario.

[0089] The event timeline generated by the queuing system under test is normalized and time-aligned with the event timeline generated by the reference model.

[0090] Automatically calculate and generate multi-dimensional quantitative difference indicators, which include at least the difference in the number of FIFO violations, the difference in the system's proactive hang-up rate, the difference in the average number of customer operations, the difference in the connection rate, and the difference in the quantile of waiting time.

[0091] Furthermore, the reference models include built-in industry-standard reference models that conform to the ITIL (Information Technology Infrastructure Library) specification, and / or custom-injected expected behavior models based on testing requirements.

[0092] Furthermore, the test scenario description file uses YAML or JSON format to define the test intent in a declarative manner; when generating the test case matrix, the test parameters are expanded by combining all factors to automatically generate a set of test cases covering multiple routing strategies, multiple agent behaviors, multiple client concurrency, and multiple exception handling paths.

[0093] Furthermore, the method of the present invention also includes:

[0094] Based on multi-dimensional quantitative difference reports, assess the impact of queuing parameter changes on system performance indicators and generate parameter tuning recommendations.

[0095] Furthermore, such as Figure 1 As shown, the present invention also discloses an automated testing device for complex scenarios of a call center queuing system, which applies the aforementioned automated testing method for complex scenarios of a call center queuing system. The device includes:

[0096] The test scenario orchestration engine is used to acquire and parse pre-configured test scenario description files, generating a test case matrix from the test parameters. Specifically, as a feasible implementation, the test scenario orchestration engine is responsible for parsing the scenario description files, transforming declarative test intentions into a sequence of executable test steps. Scenario templates support parameterized configuration (number of customers N, number of agents M, routing policy S, ringing timeout T_ring, freeze period T_freeze, number of retries R, and the behavior sequence of each agent, etc.), automatically expanding into a full-factor combination test matrix.

[0097] The virtual terminal simulation layer, comprising a virtual client simulator and a virtual agent simulator, is used to execute call interactions within the test case matrix based on the Session Initiation Protocol (SIP), and to record the precise timestamp and event type of each generated call event. Specifically, as a feasible implementation, the virtual terminal simulation layer implements virtual clients and virtual agents based on the SIP protocol stack. Each virtual terminal maintains an independent state machine (idle, ringing, talking, post-processing, idle), and state transitions strictly adhere to system-defined timeout parameters. Batch concurrent creation of virtual terminals is supported to simulate high-load stress scenarios.

[0098] The timing verification engine collects all call events based on a global clock, constructs a global event timeline based on the timestamps of the call events, and performs automatic verification of FIFO integrity and multi-dimensional automatic verification of the freeze period. Automatic FIFO integrity verification includes checking for other virtual customers with earlier queuing times but not yet connected whenever a connection event is detected. If found, a FIFO violation is determined, and the system backtracks to check whether the customer was in a freeze period at the time of connection to output a diagnostic conclusion. Multi-dimensional automatic verification of the freeze period includes injecting assertion probes into the state transition nodes of the freeze period based on the freeze period state machine model to verify behaviors such as agent stopping ringing, system pausing allocation, resuming allocation after key press, and automatic hang-up after timeout, as well as the time deviation of state transitions. Specifically, as a feasible implementation, the timing verification engine collects the {timestamp, event type, participant} triples of all call events to construct a global event timeline. For each queuing fairness assertion (e.g., "customers who join the queue first should be assigned seats before those who join later"), efficient verification is performed by combining a single traversal of the event line with an ordered queue cache (based on the inherent ordering of event timestamps, the algorithm complexity is O(n), where n is the total number of events). For the freeze period assertion, the expected behavior is derived through a state machine and compared with the actual behavior.

[0099] The comparative analysis engine is used to compare the actual call event behavior of the queuing system under test with the expected behavior defined by the preset reference model, and generate a multi-dimensional quantitative difference report.

[0100] Furthermore, when the timing verification engine performs automatic FIFO integrity verification, it is used for:

[0101] Sort all events in ascending order of timestamp, and maintain a set of unconnected customers sorted by enqueue time;

[0102] For each connection event, if there is another customer in the set of unconnected customers who joined the queue earlier, it is determined to be a FIFO violation, and further identification is made to determine whether the other customer is in a frozen period in order to output a diagnostic conclusion containing the root cause.

[0103] Furthermore, the comparative analysis engine is used for:

[0104] In the same test scenario, the virtual terminal simulation layer is driven in parallel to perform simulations of the system under test and the reference model.

[0105] After normalizing and aligning the event timelines generated by both, multi-dimensional quantitative difference indicators are calculated and generated. These quantitative difference indicators include at least one of the following: FIFO violation count difference, system-initiated hang-up rate difference, average number of customer operations difference, and connection rate difference.

[0106] This invention upgrades the traditional queuing system testing, which relies on manual observation, into a fully automated and quantifiable engineering testing process, significantly improving test coverage and execution efficiency. This invention can be widely applied to the following testing scenarios: (1) Call center platform version regression testing: Before each version release of the call center platform, a full queuing scenario regression test suite is automatically executed. The test covers a combination of different routing strategies x different agent sizes x different RNA frequencies. A single regression can complete the number of test cases that would require 3 person-days for manual testing within 30 minutes, significantly shortening the version release cycle. (2) Decision support for queuing parameter optimization: When the product team needs to adjust timeout parameters (such as changing the ringing timeout from 20s to 15s, or the freeze period from 10s to 5s), the test engine can quickly assess the impact of parameter changes on key indicators such as FIFO violation rate, customer churn rate, and average waiting time, and provide optimal parameter suggestions in a data-driven manner. (3) Continuous queuing health monitoring: Deploy this test device as a timed task (such as executing a smoke test kit every 30 minutes) to continuously detect the health status of the queuing system in a pre-production or production environment. When a sudden increase in FIFO violation rate or timeout accuracy drift is detected, an alarm is automatically triggered, realizing the upgrade of the test mode from "passively waiting for complaints" to "actively discovering defects".

[0107] The present invention adopts the above technical solution and has the following beneficial effects compared with the prior art:

[0108] (1) Through the scenario orchestration engine and the virtual terminal simulation layer, routing strategies, agent behaviors, timeout parameters, etc. can be automatically combined to generate exponential test scenarios, realizing accurate reproduction of multi-customer concurrency and abnormal timing interactions, completely solving the problems of low coverage and inability to simulate concurrent scenarios in manual testing, and improving regression testing efficiency by tens of times. (2) The FIFO integrity automatic verification algorithm based on the global event timeline can accurately detect the hidden fairness defects of "last-in, first-out" from the event sequence, and automatically trace back the frozen period state to distinguish the root cause, fundamentally ensuring the fairness verification of the queuing system. (3) This invention utilizes a state machine assertion framework dedicated to the frozen period, which can automatically complete multi-dimensional verification such as agent ringing stop, pause allocation, key recovery, timeout hang-up and time accuracy in a single execution, ensuring that the abnormal handling logic is completely complete. (4) By comparing the actual behavior with the industry standard reference model in multiple dimensions, subjective experience is transformed into objective indicators such as FIFO violation rate and active hang-up rate, providing data decision support for parameter optimization and product iteration, and supporting continuous monitoring of the production environment, thus realizing the upgrade from passive complaints to active discovery.

[0109] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A method and apparatus for automated testing of complex scenarios in a call center queuing system, characterized in that, Includes the following steps: Obtain the pre-configured test scenario description file of the business process based on the queuing system, and parse the test parameters in the test scenario description file to generate a test case matrix; Test parameters should include at least the virtual client parameters, virtual agent parameters, routing policy, ringing timeout, freeze period duration, and number of retries; The virtual client simulator and virtual agent simulator are created and driven based on the Session Initiation Protocol (SIP). The call interaction is executed according to the test case matrix, and the precise timestamp and event type of each call event are recorded. All call events are collected based on a global clock, and a global event timeline is constructed based on the timestamps of the call events. Automatic verification of FIFO integrity is performed on the global event timeline: Whenever a connection event is detected, it is checked whether there are other virtual customers with earlier queuing times who have not yet been connected; if so, a FIFO violation event is determined, and the virtual customers with earlier queuing times are checked back to see if they are in the freeze period at the current connection time, and a diagnostic conclusion containing the root cause of the violation is generated. Multi-dimensional automatic verification is performed on the freeze period triggered by no answer to the ringing: Based on the preset freeze period state machine model, assertion probes are injected into the state transition nodes of the freeze period to automatically verify the preset set of behaviors during the freeze period and verify the time deviation of the state transition; the preset set of behaviors includes the agent stopping ringing, the system pausing the allocation, resuming the allocation after receiving the customer's key, and automatically hanging up after the freeze period expires. The test case matrix is ​​executed, and the actual call event behavior of the queuing system under test is compared with the expected behavior defined by the preset reference model through the comparative analysis engine, generating a multi-dimensional quantitative difference report.

2. The automated testing method for complex scenarios of a call center queuing system according to claim 1, characterized in that, The implementation of automatic FIFO integrity verification on the global event timeline includes: Sort all events in the global event timeline in ascending order by timestamp; Maintain a set of unconnected customers sorted by their enqueue time; Iterate through the sorted events, and for each connection event, obtain the queuing time of the currently connected customer; If there is another customer in the set of unconnected customers whose queuing time is earlier than that of the currently connected customer, a FIFO violation event is determined to have occurred. The system will then backtrack to check whether the virtual customer with the earlier queuing time was in a frozen period at the time of the current connection and generate a diagnostic conclusion containing the root cause of the violation. Otherwise, the currently connected customer will be removed from the set of unconnected customers. The implementation of the diagnostic conclusion that includes the root cause violation includes: checking whether another customer with an earlier queuing time was in a frozen state when the connection event occurred; if so, outputting that the root cause is caused by the frozen state mechanism; if not, outputting that the root cause is an error in the routing policy execution.

3. The automated testing method for complex scenarios of a call center queuing system according to claim 1, characterized in that, Based on a pre-defined state machine model for the freeze period, assertion probes are injected into the state transition nodes during the freeze period to automatically verify a pre-defined set of behaviors within the freeze period, specifically including: Define the states of the frozen state machine model, including: frozen state, recovery state, and timeout disconnected state; When an RNA event is detected that triggers the start of freezing, the system enters a frozen state and triggers the first assertion probe to verify whether the agent has stopped ringing and whether the system has paused allocation. If a key signal sent by a customer is detected before the freeze timer expires, the system transitions from the frozen state to the recovery state and triggers the second assertion probe to verify whether the system has reassigned the agent and resumed ringing. When the freeze timer expires and no key signal is detected, the system transitions from the frozen state to the timeout hang-up state and triggers the third assertion probe to verify whether the hang-up command was issued and whether the reason code is correct. The implementation of verifying the time deviation of state transition is as follows: verify whether the deviation between the actual trigger duration of the freeze period timer and the pre-configured freeze period duration is within the preset threshold range.

4. The automated testing method for complex scenarios of a call center queuing system according to claim 1, characterized in that, The specific implementation of generating multi-dimensional quantitative difference reports includes: Simulations of the queuing system under test and the reference model are run in parallel under the same test scenario. The event timeline generated by the queuing system under test is normalized and time-aligned with the event timeline generated by the reference model. Automatically calculate and generate multi-dimensional quantitative difference indicators, which include at least the difference in the number of FIFO violations, the difference in the system's proactive hang-up rate, the difference in the average number of customer operations, the difference in the connection rate, and the difference in the quantile of waiting time.

5. The automated testing method for complex scenarios of a call center queuing system according to claim 1, characterized in that, The reference models include built-in industry-standard reference models that conform to the ITIL (Information Technology Infrastructure Library) specification, and / or custom-injected expected behavior models based on testing requirements.

6. The automated testing method for complex scenarios of a call center queuing system according to claim 1, characterized in that, The test scenario description file uses YAML or JSON format to define the test intent in a declarative manner; When generating the test case matrix, the test parameters are expanded by combining all factors to automatically generate a set of test cases covering multiple routing strategies, multiple agent behaviors, multiple client concurrency, and multiple exception handling paths.

7. The automated testing method for complex scenarios of a call center queuing system according to claim 1, characterized in that, The method also includes: Based on multi-dimensional quantitative difference reports, assess the impact of queuing parameter changes on system performance indicators and generate parameter tuning recommendations.

8. An automated testing device for complex scenarios of a call center queuing system, employing the automated testing method for complex scenarios of a call center queuing system as described in any one of claims 1 to 7, characterized in that, The device includes: The test scenario orchestration engine is used to acquire and parse pre-configured test scenario description files and generate a test case matrix from the test parameters; The virtual terminal simulation layer, including a virtual client simulator and a virtual agent simulator, is used to execute call interactions in the test case matrix based on the Session Initiation Protocol (SIP) and record the precise timestamp and event type of each call event generated. The timing verification engine collects all call events based on a global clock, constructs a global event timeline based on the timestamps of the call events, and performs automatic verification of FIFO integrity and multi-dimensional automatic verification of the freeze period. Automatic FIFO integrity verification includes checking for other virtual customers with earlier queuing times but not yet connected whenever a connection event is detected. If found, a FIFO violation is determined, and the system backtracks to check whether the customer was in the freeze period at the time of connection to output a diagnostic conclusion. Multi-dimensional automatic verification of the freeze period includes injecting assertion probes into the state transition nodes of the freeze period based on the freeze period state machine model to verify behaviors such as agent stopping ringing, system pausing allocation, resuming allocation after key press, and automatic hang-up after timeout, as well as the time deviation of state transitions. The comparative analysis engine is used to compare the actual call event behavior of the queuing system under test with the expected behavior defined by the preset reference model, and generate a multi-dimensional quantitative difference report.

9. The automated testing device for complex scenarios of a call center queuing system according to claim 8, characterized in that, When the timing verification engine performs automatic FIFO integrity verification, it is used for: Sort all events in ascending order of timestamp, and maintain a set of unconnected customers sorted by enqueue time; For each connection event, if there is another customer in the set of unconnected customers who joined the queue earlier, it is determined to be a FIFO violation, and further identification is made to determine whether the other customer is in a frozen period in order to output a diagnostic conclusion containing the root cause.

10. The automated testing device for complex scenarios of a call center queuing system according to claim 8, characterized in that, The comparative analysis engine is used for: In the same test scenario, the virtual terminal simulation layer is driven in parallel to perform simulations of the system under test and the reference model. After normalizing and aligning the event timelines generated by both, multi-dimensional quantitative difference indicators are calculated and generated. These quantitative difference indicators include at least one of the following: FIFO violation count difference, system-initiated hang-up rate difference, average number of customer operations difference, and connection rate difference.