Interactive digital human testing method

Abstract

The application relates to an interactive digital human test method, which comprises the following steps: assigning an associated identifier to a user request and constructing a request load to be sent to a server, receiving a response message with the associated identifier and returning to a corresponding message queue. According to whether the intermediate state response message of the server starting to call the digital human reply is received, the timeout threshold is dynamically switched: if not received, a shorter first timeout threshold is used, if received, a longer second timeout threshold is switched to, and the round of response collection is terminated when the end state response message is received or the timeout threshold is reached. The application can adapt to the response delay change in the digital human reply process, and improve the test efficiency and stability.

Description

Technical Field

[0001] This application relates to the field of digital human testing, and in particular to an interactive digital human testing method. Background Technology

[0002] Interactive digital humans are virtual interactive entities driven by artificial intelligence, typically possessing capabilities such as natural language understanding, multi-turn dialogue, tool or skill invocation, speech synthesis, and digital avatar-driven functions. After a user issues a request, the digital human server often performs processes such as intent understanding, tool invocation, knowledge retrieval, and speech synthesis sequentially or in parallel. By integrating the results of tool invocation and knowledge retrieval, the final result is returned to the user in natural language.

[0003] Existing technologies include various methods for testing the capabilities of digital humans. For example, methods can be used to test whether a digital human can accurately understand user intentions by evaluating whether the intent sequence recognized by a large model is the same as a pre-labeled intent path; methods can be used to test the task completion and response coherence of a digital human in multiple contexts by generating multiple rounds of user input through a user simulator; and methods can be used to test whether a digital human can correctly select and invoke tools by comparing tool invocation instructions generated by a large model with standard answers.

[0004] However, the above-mentioned solutions typically only test a single capability of the digital human, and currently lack a testing scheme for the complete interactive process of real-time digital human interaction, from sending requests, receiving packets one by one, and collecting multiple streaming responses. Therefore, it is necessary to provide an interactive digital human testing scheme that can adapt to the characteristics of real-time digital human interaction and completely collect and aggregate multiple streaming responses. Summary of the Invention

[0005] Therefore, it is necessary to provide an interactive digital human testing method, apparatus, computer equipment, storage medium, and computer program product to address the aforementioned technical problems.

[0006] Firstly, this application provides an interactive digital human testing method. The method includes:

[0007] Assign an association identifier to the user request to be sent, and construct a request payload based on the user request and the association identifier;

[0008] Send the request payload to the server;

[0009] Receive at least one response message sent by the server, and classify the response message into the message queue corresponding to the association identifier according to the association identifier carried by the response message;

[0010] During the process of collecting responses for a single request in this round, it is determined whether an intermediate response message has been received. The intermediate response message is used to indicate that the server has started to call the digital human to reply.

[0011] If the intermediate response message is not received, wait for the response message within the first timeout threshold. When the first timeout threshold is reached, terminate the current round of response collection and return to the message queue of the current round.

[0012] If the intermediate response message is received, the system switches to the second timeout threshold from the moment of receipt to continue waiting for a response message, wherein the second timeout threshold is greater than the first timeout threshold; and when an end-state response message indicating the end of the service is detected or the second timeout threshold is reached, the current round of response collection is terminated and the system returns to the message queue of the current round.

[0013] In one embodiment, the determination condition of the intermediate response message includes at least one of the following: the business step field is set to tool call start, tool call completion, speech synthesis segment, or quick text reply; or, the business status field indicates in progress and carries a non-empty tool call identifier or a non-empty speech synthesis text segment.

[0014] In one embodiment, the method further includes a multi-turn interaction step, the multi-turn interaction step comprising:

[0015] Maintain the list of constructed sessions, and perform the following steps for each round of user requests in sequence:

[0016] Append the user request of the current round to the end of the constructed session list;

[0017] Send a request payload carrying the constructed session list and collect all responses for this round;

[0018] Extract tool call identifiers, skill codes, and sequentially concatenated speech-synthesized text from the collected current round responses. Encapsulate the extraction results into a digital human extended track message and append it to the end of the constructed session list.

[0019] In one embodiment, the method further includes a connection scheduling step during the execution of batch test cases, wherein the test cases consist of at least one round of user requests; the request payload is sent to the server based on a WebSocket connection; the connection scheduling step includes:

[0020] Multiple test cases are executed sequentially. When adjacent test cases point to the same environment and the same WebSocket target address, the established WebSocket connection is reused.

[0021] When adjacent test cases point to different environments or different WebSocket target addresses, close the current connection and rebuild the connection for the next test case.

[0022] In one embodiment, the method further includes a large model judgment step, the large model judgment step comprising:

[0023] The reference text, the user question in the user request, and the test text extracted from each of the response messages in this round's message list are combined to form the referee's prompt;

[0024] Call the large model interface to obtain the judgment text returned by the large model;

[0025] The judgment text is normalized by labeling. The label normalization includes parsing the label of the final conclusion section in the judgment text. If the label of the final conclusion section does not exist, the last discrete label in the judgment text is parsed and the reason field corresponding to the discrete label is extracted.

[0026] The assertion is passed or failed based on the label after label normalization.

[0027] In one embodiment, the association identifier includes a primary association identifier and a secondary global identifier, wherein the secondary global identifier is suffixed with the primary association identifier; the step of assigning the response message to the message queue corresponding to the association identifier based on the association identifier carried in the response message includes:

[0028] When the main association identifier exists in the response message, the response message is assigned to the message queue corresponding to the request based on the main association identifier;

[0029] If the primary association identifier is not present in the response message, but the secondary global identifier is present, a candidate primary association identifier with the secondary global identifier as a suffix is ​​searched in the set of primary association identifiers that have been sent but have not yet completed the response, and the response message is assigned to the message queue corresponding to the candidate primary association identifier.

[0030] Secondly, this application also provides an interactive digital human testing device. The device includes:

[0031] The identifier allocation module is used to assign an associated identifier to the user request to be sent, and to construct the request payload based on the user request and the associated identifier;

[0032] The sending module is used to send the request payload to the server;

[0033] The message categorization module is used to receive at least one response message sent by the server, and categorize the response message into the message queue corresponding to the association identifier according to the association identifier carried by the response message;

[0034] The intermediate state judgment module is used to determine whether an intermediate state response message has been received during the process of collecting all responses in the current round corresponding to a single request. The intermediate state response message is used to indicate that the server has started to call the digital human to reply.

[0035] The first timeout waiting module is used to wait for a response message within a first timeout threshold if the intermediate response message is not received, and terminate the current round of response collection and return to the message queue of the current round when the first timeout threshold is reached.

[0036] The second timeout waiting module is used to switch to the second timeout threshold to continue waiting for the response message from the moment the intermediate response message is received, wherein the second timeout threshold is greater than the first timeout threshold; and when an end-state response message indicating the end of the service is detected or the second timeout threshold is reached, the current round of response collection is terminated and the message queue of the current round is returned.

[0037] In one embodiment, the determination condition of the intermediate response message includes at least one of the following: the business step field is set to tool call start, tool call completion, speech synthesis segment, or quick text reply; or, the business status field indicates in progress and carries a non-empty tool call identifier or a non-empty speech synthesis text segment.

[0038] In one embodiment, the device further includes a multi-turn interaction module for:

[0039] Maintain the list of constructed sessions, and perform the following steps for each round of user requests in sequence:

[0040] Append the user request of the current round to the end of the constructed session list;

[0041] Send a request payload carrying the constructed session list and collect all responses for this round;

[0042] Extract tool call identifiers, skill codes, and sequentially concatenated speech-synthesized text from the collected current round responses. Encapsulate the extraction results into a digital human extended track message and append it to the end of the constructed session list.

[0043] In one embodiment, the apparatus further includes a connection scheduling module for:

[0044] Multiple test cases are executed sequentially. When adjacent test cases point to the same environment and the same WebSocket target address, the established WebSocket connection is reused.

[0045] When adjacent test cases point to different environments or different WebSocket target addresses, close the current connection and rebuild the connection for the next test case.

[0046] In one embodiment, the device further includes a large model judgment statement module for:

[0047] The reference text, the user question in the user request, and the test text extracted from each of the response messages in this round's message list are combined to form the referee's prompt;

[0048] Call the large model interface to obtain the judgment text returned by the large model;

[0049] The judgment text is normalized by labeling. The label normalization includes parsing the label of the final conclusion section in the judgment text. If the label of the final conclusion section does not exist, the last discrete label in the judgment text is parsed and the reason field corresponding to the discrete label is extracted.

[0050] The assertion is passed or failed based on the label after label normalization.

[0051] In one embodiment, the association identifier includes a primary association identifier and a secondary global identifier, wherein the secondary global identifier is suffixed with the primary association identifier; the message merging module is further configured to:

[0052] When the main association identifier exists in the response message, the response message is assigned to the message queue corresponding to the request based on the main association identifier;

[0053] If the primary association identifier is not present in the response message, but the secondary global identifier is present, a candidate primary association identifier with the secondary global identifier as a suffix is ​​searched in the set of primary association identifiers that have been sent but have not yet completed the response, and the response message is assigned to the message queue corresponding to the candidate primary association identifier.

[0054] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any of the methods described above.

[0055] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements any of the above methods.

[0056] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements any of the above methods.

[0057] The interactive digital human testing method provided in this application assigns an association identifier to the user request to be sent, constructs and sends the request payload, receives the response messages sent by the server packet by packet and categorizes them into the corresponding message queue according to the association identifier. During the collection process, based on whether an intermediate response message indicating that the server has started to call the digital human to reply is received, if not received, it waits within a first timeout threshold; if received, it switches to a larger second timeout threshold to continue waiting, and terminates the current round of response collection with the end-state response message or timeout as the boundary. This achieves complete collection of multi-stream responses by the test client under real-time interactive conditions, making up for the shortcomings of existing solutions that only test partial capabilities such as intent understanding, multi-turn dialogue, or tool calls separately, without covering the complete "send request - receive packet by packet - collect and gather" link. At the same time, through phased timeout switching, it can quickly detect no-response anomalies with a shorter timeout before the server has started to reply, and also reserves sufficient processing time for the digital human in long-time-consuming links such as tool calls and knowledge retrieval, reducing the false alarm rate under a fixed single timeout strategy and improving the efficiency, stability, and interpretability of automated regression testing of interactive digital humans. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating an interactive digital human testing method in one embodiment;

[0059] Figure 2 This is a flowchart illustrating multiple rounds of interaction steps in one embodiment;

[0060] Figure 3 This is a schematic diagram of the connection scheduling steps in one embodiment;

[0061] Figure 4 This is a schematic diagram of the large model adjudication statement steps in one embodiment;

[0062] Figure 5 This is a schematic diagram of step 106 in one embodiment;

[0063] Figure 6 This is a structural block diagram of an interactive digital human testing device in one embodiment;

[0064] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] In one embodiment, such as Figure 1 As shown, an interactive digital human testing method is provided, including:

[0067] Step 102: Assign an association identifier to the user request to be sent, and construct a request payload based on the user request and the association identifier.

[0068] In this embodiment, the user request to be sent is constructed based on a single user input that needs to be sent to the digital human during testing. For example, the user request may include at least: the single user input and context information (if the context information exists). The association identifier is a string used to identify which user request the response message sent by the server corresponds to.

[0069] After the test client constructs the user request to be sent, it can assign an associated identifier to the user request. A typical implementation is a millisecond-level timestamp string. Then, the user request text and the associated identifier are encapsulated together into a JSON-formatted request payload.

[0070] Step 104: Send the request payload to the server.

[0071] In this embodiment, the server is the gateway server corresponding to the digital human under test. A WebSocket long connection is established between the test client and the gateway of the digital human under test, and the constructed JSON format request payload is sent to the server through this connection.

[0072] Step 106: Receive at least one response message sent by the server, and classify the response message into the message queue corresponding to the association identifier according to the association identifier carried by the response message.

[0073] In this embodiment, the response message is a multi-packet JSON response sent by the server; the message queue is a temporary container maintained by the test client with the association identifier as the key, used to store each response message belonging to the current round of requests.

[0074] When the test client establishes a WebSocket connection with the server, it registers a message receiving callback function. When the server sends a response message, the network layer assembles the received data into a WebSocket text frame and automatically calls the callback function, passing the text frame as a parameter. Once the callback function is triggered, it first extracts the payload data (i.e., a JSON-formatted string) from the text frame, and then calls a JSON parsing library to convert the string into a JSON object in memory. Next, the client reads the value of the association identifier returned by the server by accessing a specific property name of the JSON object (e.g., jsonObj["association identifier"] or jsonObj.requestId). Finally, the client uses this read string value as a key to look up the corresponding message queue, thus correctly placing the current response message into the message queue associated with the request.

[0075] Step 108: During the process of collecting all responses for a single request in this round, determine whether an intermediate response message has been received. The intermediate response message is used to indicate that the server has started to call the digital human to reply.

[0076] In this embodiment, the intermediate response message is the response message sent by the server when it begins to invoke the digital human to reply. The specific rules for judging the intermediate response message can be pre-configured by technical personnel. For example, setting the business step field of the response message to "speech synthesis segment" can be used to determine if the response message is an intermediate response message. During the process of collecting responses for a single request, the test client continuously checks whether it has received the intermediate response message.

[0077] Step 110: If the intermediate response message is not received, wait for the response message within the first timeout threshold. When the first timeout threshold is reached, terminate the current round of response collection and return to the message queue of the current round.

[0078] In this embodiment, the first timeout threshold is used to wait for the first intermediate response message, and its typical value ranges from 2 to 120 seconds. The first timeout threshold can be pre-configured by the user and read by the test client when the test starts.

[0079] The test client records the completion time of the request payload and calculates the first deadline as the sending time plus the first timeout threshold. If no intermediate response message has been received, the test client checks the message queue for non-empty status before reaching the first deadline. If a new message appears in the queue, the message processing flow is entered. If the queue is still empty and no intermediate response message has been received by the time the first deadline is reached, the current round of response collection is terminated, the currently collected message queue is returned, and a timeout exception of "No response triggered" is thrown.

[0080] Step 112: If the intermediate response message is received, switch to the second timeout threshold from the time of receipt to continue waiting for the response message, wherein the second timeout threshold is greater than the first timeout threshold; and when an end-state response message indicating the end of the service is detected or the second timeout threshold is reached, terminate the current round of response collection and return to the message queue of the current round.

[0081] In this embodiment, the second timeout threshold is used to continue waiting for the remaining response after receiving the intermediate response message. Its value is greater than the first timeout threshold, typically ranging from 20 to 300 seconds. The second timeout threshold can also be pre-configured by the user and read by the test client when starting the test. The end-state response message is used to indicate the end of the service. The specific judgment rules for the end-state response message can be pre-configured by technical personnel. For example, the business rule field value of "end" can be set to represent an end-state response message. Alternatively, the business status field can be set to indicate success or completion to represent an end-state response message.

[0082] If at least one intermediate response message is received during the waiting process, the test client switches the timeout control to the second timeout threshold from the moment the first intermediate response message is received. The test client records the moment the intermediate response message is received, calculates the second cutoff time as that moment plus the second timeout threshold, and then continues to wait using the second cutoff time as the new valid cutoff time, consuming subsequent response messages from the message queue. Whenever a closing response message indicating the end of the service is detected, the current round of response collection is terminated, and the complete message queue including the closing response message is returned. If no closing response message is received by the second cutoff time, the collection is terminated and a timeout exception of "failed to collect closing packet after triggering start" is thrown.

[0083] The message queue contains all response messages returned by the digital human to the user during this round of testing (e.g., natural language replies, synthesized speech replies, tool call results, etc.). After obtaining the complete message queue, the user can compare the response messages in the message queue with the expected results to obtain the test results of the digital human.

[0084] In one embodiment, the determination criteria for intermediate response messages are further defined.

[0085] In this embodiment, the determination conditions for the intermediate response message include at least one of the following: the business step field is set to tool call start, tool call completion, speech synthesis segment, or quick text reply; or, the business status field indicates in progress and carries a non-empty tool call identifier or a non-empty speech synthesis text segment.

[0086] After receiving a response message, the test client checks several business fields of the message to determine whether it is an intermediate response message. The specific criteria include the following two categories; if either category is met, the message is considered an intermediate response message.

[0087] The first type of judgment criterion is based on the business step field. The test client reads the "step" field or its semantically equivalent field (such as "action," "stage," etc.; the field names can be adjusted according to the protocol or user configuration during implementation) from the response JSON and matches the value of this field against a predefined set of intermediate state values. This set includes, but is not limited to, any user-defined intermediate state value such as "tool call start," "tool call complete," "speech synthesis segment," and "quick text reply." The matching is case-insensitive. For example, if the server sends a JSON message containing "step": "tts_segment," the client determines that the message is an intermediate state response message.

[0088] The second type of judgment condition is based on a combination of the business status field and a non-empty auxiliary field. The test client reads the "status" field or its equivalent field from the response JSON and checks whether the value of this field indicates a "processing" status (e.g., "processing", "running", or the numeric code 1). Simultaneously, the client checks whether the message carries a non-empty tool call identifier (e.g., the "tool_call_id" field) or a non-empty speech-synthesized text fragment (e.g., the "text" or "content" field). If both conditions are met—that is, the business status is in progress and a non-empty tool call identifier or a non-empty speech-synthesized text fragment exists—then the message is also determined to be an intermediate response message.

[0089] For example, if the server sends a JSON message containing "status": "processing" and "tool_call_id": "call_12345", the client will interpret it as an intermediate response message. Similarly, a message containing "status": "running" and "text": "Hello, how can I help you?" will also be interpreted as an intermediate response message.

[0090] By introducing an intermediate response message determination mechanism and a phased switching strategy between the first and second timeout thresholds, this embodiment can accurately distinguish between the two states of "not yet started replying" and "started but not completed replying" on the server side. In the first stage, a smaller first timeout threshold can quickly detect server-side non-response anomalies. After detecting an intermediate state, it automatically switches to a larger second timeout threshold, reserving sufficient processing time for time-consuming steps such as tool invocation and knowledge retrieval, thereby significantly reducing the false positive rate without sacrificing testing efficiency. Furthermore, this method uses the end-state response message as a boundary marker for the collection of responses in this round, ensuring that the response message set of each round of dialogue is complete and ordered. This provides a reliable data foundation for subsequent verification of whether the digital human's responses meet expectations, improving the stability and interpretability of automated regression testing.

[0091] In one embodiment, such as Figure 2 As shown, this embodiment further includes multi-turn interaction steps to support testing of complex dialogue scenarios involving multiple user turns.

[0092] Step 202: Maintain the list of constructed sessions.

[0093] In this embodiment, the constructed session list is an ordered collection of messages representing all interactions that have occurred in the current dialogue context, including request messages from the user role and response messages from the digital human / assistant role. The test client maintains a constructed session list, and test cases can be configured with multiple user rounds, each round containing one user text message. The test client sequentially processes the user requests from each round.

[0094] Step 204: Append the user request of the current round to the end of the constructed session list.

[0095] In this embodiment, the user request for the current round is the user input text corresponding to the currently being processed round. For the current round, the test client first encapsulates the user text of that round into a user role message and appends it to the end of the constructed session list. The user role message contains at least two fields: "role": "user" and "content": user input text.

[0096] Step 206: Send a request payload carrying the constructed session list and collect all responses for this round.

[0097] In this embodiment, the request payload is a JSON structure containing a list of constructed sessions as the dialogue history; collecting all responses in this round refers to waiting for and collecting all response messages sent by the server using the two-stage timeout mechanism described in the previous embodiment.

[0098] The test client checks the current WebSocket connection status. If it detects that the socket has been disconnected, it triggers a reconnection process (closing the existing connection, waiting for a configurable reconnection interval of 1 to 3 seconds, attempting to rebuild the connection 3 times according to the maximum reconnection count; if the reconnection fails, it throws an exception and terminates the test). Then, it encapsulates the constructed session list as the dialogue history along with the user request of the current round into a request payload. Specifically, the JSON structure of the request payload contains a "messages" array, the elements of which are the messages in the constructed session list. After sending the request payload, the client enters the response collection process for this round, calling the two-stage timeout mechanism to collect all response messages belonging to the current round of requests, and obtains the message list for this round.

[0099] Step 208: Extract tool call identifiers, skill codes, and sequentially concatenated speech-synthesized text from the collected current round responses, encapsulate the extraction results into a digital human extended track message, and append it to the end of the constructed session list.

[0100] In this embodiment, the tool call identifier is used to uniquely identify a tool call; the skill code is used to represent the type of skill called by the digital human; the speech-synthesized text is the text content that needs to be read aloud in the digital human's reply; and the digital human extension track message is a structured message containing digital human extension information, which is added to the conversation list to maintain the context.

[0101] After collecting all the information, the test client extracts the following three types of information from the current message list: the first type is the tool call identifier, which is the first non-empty tool call identifier field (e.g., "tool_call_id") in the current message list; the second type is the skill code, which is the "skill_code" field or its equivalent field from the top-level or nested business object; and the third type is the speech-synthesized text, which is extracted from the "tts_text" field or its equivalent field in each message in the current message list. These text fragments are then concatenated in the order the messages arrive to form a complete speech-synthesized text. The extracted results are then packaged into a digital human extended track message. This message contains at least "role":"assistant" or "role":"digital_human" and an extended object field. This extended object records the tool call identifier, skill code, and the concatenated complete speech-synthesized text. Optionally, the extended object also contains a skill list sub-object to record all skill call information involved in this round. Finally, this digital human extended track message is appended to the end of the constructed session list.

[0102] The following example illustrates this. Suppose that in the first round of requests, the user's request content is "How's the weather?". After the test client sends this request, it appends it to the end of the constructed session list. At this point, the constructed session list is: [{"role": "user", "content": "How's the weather?"}).

[0103] The digital human's reply, in its speech-synthesized text, reads: "Today's weather is sunny, with temperatures ranging from 21 to 34 degrees Celsius, and a southeast wind of level 3." The skill code for the weather query is "weather_query," and the tool call identifier for this tool call is "call_123." Therefore, after the first round of responses are collected, the constructed digital human extended track message is: {"role": "assistant", "content": "Today's weather is sunny, with temperatures ranging from 21 to 34 degrees Celsius, and a southeast wind of level 3", "skill_code": "weather_query", "tool_call_id": "call_123"}.

[0104] Then, this message is appended to the end of the constructed session list, which is now: [{"role": "user", "content": "How's the weather?"},{"role": "assistant", "content": "Today's weather is sunny, temperature 21 to 34 degrees Celsius, southeast wind level 3", "skill_code": "weather_query", "tool_call_id":"call_123"}]. In this way, the constructed session list can carry context information from all previous rounds in the next test round, allowing for multi-round testing.

[0105] Through this embodiment, testers can obtain a context consistent with the real link in multiple rounds of regression testing without manually constructing intermediate state fields for the digital human, significantly reducing the risk of errors in manual assembly.

[0106] In one embodiment, this embodiment further includes a connection scheduling step during the execution of batch test cases.

[0107] Step 302: Execute multiple test cases sequentially. When adjacent test cases point to the same environment and the same WebSocket target address, reuse the established WebSocket connection.

[0108] In this embodiment of the application, the test case is a test unit consisting of at least one round of user requests; the environment refers to the environment in which the digital human service under test is located (e.g., test environment, pre-release environment, production environment); the WebSocket target address is the target URL of the WebSocket connection (including protocol, host, port, and path).

[0109] The test client executes multiple test cases sequentially. Before execution begins, it reads the batch scheduling configuration (including whether connection reuse is enabled by default and the typical value range of 0 to 2 seconds for the interval between adjacent test cases). When executing the first test case, it queries the corresponding WebSocket gateway address based on the environment identifier specified in the test case, establishes a new WebSocket connection, and executes each round of interaction in the test case. After the test case is completed, it records the environment identifier and WebSocket target address used by the current connection but does not immediately close the connection. Before starting to execute the next test case, it compares the environment identifier and WebSocket target address of the current test case with those of the previous test case (the comparison rule is that the environment identifier string is completely equal and the WebSocket URL string is completely equal). If both are equal, it is determined that they point to the same environment and the same target address. At this time, the established WebSocket connection is reused. When reusing the connection, the connection state is not cleared, but the message queue and associated identifier mapping table belonging to the previous test case are reset to avoid data interference between test cases. If the connection is abnormally disconnected during the execution of the previous test case, the reuse logic fails and the reconstruction process is executed instead.

[0110] Step 304: When adjacent test cases point to different environments or different WebSocket target addresses, close the current connection and rebuild the connection for the next test case.

[0111] In this embodiment of the application, different environments refer to different environment identifier strings; different WebSocket target addresses refer to different WebSocket URL strings (including different protocols, hosts, ports, and paths).

[0112] If the environment identifier or WebSocket target address of the current test case is different from that of the previous test case, the test client will first actively close the established WebSocket connection (if it is still open) and release the related resources. Then, it will establish a new WebSocket connection for the current test case. The process of establishing a new connection is the same as that of the first test case. Between two adjacent test cases, a wait is performed according to the configured interval between adjacent test cases (for example, if the configured interval is 0.5 seconds, the client will call the thread to sleep for 0.5 seconds after completing the closing work of the previous test case, and then start the parameter loading and connection scheduling judgment of the next test case). This interval mechanism is used to reduce the probability of rate limiting triggering caused by continuous sending of requests to the gateway under test.

[0113] The connection scheduling method in this embodiment greatly reduces unnecessary WebSocket handshake operations in batch regression testing, which reduces the pressure on the gateway and avoids false test failures caused by connection rate limiting triggered by frequent handshakes.

[0114] In one embodiment, this embodiment further includes a large model judgment statement step for assisting in the determination of the quality of the digital human's response.

[0115] Step 402: Combine the reference text, the user question in the user request, and the test text extracted from each of the response messages in the current round message list to form the referee's prompt.

[0116] In this embodiment, the reference text is a pre-set benchmark text for comparison, which may be a manually annotated standard answer or a historical reply captured from the previous stable version server; the user question is the request text sent by the user in the current test case; the text to be tested is the text to be evaluated extracted and concatenated from the response message of this round; the referee prompt is the prompt word sent to the large model for judgment after concatenation.

[0117] After the test is completed and the current round of messages is collected, the text to be tested is extracted from each response message in the current round of messages. The extraction method is to traverse the current round of messages and extract the text content fields (such as "content", "text", "message", etc.) from each message in sequence. The extracted text fragments are then concatenated according to the order in which the messages arrived to form the complete text to be tested. Then, a referee's prompt is constructed. This prompt should contain at least the following three parts: reference text, the user question in the user request, and the extracted text to be tested. The specific content of the prompt can be set by those skilled in the art according to actual needs. In one example, the prompt can be set as follows:

[0118] "You are a test result evaluation expert. Please determine whether the text to be tested is qualified based on the following information."

[0119] User question: {user_question}

[0120] Reference text (correct answer): {reference_text}

[0121] Test text (digital human's reply): {candidate_text}

[0122] Please output in the following format:

[0123] Reason: <State the reason for the judgment here>

[0124] Final conclusion: <Output only one word here: pass or fail>

[0125] The "Final Conclusion" section is the final conclusion segment, and the "Pass / Fail" sections that follow are discrete labels.

[0126] Step 404: Call the large model interface to obtain the judge's text returned by the large model.

[0127] In this embodiment, the large model interface is a language model service interface for external calls, typically an OpenAI-compatible chat completion interface; the judge's text is the natural language text returned by the large model, which may contain the reasoning process, restatement content, and final conclusion.

[0128] The test client calls the large model interface via HTTP, sets a reasonable timeout (typically 10 to 60 seconds), and passes the above-mentioned referee prompt as a user message during the call. The large model returns the referee text.

[0129] Step 406: Perform label normalization on the judgment text. The label normalization includes: parsing the labels of the final conclusion section in the judgment text; if the labels of the final conclusion section do not exist, then parse the last discrete label that appears in the judgment text and extract the reason field corresponding to the discrete label.

[0130] In this embodiment, label normalization refers to uniformly mapping the conclusion statements in the text returned by the large model to specific values ​​predefined internally (e.g., "pass" or "fail"); the final conclusion section is the conclusive text placed between special tags by the large model according to convention; discrete labels are a predefined set of conclusion keywords (e.g., "pass", "fail", "pass", "fail"); and the reason field is the text describing the basis for the judgment.

[0131] The test client performs label normalization on the returned judge's text. The first step in the normalization process is to attempt to parse the labels of the final conclusion section. The judge's prompts can instruct the large model to place conclusive text after a predefined special marker, such as "CONCLUSION" or "final conclusion." The client uses regular expressions or string search methods to extract the content following this marker as the label for the final conclusion section. If this section contains any string from a predefined set of discrete labels, it is mapped to a unified internal enumeration value.

[0132] Because the output of the large language model is uncontrollable, the judgment text may not contain the aforementioned agreed-upon final conclusion segment marker. If the judgment text does not contain the agreed-upon final conclusion segment marker, an alternative parsing strategy is adopted, which involves scanning backwards from the end of the judgment text to find the last occurrence of a phrase belonging to the predefined discrete label set as the conclusion output by the large model. Specifically, the client divides the judgment text into multiple lines by newline characters, and checks each line from the last line for discrete label keywords. If no such keyword is found, the entire text is scanned character by character in reverse order, and the last matched discrete label is taken as the normalized result. At the same time, the reason field is extracted from the judgment text. If the final conclusion segment marker exists, the descriptive text outside the marked area and immediately adjacent to the conclusion segment is extracted as the reason. If the final conclusion segment marker does not exist, all text after the last discrete label in the judgment text is taken, or the remaining part of the entire judgment text after removing the discrete labels is taken as the reason field.

[0133] The reason for performing a reverse scan is that the output of a large language model may contain reasoning processes (e.g., the test text lacks a gate, so it may not pass... but the core fact is correct, and the final conclusion is that it passes. Reason: the flight status is consistent with the reference answer). Therefore, a phrase belonging to a predefined discrete label set is more likely to be the final conclusion in the final output of the large language model.

[0134] Step 408: Determine whether the assertion passes or fails based on the label after label normalization.

[0135] In this embodiment, asserting pass or fail refers to determining whether a test case passes validation based on the normalized labels. The normalized labels are mapped to Boolean values ​​(e.g., "pass" is mapped to True, "fail" is mapped to False), and logically ANDed with the results of the structured assertions to form the final judgment status of the test case. The test case report simultaneously records the discrete labels and the extracted reason text.

[0136] With the large model judgment statement in this embodiment, even if the model output contains a lengthy reasoning process, it can stably extract enumerable judgment conclusions, thereby improving the reliability of automated testing.

[0137] In one embodiment, this embodiment further defines the method for allocating associated identifiers and bucketing response messages.

[0138] Step 502, the association identifier includes a main association identifier and an auxiliary global identifier, wherein the auxiliary global identifier is suffixed with the main association identifier.

[0139] In this embodiment, the association identifier is an identifier assigned by the test client to each user request to be sent, used to establish a correspondence between the request and each response message subsequently sent by the server; the main association identifier is a unique string within the current test process, typically constructed as a millisecond-level timestamp string such as "1699123456789"; the auxiliary global identifier is a longer string, constructed by concatenating the device prefix string with the main association identifier. For example, if the device prefix string is "device_abc_", then the auxiliary global identifier is "device_abc_1699123456789", therefore the auxiliary global identifier has the main association identifier as a suffix.

[0140] The test client assigns a pair of associated identifiers to each user request to be sent, namely a primary associated identifier and a secondary global identifier. When sending the request payload, the primary associated identifier is written to the "track_id" field in the JSON payload, and the secondary global identifier is written to the "trace_id" field. At the same time, a "set of primary associated identifiers to be sent but not yet completed" is maintained in memory, and the primary associated identifier of the request is added to this set.

[0141] Step 504: When the main association identifier exists in the response message, the response message is assigned to the message queue corresponding to the request based on the main association identifier.

[0142] In this embodiment, the primary association identifier is the value of the "track_id" field carried in the response message. When the test client receives the response message sent by the server via WebSocket, it first parses the JSON object of the response message and attempts to read the "track_id" field. If the field exists and is a non-empty string, it directly uses this string as the key to append this response message to the end of the message queue corresponding to the primary association identifier.

[0143] Step 506: If the primary association identifier does not exist in the response message, but the secondary global identifier does exist, search for a candidate primary association identifier with the secondary global identifier as a suffix in the set of primary association identifiers that have been sent but have not completed the response, and classify the response message into the message queue corresponding to the candidate primary association identifier.

[0144] In this embodiment, the auxiliary global identifier is the value of the "trace_id" field carried in the response message; the set of main associated identifiers that have been sent but not yet completed is a set maintained in the memory of the test client that records all main associated identifiers that have been sent but have not yet received responses. Specifically, if the "track_id" field is absent or empty in the response message, the "trace_id" field is read. If the "trace_id" field exists and is not empty, the test client searches for candidate main associated identifiers in the "set of main associated identifiers to be sent but not yet completed". The search rule is to check whether the auxiliary global identifier in the response message has that main associated identifier as a suffix for each main associated identifier in the set, i.e., to determine whether the end of the auxiliary global identifier is exactly equal to the main associated identifier. For example, if the "trace_id" in the response message is "device_abc_1699123456789" and the set contains the main associated identifier "1699123456789", the string "..." will be considered as a suffix. If "device_abc_1699123456789" ends with "1699123456789", a successful match is determined. After finding a matching candidate primary association identifier, the response message is added to the message queue corresponding to that identifier. If multiple candidate primary association identifiers satisfy the suffix matching relationship, the most recently sent primary association identifier that has not yet been fully collected is taken as the matching result. Specifically, the test client records the timestamp of the sending time for each request to be sent, and selects the latest sending time when multiple candidates match. If the response message contains neither a primary association identifier nor an auxiliary global identifier, or if the auxiliary global identifier cannot match any primary association identifier to be sent, the response message is recorded in the exception log, and the message is discarded or the test fails depending on the configuration.

[0145] Through the aforementioned phased timeout control, this embodiment can accurately distinguish between two fault types: "no response from the server" and "long-tail business not returning the end packet," thereby reducing the false alarm rate in regression testing.

[0146] The following practical example illustrates the process of this application. In one embodiment, this application uses an interactive digital human in an airport scenario as the test object to perform a single interaction test on the question, "Can I board flight 1234 now?". A WebSocket long connection has been established between the test client and the gateway of the digital human under test, with a first timeout threshold of 15 seconds and a second timeout threshold of 60 seconds.

[0147] The test client first reads the user input text to be tested and assigns a primary associated identifier to the request: a millisecond-level timestamp string "1780125872209". This primary identifier is then concatenated with the device prefix string "device_abc_" to obtain the secondary global identifier "device_abc_1780125872209". The test client encapsulates the user input text, the dialogue context, descriptions of various tools that can be called by the large model (e.g., a flight query tool described as "for querying flight status information", a location query tool described as "for querying locations within the terminal", etc.), and the aforementioned associated identifier into a JSON-formatted request payload. After encapsulation, the test client sends the request payload via a WebSocket connection and records the primary associated identifier in the set of primary associated identifiers that have been sent but have not yet received a response. Simultaneously, it maintains a temporary mapping locally with the primary associated identifier as the key and an empty message queue as the value.

[0148] The test client registers a message receiving callback. When the server sends responses packet by packet, it parses the JSON and categorizes them into the corresponding message queues based on their association identifiers. In this embodiment, it is assumed that a total of 10 responses are received. The first response only carries an auxiliary association identifier and does not carry a primary association identifier. The message categorization module does not find the primary association identifier in the response. After reading the auxiliary association identifier, it searches for a candidate primary association identifier with a matching suffix in the set of primary association identifiers that have been sent but not yet completed. It finds "1780125872209" and appends the response to the message queue corresponding to this primary association identifier. Responses 2 through 10 all carry both the primary and auxiliary association identifiers. The test client directly appends these responses to the same message queue sequentially using the primary association identifier as the key.

[0149] During this round of response collection, the test client retrieves responses one by one from the message queue and determines whether an intermediate response message has been received. The intermediate state determination rule is as follows: the business step field value is tool call start, tool call completion, speech-synthesized segment, or quick text reply, or the business status field indicates in progress and simultaneously carries a non-empty tool call identifier or a non-empty speech-synthesized text segment. In this embodiment,

[0150] Assume the first response's business step field is "llm-begin" (i.e., large language model startup) and the business status field does not indicate in progress, while the second response's business step field is "fast-reply" (i.e., fast text reply). According to this rule, the first response is not an intermediate response message; the second response is.

[0151] During the initial packet waiting phase, the test client polls the message queue with a first timeout threshold of 15 seconds as the upper limit. If no intermediate response message is received, the current round of response collection is terminated and a "no triggering of started response" timeout exception is returned when the first timeout threshold is reached. In this embodiment, after the second response arrives, the client switches the timeout control to the second timeout threshold of 60 seconds from the moment it receives the intermediate response message, and recalculates the second deadline.

[0152] Within the second timeout threshold, the test client continues to consume subsequent responses from the message queue. Assume the third response carries a tool invocation identifier, indicating that the digital human has begun invoking the flight query skill; the fourth response returns the structured query results for flight 1234, including departure city "City A", destination city "City B", flight status "Check-in in progress", gate "65", and check-in counters "K3-K15, L2-L15". Subsequently, the server continuously sends five business step fields that are synthesized speech snippets, with the text concatenated in arrival order as: "Flight 1234 is currently in 'Check-in in progress' and has not yet started boarding. The gate is 65. We suggest you proceed to counter K3-K15 or L2-L15 to complete the check-in process and pay attention to subsequent boarding announcements." The tenth response has the business step field "end", indicating the end of the service. The test client terminates this round of response collection and returns a complete message queue containing all 10 responses.

[0153] The test client selects response messages from the collected message queue whose business step field is a speech-synthesized segment as the business result for this round. These response messages are used as the test text. The user's question, "Can I board flight 1234 now?", and a reference text (e.g., "Flight 1234 is currently 'Check-in in progress'") are input into a predefined large language model. The large language model outputs the judgment text: "Final conclusion: Pass, Reason: Covers all key points and is consistent." Therefore, this test case is judged to have passed.

[0154] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0155] Based on the same inventive concept, this application also provides an interactive digital human testing apparatus for implementing the interactive digital human testing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the interactive digital human testing apparatus provided below can be found in the limitations of the interactive digital human testing method described above, and will not be repeated here.

[0156] In one embodiment, such as Figure 6 As shown, an interactive digital human testing device is provided, comprising: an identifier allocation module 602, a sending module 604, a message retrieval module 606, an intermediate state judgment module 608, a first timeout waiting module 610, and a second timeout waiting module 612, wherein:

[0157] The identifier allocation module 602 is used to allocate an associated identifier to the user request to be sent, and to construct a request payload based on the user request and the associated identifier;

[0158] Sending module 604 is used to send the request payload to the server;

[0159] The message categorization module 606 is used to receive at least one response message sent by the server, and categorize the response message into the message queue corresponding to the association identifier according to the association identifier carried by the response message.

[0160] The intermediate state judgment module 608 is used to determine whether an intermediate state response message has been received during the process of collecting the current round of responses corresponding to a single request. The intermediate state response message is used to indicate that the server has started to call the digital human to reply.

[0161] The first timeout waiting module 610 is used to wait for a response message within a first timeout threshold if the intermediate response message is not received, and terminate the current round of response collection and return to the message queue of the current round when the first timeout threshold is reached.

[0162] The second timeout waiting module 612 is used to switch to the second timeout threshold to continue waiting for the response message from the moment the intermediate response message is received, wherein the second timeout threshold is greater than the first timeout threshold; and when an end-state response message indicating the end of the service is detected or the second timeout threshold is reached, the current round of response collection is terminated and the message queue of the current round is returned.

[0163] In one embodiment, the determination condition of the intermediate response message includes at least one of the following: the business step field is set to tool call start, tool call completion, speech synthesis segment, or quick text reply; or, the business status field indicates in progress and carries a non-empty tool call identifier or a non-empty speech synthesis text segment.

[0164] In one embodiment, the device further includes a multi-round interaction module 614, configured to: maintain a constructed session list and sequentially perform the following steps on each round of user requests: append the current round of user requests to the end of the constructed session list; send a request payload carrying the constructed session list and collect the responses of the current round; extract tool call identifiers, skill codes, and sequentially concatenated speech-synthesized text from the collected responses of the current round, encapsulate the extraction results into a digital human extended track message, and append it to the end of the constructed session list.

[0165] In one embodiment, the apparatus further includes a connection scheduling module 616, configured to: sequentially execute multiple test cases; when adjacent test cases point to the same environment and the same WebSocket target address, reuse an established WebSocket connection; when adjacent test cases point to different environments or different WebSocket target addresses, close the current connection and rebuild the connection for the next test case.

[0166] In one embodiment, the apparatus further includes a large model adjudication module 618, configured to: concatenate the reference text, the user question in the user request, and the test text extracted from each of the response messages in the current round message list into a referee prompt; call the large model interface to obtain the adjudication text returned by the large model; perform label normalization on the adjudication text, wherein the label normalization includes: parsing the label of the final conclusion segment in the adjudication text, and if the label of the final conclusion segment does not exist, parsing the last discrete label in the adjudication text and extracting the reason field corresponding to the discrete label; and determining whether the assertion passes or fails based on the label after label normalization.

[0167] In one embodiment, the association identifier includes a primary association identifier and an auxiliary global identifier, wherein the auxiliary global identifier is suffixed with the primary association identifier; the message categorization module 606 is further configured to: when the primary association identifier exists in the response message, categorize the response message into the message queue corresponding to the request based on the primary association identifier; when the primary association identifier does not exist in the response message, but the auxiliary global identifier exists, search for a candidate primary association identifier with the auxiliary global identifier as a suffix in the set of primary association identifiers that have been sent but have not completed the response, and categorize the response message into the message queue corresponding to the candidate primary association identifier.

[0168] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an interactive digital human testing method.

[0169] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0170] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0171] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0172] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0174] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An interactive digital human testing method, characterized in that, include: Assign an association identifier to the user request to be sent, and construct a request payload based on the user request and the association identifier; Send the request payload to the server; Receive at least one response message sent by the server, and classify the response message into the message queue corresponding to the association identifier according to the association identifier carried by the response message; During the process of collecting responses for a single request in this round, it is determined whether an intermediate response message has been received. The intermediate response message is used to indicate that the server has started to call the digital human to reply. If the intermediate response message is not received, wait for the response message within the first timeout threshold. When the first timeout threshold is reached, terminate the current round of response collection and return to the message queue of the current round. If the intermediate response message is received, the system switches to the second timeout threshold from the moment of receipt to continue waiting for a response message, wherein the second timeout threshold is greater than the first timeout threshold; and when an end-state response message indicating the end of the service is detected or the second timeout threshold is reached, the current round of response collection is terminated and the system returns to the message queue of the current round.

2. The method according to claim 1, characterized in that, The determination conditions for the intermediate response message include at least one of the following: the business step field is set to tool call start, tool call completion, speech synthesis segment, or quick text reply; or, the business status field indicates in progress and carries a non-empty tool call identifier or a non-empty speech synthesis text segment.

3. The method according to claim 1, characterized in that, The method further includes a multi-round interaction step, which includes: Maintain the list of constructed sessions, and perform the following steps for each round of user requests in sequence: Append the user request of the current round to the end of the constructed session list; Send a request payload carrying the constructed session list and collect all responses for this round; Extract tool call identifiers, skill codes, and sequentially concatenated speech-synthesized text from the collected current round responses. Encapsulate the extraction results into a digital human extended track message and append it to the end of the constructed session list.

4. The method according to claim 1, characterized in that, The method further includes a connection scheduling step during the execution of batch test cases, wherein the test cases consist of at least one round of user requests; The request payload is sent to the server via a WebSocket connection; The connection scheduling steps include: Multiple test cases are executed sequentially. When adjacent test cases point to the same environment and the same WebSocket target address, the established WebSocket connection is reused. When adjacent test cases point to different environments or different WebSocket target addresses, close the current connection and rebuild the connection for the next test case.

5. The method according to claim 1, characterized in that, The method further includes a large model judgment statement step, which includes: The reference text, the user question in the user request, and the test text extracted from each of the response messages in this round's message list are combined to form the referee's prompt; Call the large model interface to obtain the judgment text returned by the large model; The judgment text is normalized by labeling. The label normalization includes parsing the label of the final conclusion section in the judgment text. If the label of the final conclusion section does not exist, the last discrete label in the judgment text is parsed and the reason field corresponding to the discrete label is extracted. The assertion is passed or failed based on the label after label normalization.

6. The method according to claim 1, characterized in that, The association identifier includes a primary association identifier and a secondary global identifier, wherein the secondary global identifier is suffixed with the primary association identifier; the step of assigning the response message to the message queue corresponding to the association identifier based on the association identifier carried in the response message includes: When the main association identifier exists in the response message, the response message is assigned to the message queue corresponding to the request based on the main association identifier; If the primary association identifier is not present in the response message, but the secondary global identifier is present, a candidate primary association identifier with the secondary global identifier as a suffix is ​​searched in the set of primary association identifiers that have been sent but have not yet completed the response, and the response message is assigned to the message queue corresponding to the candidate primary association identifier.

7. An interactive digital human testing device, characterized in that, include: The identifier allocation module is used to assign an associated identifier to the user request to be sent, and to construct the request payload based on the user request and the associated identifier; The sending module is used to send the request payload to the server; The message categorization module is used to receive at least one response message sent by the server, and categorize the response message into the message queue corresponding to the association identifier according to the association identifier carried by the response message; The intermediate state judgment module is used to determine whether an intermediate state response message has been received during the process of collecting all responses in the current round corresponding to a single request. The intermediate state response message is used to indicate that the server has started to call the digital human to reply. The first timeout waiting module is used to wait for a response message within a first timeout threshold if the intermediate response message is not received, and terminate the current round of response collection and return to the message queue of the current round when the first timeout threshold is reached. The second timeout waiting module is used to switch to the second timeout threshold to continue waiting for the response message from the moment the intermediate response message is received, wherein the second timeout threshold is greater than the first timeout threshold; and when an end-state response message indicating the end of the service is detected or the second timeout threshold is reached, the current round of response collection is terminated and the message queue of the current round is returned.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images