Automated test anomaly detection and recovery method, electronic device, storage medium

Abstract

An embodiment of the present disclosure provides an automated test exception detection and recovery method, an electronic device and a storage medium, comprising: embedding a unique identifier corresponding to a test task in the test task, and controlling the execution of the test task to make the program under test output log messages carrying the unique identifier; establishing a communication connection with the program under test, receiving the log messages, and according to the unique identifier, shunting the log messages to an independent log buffer corresponding to the test task to obtain a dedicated log sequence corresponding to the target test task; within a preset time window, extracting at least two types of time sequence features from the dedicated log sequence, the time sequence features including log interval metrics and log content diversity metrics; based on the time sequence features, the detected target process state and the log message output state of the dedicated log sequence, making an exception determination on the running state of the target test task; and according to the exception determination result, performing a differentiated recovery on the target test task that is suitable for the exception type of the target test task.

Description

Technical Field

[0001] This disclosure relates to the field of software automated testing, and more particularly to automated testing anomaly detection and recovery methods, electronic devices, and storage media. Background Technology

[0002] In the field of software automation testing, using test scripts to drive the tested object to perform preset operations and collect its output logs is the basic method for detecting program runtime anomalies. As the scale of testing expands and the number of concurrent tasks increases, a large amount of runtime logs are aggregated to the server for processing via a unified transmission channel. This leads to high bandwidth consumption on communication links, a significant increase in server-side parsing and storage load, and difficulty in quickly associating anomaly information with specific task contexts. Existing anomaly detection methods mostly rely on manual screening or fixed rule matching, making it difficult to accurately quantify and assess the dynamic changes in program runtime status, resulting in a high false positive rate. Frequent false triggers generate a large number of redundant restart commands, not only exacerbating server-side scheduling overhead but also causing ineffective consumption of the tested device and server computing resources. When an anomaly is detected, existing solutions typically employ a uniform global restart or rely on manual investigation, failing to adaptively handle the actual severity of different anomalies. This results in long test interruption times, low overall system resource utilization, and difficulty in meeting the real-time self-healing requirements of long-term unattended testing scenarios. Summary of the Invention

[0003] This disclosure provides an automated testing anomaly detection and recovery method, apparatus, electronic device, and computer-readable storage medium to at least partially solve the aforementioned problems existing in the related art.

[0004] According to one aspect of this disclosure, an automated test anomaly detection and recovery method is provided. The method includes: embedding a unique identifier corresponding to the test task into the test task and controlling the execution of the test task so that the program under test outputs log messages carrying the unique identifier; establishing a communication connection with the program under test, receiving log messages, and diverting the log messages to an independent log buffer of the corresponding test task according to the unique identifier to obtain a dedicated log sequence corresponding to the target test task; extracting at least two types of time-series features from the dedicated log sequence within a preset time window, the time-series features including log interval measurement and log content diversity measurement; determining the anomaly of the running state of the target test task based on the time-series features, the detected target process state, and the log message output state of the dedicated log sequence; and performing differentiated recovery on the target test task according to its anomaly type based on the anomaly determination result.

[0005] According to one aspect of this disclosure, an automated test anomaly detection and recovery device is provided, comprising: a log embedding and execution module, used to embed a unique identifier corresponding to the test task into the test task and control the execution of the test task so that the program under test outputs log messages carrying the unique identifier; a log distribution module, used to establish a communication connection with the program under test, receive log messages, and distribute the log messages to an independent log buffer of the corresponding test task according to the unique identifier to obtain a dedicated log sequence corresponding to the target test task; a feature extraction module, used to extract at least two types of time-series features from the dedicated log sequence within a preset time window, the time-series features including log interval measurement and log content diversity measurement; an anomaly determination module, used to determine the anomaly of the running state of the target test task based on the time-series features, the detected target process state, and the log message output state of the dedicated log sequence; and a differentiated recovery module, used to perform differentiated recovery on the target test task according to the anomaly determination result output by the anomaly determination module, which is adapted to the anomaly type.

[0006] According to one aspect of this disclosure, an electronic device is provided, comprising: a memory storing computer-executable instructions executable by a processor; and a processor for executing the computer-executable instructions to implement any of the above methods.

[0007] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements any of the above methods.

[0008] One embodiment of this disclosure provides an automated test anomaly detection and recovery method, comprising: embedding a unique identifier corresponding to the test task into the test task and controlling the execution of the test task so that the program under test outputs log messages carrying the unique identifier; establishing a communication connection with the program under test, receiving log messages, and diverting the log messages to an independent log buffer of the corresponding test task according to the unique identifier to obtain a dedicated log sequence corresponding to the target test task; extracting at least two types of time-series features from the dedicated log sequence within a preset time window, the time-series features including log interval measurement and log content diversity measurement; determining anomalies in the running state of the target test task based on the time-series features, the detected target process state, and the log message output state of the dedicated log sequence; and performing differentiated recovery on the target test task according to its anomaly type based on the anomaly determination result. In this way, anomaly determination and differentiated recovery based on unique identifier diversion and time-series feature extraction achieve precise isolation of multi-task logs, reduce the false fault rate and recovery time, and improve the efficiency of automated test monitoring. Furthermore, it helps to reduce the number of unnecessary restarts triggered by false faults, reduce computational resource overhead during the testing process, and improve system resource utilization. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of a system architecture is shown in one exemplary embodiment of this disclosure. Figure 2 This diagram illustrates an automated test anomaly detection and recovery method according to one exemplary embodiment of the present disclosure. Figure 3 This diagram illustrates the structure of an automated test anomaly detection and recovery device according to one exemplary embodiment of the present disclosure. Figure 4 A schematic diagram of the structure of an electronic device is shown in one exemplary embodiment of the present disclosure. Detailed Implementation

[0011] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0012] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0013] The accompanying drawings are schematic illustrations of this disclosure and are not necessarily drawn to scale. Some block diagrams shown in the drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in hardware modules or integrated circuits, or in networks, processors, or microcontrollers. Implementations can be carried out in various forms and should not be construed as limited to the examples set forth herein. The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough description of embodiments of this disclosure. However, those skilled in the art will recognize that one or more specific details may be omitted when implementing the technical solutions of this disclosure, or other methods, components, apparatuses, steps, etc., may be used to replace one or more specific details.

[0014] Figure 1 A system architecture diagram of the operating environment of this exemplary embodiment is shown. This system architecture may include a terminal device 110 and a server 120. The terminal device 110 may be a mobile phone, tablet computer, personal computer, smart wearable device, game console, etc., and has a display function capable of displaying a graphical user interface, which may include the operating system interface or the application interface. An application, such as a game program, is installed on the terminal device 110. The server 120 generally refers to the backend system providing application services in this exemplary embodiment; it may be a single server or a cluster of multiple servers. For example, a game server program is deployed on the server 120 to perform server-side game data processing. The terminal device 110 and the server 120 can be connected via a wired or wireless communication link for data transmission. The method in one exemplary embodiment of this disclosure can be executed by any one or more of the terminal device 110 and the server 120.

[0015] In one implementation, the above method can be implemented and executed based on a cloud interaction system. The cloud interaction system can be the system architecture described above. Various cloud applications, such as cloud gaming, can run under the cloud interaction system. Taking cloud gaming as an example, cloud gaming can be a game mode based on cloud computing. In the cloud gaming operation mode, the game program's execution entity and the game screen presentation entity are separated. The storage and execution of the game's control and interaction methods are completed on the cloud gaming server (such as the aforementioned server 120). The cloud gaming client (such as the aforementioned terminal device 110) is responsible for receiving and sending data and presenting the game screen. For example, the cloud gaming client can be a display device with data transmission capabilities located close to the user, such as a mobile terminal, television, computer, or PDA; while the cloud gaming server in the cloud performs information processing. When playing the game, the user operates the cloud gaming client to send operation commands to the cloud gaming server. The cloud gaming server runs the game according to the operation commands, encodes and compresses the game screen and other data, returns it to the cloud gaming client via the network, and finally, the cloud gaming client decodes and outputs the game screen.

[0016] In one implementation, the method described above can be implemented by the terminal device 110 alone. For example, without deploying the server 120, the terminal device 110 can run the application in a standalone environment to implement the game function and execute the method described above.

[0017] An automated test anomaly detection and recovery method according to one embodiment of this disclosure, such as Figure 2 As shown, the method may include: Step S210: Embed a unique identifier corresponding to the test task in the test task, and control the execution of the test task so that the program under test outputs a log message carrying the unique identifier. Step S220: Establish a communication connection with the program under test, receive log messages, and distribute the log messages to the independent log buffer of the corresponding test task according to the unique identifier to obtain the exclusive log sequence corresponding to the target test task. Step S230: Within a preset time window, extract at least two types of time-series features from the dedicated log sequence. The time-series features include log interval measurement and log content diversity measurement. Step S240: Based on the time sequence characteristics, the detected target process status, and the log message output status of the exclusive log sequence, anomaly determination is made on the running status of the target test task. Step S250: Based on the anomaly determination result, perform differentiated recovery on the target test task that is adapted to its anomaly type.

[0018] According to one embodiment of this disclosure, anomaly detection and differentiated recovery are performed based on unique identifier traffic splitting and temporal feature extraction. This achieves precise isolation of multi-task logs, reduces the false fault rate and recovery time, and improves the efficiency of automated test monitoring. Furthermore, it helps reduce the number of unnecessary restarts triggered by false faults, lowers computational resource overhead during testing, and improves system resource utilization.

[0019] The embodiments of this disclosure will now be further described.

[0020] In step S210, a unique identifier corresponding to the test task is embedded in the test task, and the execution of the test task is controlled so that the program under test outputs a log message carrying the unique identifier.

[0021] Optionally, a unique identifier provides a globally unique ownership key for the test task, enabling its output log messages to be independently traceable and isolated within the global log stream. Optionally, the unique identifier can be constructed using Universal Unique Identifier (UUID) generation rules to ensure that in a distributed multi-server parallel scheduling environment, task identifiers assigned to any two test servers will not collide spatially or temporally. In actual embedding, this unique identifier can be actively written into the log message body as a formatted string by a script injected into the game client each time the log output interface is called, forming a structured record such as data marker followed by timestamp, batch number, task identifier, session sequence number, running stage, and specific content. The position of the task identifier field remains fixed so that the server can quickly extract this field through text parsing after receiving the global log stream and match it with the preset task queue, thereby achieving accurate identification of task-level log ownership without modifying the core source code of the game engine. It should be noted that the above description of the identifier generation rules and embedding format is only one example. In actual deployment, the field order, delimiter type or identifier encoding method can be adjusted according to the engine type or communication protocol requirements of the object under test. This disclosure is not intended to limit this.

[0022] Optionally, log messages are output by the program under test when running test scripts, and at least carry an attribution identifier and event content, serving as the data source for subsequent time-series feature extraction and anomaly detection. Optionally, log messages can be in a structured record format containing a timestamp, message level, attribution identifier, and event description field. The attribution identifier field corresponds to the aforementioned unique identifier, so that after the global log stream enters the real-time communication channel, the server can use this field to distribute log messages generated by different test tasks to their respective independent log buffers. For example, in a racing game's minimum pass level detection scenario, a log message might be a structured text message with a timestamp of March 26, 2026, 14:30:25, a message level of "information," an attribution identifier of "min_pass_level_20260326_143022_uuid1234," and an event content of "Battle Start at Level 101." In another log generated by a system background task, the attribution identifier might be "system monitoring," thus being filtered by the server and excluded from the current task's dedicated log sequence. Furthermore, the structured format of the log messages can be adapted to the scripting language or communication protocol used by the program under test. For example, the separator can be replaced with a key-value pair, or the message level can be refined into multiple levels such as debug, warning, and exception. This disclosure does not limit this.

[0023] In an optional implementation, the unique identifier is a globally unique identifier, which is embedded in the log message body as a structured field. This way, by embedding the globally unique identifier as a structured field in the log message body, not only can duplicate task identifiers be avoided in a distributed concurrent testing environment, but also precise isolation and traceability of multi-task logs can be achieved without modifying the game engine's log interface.

[0024] For example, when orchestrating automated test scripts, the test server generates a task identifier for the current task using a Universally Unique Identifier (UUID) format, such as "min_pass_level_20260326_143022_uuid1234". This identifier is then declared as a local variable in the script. Each time the log output interface is called, the task identifier is appended to the log text according to a predefined structured template. This template begins with a specific marker and uses delimiters to distinguish time, batch, task identifier, session, stage, and content into different categories. This allows the server, upon receiving the global mixed log stream, to accurately distribute logs belonging to different tasks to their respective independent buffers by parsing the log text and extracting fields.

[0025] Optionally, a globally unique identifier aims to ensure the uniqueness of task identifiers in a distributed concurrent testing environment, preventing log distribution errors. Optionally, the aforementioned globally unique identifier can be formatted using a Universally Unique Identifier (UUID). Specifically, it can be generated by combining the test task type, execution date, and a random string, such as "min_pass_level_20260326_143022_uuid1234". Considering that in multi-device concurrent testing scenarios, using simple auto-incrementing sequence numbers or pure timestamps as task identifiers is highly susceptible to identifier conflicts due to clock asynchrony or duplicate numbering rules between different test servers, leading to task logs being incorrectly distributed to the buffers of other tasks. To avoid this problem, using a globally unique identifier can guarantee statistically unique identifiers for tasks from different sources without requiring additional central coordination services. It should be noted that the above description of the universally unique identifier is only one example. In actual implementation, the above-mentioned globally unique identifier can also be a distributed unique sequence number allocated by a centralized number generator, or a unique string generated based on a hash algorithm and a random seed. This disclosure is not intended to limit the specific generation algorithm of the identifier.

[0026] Optionally, structured fields are used to carry task identifiers and context information in a parsable format within the log message body, facilitating automatic extraction and routing by the receiving end. Optionally, the aforementioned structured fields can be string templates composed of a start marker, delimiters, and multiple data columns. For example, in the log body, they could begin with "[DATA]", followed by a vertical bar to sequentially string together time information, batch number, task identifier, current session number, stage status, and specific content text. Considering that if the game engine's native log interface only supports outputting plain text strings, modifying the engine's underlying layer to support custom key-value pairs would not only require intrusive changes but also incur significant migration costs between different engines. As a possible implementation, by encapsulating the task identifier as a structured field at the script level and directly concatenating it into the log string, identifier embedding can be achieved without affecting the engine's core code. Furthermore, the delimiters, besides vertical bars, can also be commas, tabs, or other predefined characters. The order of the columns and the types of information they contain can be flexibly adjusted according to the server-side parsing logic.

[0027] Optionally, the log message body serves as the carrier of the task identifier, allowing the identifier information to enter the collection channel naturally with the log output stream, without requiring modification to the engine's underlying log protocol. Alternatively, the specific method of embedding the task identifier into the log message body can be that the test script, when performing log output operations, directly concatenates the structured field carrying the task identifier into the string content to be printed, and then outputs the complete message to the log stream through the string printing interface provided by the game engine. Considering the differences in log system interfaces among different game engines, some engines may only provide a simple text output channel and not support additional structured fields or extended metadata; to avoid the task identifier not being output with the log due to engine limitations, writing the identifier information into the message body adapts to various engine environments. Whether logs are transmitted back through a real-time communication channel or batch collection is performed on the server side, the task identifier can be extracted by parsing the text. It should be understood that, in addition to the middle area of ​​the log text, the above embedding position can also be by appending the structured field to the message header as a prefix or appending it to the message tail as a suffix; this disclosure does not limit this.

[0028] In step S220, a communication connection is established with the program under test, log messages are received, and log messages are distributed to the independent log buffer of the corresponding test task based on the unique identifier, resulting in a unique log sequence corresponding to the target test task. In this way, by establishing a real-time communication connection and performing log distribution and isolation based on the unique identifier, not only can accurate log isolation be achieved in multi-task concurrent scenarios, avoiding log interference between tasks, but it also provides an independent and complete data foundation for subsequent time-series feature analysis.

[0029] Optionally, the communication connection is used to establish a continuous data channel between the test server and the program under test (DUT) to transmit log messages carrying unique identifiers in real time. Optionally, the communication connection can be established based on SocketIO (a real-time bidirectional communication framework) or WebSocket (a full-duplex communication protocol), enabling the test server to continuously subscribe to and receive log stream data during the execution of the DUT, without relying on high-latency mechanisms such as timed polling or batch file reading. Considering the differences in actual deployment environments, the communication connection can also be adapted to TCP (Transmission Control Protocol) direct connection, communication paths via proxy forwarding, or hybrid deployments. It should be noted that the above description of the communication connection protocol is only one example, and this disclosure is not intended to limit the specific protocol type of the communication connection. Furthermore, the two ends of the communication connection can be deployed on a remote test server and a local test device respectively, and cross-platform adjustments can be made according to the actual data center topology, as long as low-latency, continuous transmission of log data can be achieved.

[0030] Optionally, traffic splitting aims to direct log messages to the independent log buffer of the corresponding task based on a unique identifier, thereby achieving task-level log isolation. Optionally, the traffic splitting operation may include parsing the received global log stream, extracting the identifier field, and using string matching or structured field queries to write log messages whose identifier field matches the unique identifier of the target test task to the task's dedicated independent log buffer, while filtering and discarding log messages with mismatched identifiers. In one possible implementation, the traffic splitting logic is implemented internally on the test server using a hash mapping, with each unique identifier corresponding to an independent buffer handle, thus ensuring complete isolation of logs for concurrent tasks at the memory level. Furthermore, the traffic splitting process may also include timestamp verification and out-of-order reordering to address log arrival order disruptions caused by network jitter, ensuring the timing accuracy of the dedicated log sequence.

[0031] Optionally, an independent log buffer is a separate storage area maintained for each test task, used to temporarily store log messages after they have been split according to their reception order. Optionally, the independent log buffer can be a memory-based circular buffer, a dynamic array, or a queue structure. Its capacity can be pre-configured according to the expected log generation rate of the test task, or it can be dynamically expanded or reduced according to the actual write pressure. Considering the possibility of a surge in log volume during long-term testing scenarios, the independent log buffer can also be configured with multi-level storage strategies. For example, when the memory buffer reaches its capacity limit, earlier logs can be automatically transferred to the local disk or a remote log storage node to avoid data loss. It should be noted that the above description of the buffer storage medium is only one example, and this disclosure is not intended to limit the specific data structure or storage medium of the independent log buffer. In actual implementation, it can be flexibly selected according to the server memory capacity and the scale of test concurrency.

[0032] Optionally, the dedicated log sequence is a set of logs corresponding to the target test task, which is split and ordered temporally to reflect the behavioral trajectory during the task's execution. Optionally, each log message in the dedicated log sequence is arranged in ascending order according to the timestamp when it was written to an independent log buffer, forming strict time-series data. This sequence not only contains the original content of the logs but also retains their corresponding unique identifiers and reception time information, so that the time interval between adjacent logs and the content diversity measure can be accurately calculated during subsequent temporal feature extraction. Furthermore, the dedicated log sequence can also support slicing according to a preset time window, thereby providing the anomaly detection module with a fixed-length and temporally continuous subset of logs, improving the statistical stability of feature calculation. It should be noted that the above description of the sorting method and slicing logic is only one example, and this disclosure is not intended to limit the specific organizational form of the dedicated log sequence.

[0033] In an optional implementation, the communication connection is established based on a full-duplex long-connection protocol. Traffic splitting includes: parsing each log message in the received global log stream, extracting the identifier field, and writing the log message whose identifier field matches the unique identifier corresponding to the target test task into the independent log buffer corresponding to that target test task. This not only enables low-latency real-time reception of the log stream and precise task-level isolation, effectively avoiding the difficulty in locating issues caused by log congestion during concurrent testing on multiple devices, but also allows for rapid downlink recovery commands based on the bidirectional characteristics of the full-duplex channel, improving anomaly response and self-healing efficiency.

[0034] In one implementation, a full-duplex WebSocket-based connection is pre-established between the test server and the game clients on multiple test devices. When each device outputs logs concurrently, the log messages of all tasks are first fed into the global log stream. The test server receives each message and parses it to extract the task identifier field. This field is then compared with the assigned task identifiers. Only the log messages that match are written to the independent log buffer of the corresponding task, while other irrelevant logs are filtered out and discarded. This achieves physical isolation and independent tracking of the logs of each task in a multi-concurrency scenario.

[0035] Optionally, the communication connection is established using a full-duplex long-lived connection protocol that supports bidirectional continuous push, aiming to achieve low-latency uplink reporting of log streams and immediate downlink of recovery commands. Optionally, the aforementioned full-duplex long-lived connection protocol can include, but is not limited to, technologies such as WebSocket (network sockets) or SocketIO (real-time communication library) that can simultaneously support uplink log transmission and downlink control commands on a single persistent connection. Considering the objective requirements of high log generation frequency and second-level response time for anomaly recovery in game automation testing scenarios, using a full-duplex long-lived connection protocol can avoid the additional overhead and latency accumulation caused by traditional polling mechanisms. In actual deployment, the test server acts as a long-lived connection server listening on a designated port, and the game client or test framework on each test device initiates connections and maintains a heartbeat; when network fluctuations occur, the long connection can be quickly restored through an automatic reconnection mechanism to ensure uninterrupted log stream transmission. It should be noted that the specific choice of the above protocol can be determined based on the network conditions and device resources of the test environment.

[0036] Optionally, the global log stream refers to the undifferentiated, mixed collection of logs output by the program under test during multi-task concurrency, serving as the raw input before stream splitting. Global log streams typically arise in scenarios where multiple devices or tasks concurrently execute automated tests, where log messages from different test tasks and even system background modules are output continuously in an interleaved manner. For example, in a batch of minimum clearance level detection tasks, device A is outputting the battle log for level 101, device B is simultaneously outputting the battle log for level 102, and the operating system background may intersperse memory monitoring information. These undifferentiated logs are mixed together to form the global log stream. Considering the objective problem that directly analyzing this mixed stream would lead to difficulties in anomaly localization, this embodiment treats it as a unified whole input before stream splitting. At the receiving end, logical segmentation is achieved through text parsing and identifier field matching, thereby achieving task-level isolation without modifying the log system architecture of the program under test. It should be noted that the data format of the global log stream can be structured text or semi-structured text with delimiters.

[0037] Optionally, text parsing aims to identify and extract identifier fields from the text content of a single log message, serving as the data basis for subsequent matching and routing. Optionally, the text parsing processing method described above can be flexibly selected based on the actual format of the log message. As one implementation, when the log message uses semi-structured text concatenated with preset delimiters, it can be directly sliced ​​and extracted based on the delimiters. For example, strings containing segments such as time, batch number, task identifier, session, stage, and content can be split vertically to directly locate the column containing the task identifier. As another implementation, when the log message is output in a structured format such as JSON (JavaScript Object Notation), the identifier field can be directly read through key-value lookup. Considering that log formats may differ between different game engines or versions, the text parsing module can be configured to support multiple template switching. After establishing a communication connection, it can automatically identify or manually specify parsing rules based on the first log message to ensure accurate extraction of identifier fields. The above parsing process can be executed only on the test server side and does not require the tested game client to have additional structured field support capabilities.

[0038] Optionally, the identifier field is structured key-value information embedded in the log message, representing the target test task to which the log belongs, serving as the primary basis for traffic routing and filtering. Optionally, the identifier field in the log message can be directly embedded as a string concatenation in the body text, or it can be carried through a preset field. In one specific approach, the test script concatenates the task identifier into each output using a formatted string during execution, ensuring the log body starts with a specific marker and sequentially includes segments such as timestamp, batch number, task identifier, current session, running stage, and specific content. The server then performs text parsing on this string to extract the identifier field. In another approach, if the game engine's log interface supports custom parameters, the task identifier can be passed as an independent field when calling the log output function, directly generating a structured log in key-value pair format.

[0039] In step S230, within a preset time window, at least two types of time-series features are extracted from the dedicated log sequence. The time-series features include log interval measurement and log content diversity measurement.

[0040] Optionally, the aforementioned preset time window can be a fixed-length window or a dynamic window that adaptively slides based on log output density. Considering the objective differences in the running pace of different game types—for example, the log output frequency of fast-paced action games is usually higher than that of strategy games—using a single fixed duration might result in insufficient log samples in the window for slow-paced games or data redundancy in the window for fast-paced games. As a possible implementation, for fast-paced action games, the window length can be set to 30 to 60 seconds; for strategy or role-playing games, the window length can be extended to 120 to 300 seconds. Furthermore, the window length can be adaptively adjusted based on historical log output frequency. For example, requiring at least ten log messages within the window to ensure statistical significance; automatically extending the window duration if the actual log stream is sparse; and appropriately shortening it if the log stream is dense to improve anomaly response sensitivity. It should be noted that the specific values ​​for the window duration mentioned above are only examples and can be flexibly set according to the network latency, device performance, and specific behavioral characteristics of the program under test in actual deployment.

[0041] Optionally, the aforementioned time-series features are quantitative indicators extracted from a dedicated log sequence that reflect the changing patterns of the tested program's running state over time. Their extraction aims to transform the raw, unstructured text logs into numerical feature vectors that can be directly computed by decision rules. Considering that a single feature often fails to comprehensively depict complex abnormal states—for example, a shortening log interval might correspond to normal loading or an infinite loop, while high content repetition might indicate a freeze but not a process crash—as a possible implementation, at least two types of time-series features could include log interval measurement and log content diversity measurement. These two measures characterize the log stream from the time and content dimensions respectively, forming complementary judgment criteria. Furthermore, the extraction of time-series features can also incorporate the frequency of predefined abnormal keywords, enabling anomaly judgment to not only focus on the log's temporal behavior patterns but also capture clear error semantic signals. In other words, by extracting multi-dimensional features, misjudgments by a single indicator in boundary scenarios can be effectively avoided, thus laying a data foundation for subsequent comprehensive anomaly judgment.

[0042] Optionally, the aforementioned log interval metric includes a central tendency index (CTI) of the timestamp difference sequence of adjacent log messages, used to characterize the temporal distribution pattern of log output. Optionally, the aforementioned log interval metric can be obtained by calculating the timestamp difference of adjacent log messages in a dedicated log sequence to obtain a time interval sequence, and then quantitatively characterizing it based on the CTI of this sequence. Considering that the arithmetic mean is prone to distortion when there are extremely long intervals, for example, if log output is extremely uneven within a certain period, multiple short intervals accompanied by individual long intervals will raise the mean, thus masking the risk of system crashes implied by repeated logs at short intervals, as a possible implementation, the aforementioned CTI can use the median to reduce the interference of extreme values ​​on the overall metric and more robustly reflect the typical frequency of log output. Furthermore, in addition to the median, the CTI can also use the mode or truncated mean, etc., and this disclosure does not limit this. It should be understood that when the log interval metric is lower than the corresponding preset benchmark, it often means that the log output exhibits high-frequency repetition or bursty output, which can be used as one of the important criteria for determining the application's unresponsive state; while in the normal state of waiting for resources to load, although the log interval may be relatively long, it usually does not exhibit a dense distribution of extremely short intervals.

[0043] Optionally, the aforementioned log content diversity measure is used to measure the degree of repetition or semantic richness of log message content within a preset time window. Its calculation process typically includes normalizing the log messages to remove variable fields that change with the test instance, such as timestamps, task identifiers, or session numbers. Then, semantic templates are extracted, and logs with completely identical content are grouped into the same category. As a possible implementation, Shannon entropy can be used as a diversity measure. When log content within the window is highly repetitive, the entropy approaches zero. Under normal operating conditions, due to the inclusion of multiple semantic templates such as battle start, acceleration, and settlement, the entropy usually remains at a high level. Besides entropy, Gibbs entropy or the proportion of template types can also be used to characterize content diversity. It should be noted that the types of variable fields removed in the above normalization step can be flexibly adjusted according to the log format. They can be metadata fields automatically added by the system or sequence markers dynamically generated in the script. This disclosure is not intended to limit this. Furthermore, by jointly analyzing log content diversity and log interval metrics, we can effectively distinguish between normal waiting and dead loop states, avoiding misjudgments caused by analysis based on a single time dimension.

[0044] In an optional implementation, the length of the preset time window is dynamically adjusted based on the type of the object under test and the historical log output frequency of the target test task, so that the number of log entries in the exclusive log sequence within the preset time window meets the preset statistical requirements. This dynamic adjustment of the preset time window length based on the type of the object under test and the historical log output frequency ensures that the number of log entries within the window meets the statistical requirements. This avoids both feature loss due to an excessively short window and slowed-down anomaly response due to an excessively long window, thereby improving the accuracy of feature extraction and the reliability of anomaly detection.

[0045] Optionally, the length of the preset time window can be dynamically set based on the type of the object being tested and the frequency of historical log output to balance feature granularity and detection response speed. For example, when the object being tested is a fast-paced action game, the number of operation logs and event logs generated by the engine per unit time is large and dense. If the window is set too long, it will lead to a large accumulation of logs, increasing the redundancy of subsequent time-series feature calculations and slowing down the response speed of anomaly detection. In this case, the window length can be set to a shorter range of 30 to 60 seconds. When the object being tested is a strategy or role-playing game, the player's decision-making cycle is long, game events are triggered sparsely, and the log output frequency is low. If the window is too short, it will be difficult to collect a sufficient number of log entries for reliable statistical analysis. In this case, the window can be extended to 120 to 300 seconds. Furthermore, the system can also track the average log output interval of the target test task over several consecutive time periods. When the actual output frequency is detected to be significantly lower than the expected baseline, the window length is automatically extended to ensure that the number of accumulated log entries within the window remains at a statistically significant level.

[0046] Optionally, in addition to coarse-grained classification based on the game's broad genre, the dynamic adjustment of the window length can also be fine-grained by considering the specific stage of the current test. For example, during the resource loading phase before battle, the engine outputs relatively few log entries. The system can automatically increase the window length during this phase to 1.5 to 2 times that of the normal battle phase to avoid misjudging as unresponsiveness due to sparse logs. During the battle settlement phase, if a sudden surge in log entries is detected as normal, dense output triggered by skill combos, the window can be briefly reduced to capture transient anomalies. It should be noted that the adjustment step size for the window length can use a fixed step or a continuous function for smooth transition. This disclosure does not aim to limit the specific adjustment algorithm, as long as the number of log entries within the window meets statistical requirements and the real-time detection is considered.

[0047] Optionally, the types of test objects mentioned above are not limited to the macro-level classification of games (such as action, strategy, and role-playing games), but can also cover the characteristic tags corresponding to different gameplay modes within the same game. For example, in a comprehensive game that combines racing and character development, the log output patterns in racing mode and town interaction mode differ significantly. The system can configure different window length baseline values ​​for these two modes respectively. Furthermore, the type of test object can also reflect the hardware platform attributes on which the game relies. For instance, mobile devices are limited by performance and network environment, resulting in different log output stability compared to PCs or consoles. When the system identifies that the test object is running on a mobile device, it can appropriately widen the window length to offset occasional log latency fluctuations. It should be understood that the above classification methods can be used individually or in combination, and this disclosure does not limit this.

[0048] Optionally, the above historical log output frequency is obtained by statistically analyzing the average log output interval of the target test task in previous consecutive time periods, and serves as an important reference for dynamically adjusting the window length.

[0049] Optionally, the statistical object for the aforementioned historical log output frequency can cover all logs generated since the target test task started, or it can select only log samples from a few recent time periods to highlight the timeliness of the recent running status. To avoid interference from occasional log storms or log silences on frequency statistics, outliers in the timestamp sequence can be removed before calculating the average output interval. For example, long intervals significantly higher than three standard deviations can be considered outliers and filtered out. Furthermore, when the system detects that the historical log output frequency is continuously decreasing and falls below the preset lower limit, not only can the time window length be increased accordingly, but a health check of the log collection channel can also be triggered simultaneously to confirm whether the frequency decrease is caused by improper window settings or by a deterioration in the communication link quality between the game client and the test server.

[0050] Optionally, the aforementioned preset statistical quantity requirement refers to the lower limit of the number of log entries in the dedicated log sequence within the window, which is flexibly set based on the minimum sample size required for calculating the time-series characteristics. This preset statistical quantity requirement is not a fixed value but can be flexibly configured according to the type of time-series characteristics extracted and the sensitivity requirements of the decision rules. For example, when relying solely on log interval metrics and error keyword frequencies for lightweight judgment, 8 to 10 log entries within the window are often sufficient to reflect the basic trend; while when it is necessary to accurately calculate information entropy and fit the skewness and kurtosis of the time interval distribution, it is usually required that at least 15 to 20 log entries be accumulated within the window to ensure the stability of the probability distribution estimation; in another example, when it is necessary to perform millisecond-level jitter detection on high-frequency combat events in the physics engine, the preset statistical quantity requirement is set to at least 25 log entries within the window to ensure that the transient anomaly pattern has statistical significance. In addition, the statistical quantity requirement can also be linked to the window length: if the system determines twice in a row that the number of log entries in the window is insufficient, the current quantity requirement will be temporarily reduced by a preset ratio, and the confidence level of this feature extraction will be marked at the same time to avoid unnecessary delays in anomaly detection due to mechanically waiting for the sample size.

[0051] In an optional implementation, the timing feature also includes the frequency of occurrence of predefined exception keywords. Based on the timing feature and the detected target process state, the system determines the anomaly of the target test task's running state, including: statistically analyzing the proportion of log entries containing predefined exception keywords in a specific log sequence within a preset time window; if the proportion of log entries exceeds a preset threshold, it is determined to be a logical anomaly. Thus, by introducing the frequency of occurrence of predefined exception keywords on top of log interval measurement and log content diversity measurement, the system can promptly capture logical errors at the code level, such as null pointer access, array out-of-bounds errors, or script execution failures, even when the tested program process is still alive and continuously outputting logs. This feature effectively fills the detection blind spot when relying solely on log timing behavior pattern analysis, avoiding misjudging substantial logical anomalies as application unresponsiveness or normal operation. This allows for early identification of code-level defects and rapid isolation of test cases without triggering process restarts, significantly improving the sensitivity and accuracy of automated testing for logical errors.

[0052] In one implementation, taking the minimum completion level detection of level 101 in a racing game as an example, after the test script drives the AI ​​to start the battle, the test server collects a dedicated log sequence within a 60-second time window. If multiple log entries containing anomaly markers appear within this window, and their proportion of total log entries reaches 35%, exceeding a preset threshold, the system immediately determines that the current test task has triggered a logical anomaly, automatically marks the test case as failed, and loads the next test case, thus avoiding blockage of the overall test process.

[0053] Optionally, predefined exception keywords are designed to identify code-level error information output during the execution of the program under test. These keywords may include text markers indicating runtime errors, such as strings indicating NullPointerException, array out-of-bounds errors, memory access violations, or script execution failures. Considering the differences in log output formats used by different programs under test, as a possible implementation, these keywords can be deployed on the test server as configuration files. Testers can dynamically add or remove them based on the development language characteristics of the program under test and its historical defect database. It should be noted that the above description of keyword types is only one example, and this disclosure is not intended to limit their specific content. In actual deployment, keywords can cover not only error statements at the engine or framework layer but also custom error codes at the business logic layer, thereby quickly capturing multi-level program exception signals through flexible text matching strategies.

[0054] Optionally, the log entry ratio measures the proportion of abnormal keyword logs in the total logs within a time window, reflecting the relative level of abnormal severity. Optionally, the calculation process for the log entry ratio can specifically include: traversing all log messages in the dedicated log sequence within a preset time window, checking each message for whether its content matches a predefined abnormal keyword, and dividing the number of matches by the total number of log messages in the window to obtain the ratio value. Considering that the total log output may vary significantly in different testing phases or for different programs under test, using a relative ratio rather than an absolute count avoids excessive amplification of a small number of error logs when the total log volume is small, and prevents error signals from being overwhelmed by normal logs when the total log volume is large. In other words, the log entry ratio provides a normalized method for measuring abnormality intensity. When the log entry ratio exceeds the corresponding preset threshold, the system can accurately trigger logical abnormality judgment while the process is still alive, without waiting for more serious failure modes such as process crashes or freezes to occur.

[0055] Optionally, the specific value of the preset threshold can be flexibly adjusted according to the stability of the program under test and the testing phase. For example, in the early stages of rapid iteration development, the threshold can be set to 20% to expose potential defects as early as possible; in the version acceptance phase, the threshold can be set to 40% to avoid misjudgments caused by oversensitivity; and in the regression testing phase of multiplayer online competitive games with extremely high stability requirements, the threshold can be set to 10% to capture occasional boundary memory anomalies. As one possible implementation, the above threshold can be determined by analyzing confirmed logic anomaly samples in historical tests and statistically analyzing the distribution pattern of their abnormal keyword proportions. It should be noted that the above numerical range is only an example, and this disclosure is not intended to limit the specific size of the preset threshold. In actual implementation, testers can also customize the configuration according to the business criticality or quality gate requirements of the program under test.

[0056] Optionally, considering that a single static threshold may not be suitable for all testing scenarios, the preset threshold can be dynamically optimized using machine learning methods. For example, a classification model can be trained using historical log samples to automatically learn the optimal boundary point for different game types or different level stages. If the proportion of log entries is near the threshold boundary, leading to ambiguity in the judgment, a joint decision can be made by combining the trend changes of adjacent time windows or the diversity measure of log content, rather than relying solely on a single ratio. Based on this, by introducing dynamic adjustment and auxiliary verification mechanisms, the sensitivity and accuracy of anomaly detection can be effectively balanced, reducing the risk of false negatives or false positives caused by improper threshold settings.

[0057] Optionally, logical anomalies represent a state where the tested program's process still exists but errors occur at the code execution level, which is fundamentally different from process crashes or application unresponsiveness. Logical anomalies can include, but are not limited to, null pointer access, array out-of-bounds errors, division by zero errors, illegal state transitions, or script runtime failures. Unlike the complete interruption of the log stream caused by a process crash, the tested program in a logical anomaly state usually continues to run and output logs, but error markers repeatedly appear in the logs; unlike the highly repetitive logs caused by application unresponsiveness, the log content of logical anomalies may maintain a certain degree of diversity but is mixed with a large amount of error information. As a possible implementation, when the system determines a logical anomaly, it can automatically mark the current test case as a failure, record relevant log fragments and store them in the database, and then load the next test case to continue execution, thereby avoiding the code defects of a single test case from blocking the progress of the overall test pipeline.

[0058] Optionally, considering that the test script's own debugging output or third-party framework logs may also contain records with similar error messages, the above method can further distinguish the log source layers when statistically analyzing predefined exception keywords, assigning different weights to errors output by the engine core layer and debugging information output by the test script layer. It should be understood that in concurrent testing scenarios, the judgment result of logical exceptions can be associated with a unique identifier to accurately locate specific tasks and devices, enabling subsequent problem backtracking to directly pinpoint the code context in which the exception occurred. One objective of this embodiment is to more robustly identify genuine code logic defects, rather than normal warnings or expected exception handling processes, in complex operating environments by combining the proportion of log entries with log content diversity and log interval measurements.

[0059] Optionally, the target process status is used to indicate the survival status of the tested program process, serving as an important basis for distinguishing between process-level failures and logical failures. Optionally, the target process status can be determined by sending a status query command to the device where the tested program is deployed, for example, by obtaining the process's survival identifier via a heartbeat detection interface or process management service interface. As a possible implementation, when a dedicated log sequence exists within a time window and the proportion of log entries exceeds a preset threshold, combined with the target process status being alive, the anomaly type can be identified as a logical anomaly rather than application unresponsiveness or crash. It should be noted that the above detection methods are merely examples, and this disclosure is not intended to limit the specific means of obtaining process status.

[0060] In an optional implementation, the log interval metric includes a central tendency indicator based on the sequence of timestamp differences between adjacent log messages in a dedicated log sequence. By using a central tendency indicator to aggregate and measure the sequence of timestamp differences between adjacent log messages, the impact of individual extreme intervals on the log output frequency assessment can be effectively smoothed, making the distinction between game freezes and normal waiting more robust and reliable.

[0061] Optionally, a central tendency metric is used to aggregate the timestamp difference sequence to generate a value characterizing the typical level of log output frequency. Optionally, considering that the timestamp differences between adjacent log messages in a dedicated log sequence exhibit a non-uniform distribution in actual operation—for example, log intervals during the combat phase of a fast-paced action game may be concentrated between 0.5 and 2 seconds, while occasionally long intervals of tens of seconds may occur due to resource loading—directly taking the arithmetic mean would be distorted by extreme value perturbations. As a possible implementation, the aforementioned central tendency metric can be calculated using the median. The median is not sensitive to outlier long intervals and can more robustly characterize the central level of log output frequency, thereby avoiding misjudging normal operation with occasional waiting gaps as abnormal silence. As another possible implementation, the aforementioned central tendency indicator can also be calculated using a truncated mean. Specifically, after sorting the time interval sequence by size, the extreme values ​​at both ends of the sequence (10% at each end) are removed, and then the arithmetic mean is calculated for the remaining samples. This truncated mean retains the sensitivity of the mean to the overall distribution while effectively suppressing the interference of occasional long intervals caused by network jitter or background task switching on the evaluation of log output frequency by removing extreme abnormal intervals at the beginning and end. This establishes a more robust distinguishing boundary between normal loading and repetitive log bursts. Furthermore, the central tendency indicator can also use other statistical measures such as the geometric mean or mode. Its selection can be dynamically adjusted based on the historical log fluctuation characteristics of the tested program and the response sensitivity required for anomaly detection. This disclosure is not limited to a specific calculation form.

[0062] Optionally, the process of constructing the timestamp difference sequence of adjacent log messages may include: sorting each log message in the dedicated log sequence according to time sequence, extracting the timestamp field carried by two adjacent log messages in turn, and calculating their difference to obtain a set of discrete time interval samples. For example, if the time window contains three consecutive log messages with timestamps of 14:30:25, 14:30:27, and 14:30:30, the corresponding difference sequence will include two elements: 2 seconds and 3 seconds. This difference sequence not only covers the average density of log output but also retains the original fluctuation information of time distribution, providing a data foundation for subsequent aggregation measurement using central tendency indicators. It should be noted that the above calculation is not limited to strictly physically adjacent log entries. In scenarios with obviously noisy logs or heartbeat logs, the dedicated log sequence can be preprocessed and filtered first, and then the difference can be calculated based on the filtered valid logs. This disclosure does not limit this.

[0063] In an optional implementation, the central tendency metric includes the median of the time interval sequence between adjacent log messages in the dedicated log sequence. Thus, by using the median as the central tendency metric for the time interval between adjacent log messages, the interference of occasional abnormally long intervals on the judgment results can be effectively reduced, improving the accuracy of application unresponsiveness identification.

[0064] In one implementation, suppose the target test task generates multiple logs within a preset time window, with most adjacent logs spaced 0.5 seconds apart, but an occasional 30-second long interval occurs due to resource loading. If the arithmetic mean is used to calculate the log interval metric, this extreme value will significantly inflate the average result, masking the deadlock signal indicated by short-interval repetitive logs. In this case, the median is used as a central tendency indicator. Since the median is not sensitive to extreme values ​​and remains at 0.5 seconds, it accurately reflects the true timing characteristics of log repetition bursts, effectively reducing the probability of misjudging application unresponsiveness.

[0065] Optionally, the above-mentioned central tendency metric includes the median of the time interval sequence of adjacent log messages, which is used to resist the interference of extreme abnormal intervals on the overall measurement results.

[0066] Optionally, when calculating the log interval metric, the process of obtaining the median may include: first, parsing the timestamp field of each log message in the dedicated log sequence to obtain a timestamp sequence sorted chronologically; then, calculating the difference between two adjacent timestamps to form a time interval sequence; and finally, sorting the time interval sequence and taking the value in the middle as the median. Considering that during automated testing, the program under test may experience significantly longer log intervals due to occasional resource loading delays, background task switching, or instantaneous network fluctuations, using the arithmetic mean is easily affected by such extreme values, leading to misjudgments. The median, based on positional statistics, has a natural robustness to extreme outliers. For example, if there are ten logs in the current time window, with nine adjacent intervals of 0.5 seconds and one of 30 seconds, the arithmetic mean is approximately 3.45 seconds, which may be misjudged as normal waiting; while the median remains at 0.5 seconds, accurately indicating the risk of a freeze corresponding to a log recurrence outbreak, thereby improving the accuracy and reliability of anomaly detection.

[0067] In an optional implementation, log content diversity measurement includes: extracting semantic templates from log messages in a dedicated log sequence, grouping log messages conforming to the same semantic template into the same category; normalizing log messages to remove variable fields that change with test instances; statistically analyzing the distribution of each category, and calculating a weighted concentration index based on the proportion of each template as a measure of log content diversity. Thus, through semantic template extraction, normalization, and weighted concentration index calculation, log content diversity can be transformed into quantifiable criteria, effectively distinguishing log characteristics between normal operation and frozen states, and significantly improving the accuracy of anomaly detection.

[0068] Optionally, semantic template extraction aims to categorize log messages into different classes based on their structured text content, establishing a classification foundation for subsequent distribution statistics. Optionally, semantic template extraction is a crucial step in identifying the core semantics of log messages during the calculation of log content diversity measurement. Considering that the original log string contains variable information without event semantics, directly comparing it based on the complete text would severely exaggerate the diversity of log content. Therefore, the above extraction operation focuses on stable fields in the log message that can represent its business meaning, grouping log messages that conform to the same semantic template into the same category. Specifically, this extraction operation does not simply perform an exact match on the entire log string, but rather extracts the stage identifier and event description text as a template skeleton. For example, in an automated testing scenario for racing games, a log message, after preprocessing, might appear as "[running] AI driving the vehicle accelerates" or "[finish] First lap completed," and semantic template extraction identifies them as different categories accordingly; while two log messages with different timestamps but identical preprocessed text are grouped into the same category. Furthermore, to avoid duplicate counting of the same event due to differences in variable fields, the above extraction operation is usually performed after normalization to ensure that only the stable part of the text content participates in the category determination. It should be noted that the field selection for semantic template extraction described above is only one example. In actual implementation, different types of events such as skill release, level loading, and network synchronization can be mapped to different semantic template categories according to the log specifications of the game under test. This disclosure is not intended to limit the number of extractable semantic categories or the granularity of the division.

[0069] Optionally, the specific implementation of semantic template extraction can be flexibly adjusted according to different log formats and game types. Besides template matching based on stage identifiers and text content, a pattern matching method based on regular expressions can also be used to extract key field combinations from log messages as template identifiers; this disclosure is not limited to this. Considering the significant differences in log structures across different game types, as a possible implementation, for combat games, "[Skill Trigger][Character Identifier][Skill Identifier]" can be extracted as the core skeleton of the semantic template, ignoring variable parameters such as specific damage value fluctuations; while for strategy games, "[Turn Start][Player Identifier][Action Type]" can be used as the template basis. It should be understood that the above description of semantic template extraction rules is only one example, and this disclosure is not intended to limit the coarseness of the extraction granularity. Furthermore, developers can use coarse-grained templates to group multiple similar events into the same category, or they can use fine-grained templates to differentiate them, in order to balance detection accuracy and computational overhead.

[0070] Optionally, normalization aims to remove variable fields in log messages that vary with test instances, ensuring that repeated outputs of the same semantic event are statistically treated as the same template. Specifically, normalization removes variable fields from log messages that vary with test instances. These variable fields can include, but are not limited to, timestamps, task identifiers, batch identifiers, and session numbers—information that differs only in a specific execution. For example, the original log "[DATA][2026-03-26T14:30:25.123Z][batch_001][task_101][Session 3 / Total 5][running]AI driving vehicle accelerating" will be simplified to "[running] AI driving vehicle accelerating" after normalization. This way, two logs with essentially the same content but different timestamps will be completely consistent after normalization, allowing them to be accurately categorized into the same semantic class. The purpose of this processing step is to avoid artificially inflating information entropy due to differences in formatted peripheral information, ensuring that the diversity measurement of log content truly reflects the substantial changes in the game's running state. It should be noted that the above examples of variable fields are only illustrative. In actual implementation, the removal rules for normalization processing can be dynamically configured according to the log output specifications of the game under test. This disclosure is not limited to this.

[0071] Optionally, the weighted concentration metric measures the proportion of each type of template appearing within a statistical window, numerically reflecting the richness or repetition of log content. One specific implementation of this metric is the Shannon Entropy, calculated based on the probability of each semantic template appearing within a preset time window. When the game is running normally, log messages cover various event types such as battle start, skill trigger, and turn completion, with a relatively even distribution of the frequency of different semantic templates. At this time, the entropy value is high, for example, greater than 1.5. However, when the game freezes, the log output often shows the cyclic printing of the same error, causing the proportion of a certain semantic template to approach 1, at which point the entropy value drops sharply and approaches 0. Another specific implementation of this metric is the Gini Index, calculated by subtracting the sum of the squares of the probabilities of each semantic template from 1. For example, in the same racing game test scenario, if the probabilities of the five semantic templates appearing in the window are 0.2, 0.2, 0.2, 0.2, and 0.2 respectively, the Gini index is 0.8, indicating that the content distribution is uniform and the system is running normally. If the game freezes, causing the probability of a certain template to approach 1, the Gini index approaches 0. Based on this, the system can also establish a clear numerical boundary to distinguish between normal operation and dead loop states. By using Shannon information entropy or the Gini index as a measure of log content diversity, the system can flexibly select according to the computational resource constraints of the test scenario. Thus, under high real-time requirements, the low computational overhead of the Gini index can be used to achieve rapid anomaly screening, or when statistical analysis accuracy is prioritized, information entropy can be used to achieve a more detailed characterization of template distribution. It should be noted that the above examples of information entropy and the Gini index are only two implementation methods. Variations can also be designed by taking their reciprocals, applying linear scaling, or introducing smoothing factors. This disclosure is not intended to limit the specific mathematical transformation forms.

[0072] In step S240, based on temporal features, the detected target process state, and the log message output status of the dedicated log sequence, anomaly determination is made regarding the running status of the target test task. This multi-dimensional joint determination, combining temporal features, process state, and log output status, not only significantly reduces the false positive rate but also distinguishes between application unresponsiveness and process crashes within seconds, contributing to improved stability of unattended automated testing over extended periods.

[0073] Optionally, the above anomaly determination is not limited to a single temporal feature or a single process state detection. Instead, it involves a multi-dimensional joint analysis of temporal features comprised of log content diversity measurement, log interval measurement, and predefined abnormal keyword frequency, along with the target process's heartbeat survival status and the message output status of the specific log sequence within a preset time window. Specifically, this determination process can include two levels of classification logic: the first level classifies tasks into log message output classes and log message output classes based on the log message output status and the target process status; for the log message output class, it further distinguishes between two distinct anomaly forms—program crash and application unresponsiveness—based on the existence of the process; for the log message output class, it combines the joint deviation of information entropy and log interval, and incorporates the content anomaly indicator of error keyword frequency, to make a comprehensive determination according to a preset anomaly priority. It should be noted that when multiple anomaly conditions are met simultaneously, the determination logic prioritizes the application unresponsiveness state, thereby avoiding misjudging a freeze as a normal logical anomaly and ensuring that the recovery strategy matches the fault scenario. In addition, the length of the aforementioned time window and each preset benchmark can be dynamically adjusted according to the type and rhythm of the game being tested, in order to adapt to the judgment requirements under different testing scenarios.

[0074] Optionally, the above log message output status is used to characterize whether a new log message is generated within a preset time window for a specific log sequence. Specifically, this status can be considered a Boolean or counting monitoring marker: when no log message is received within the time window, it indicates that the tested program may be in a silent state with no output, and at this time, a first-level anomaly classification should be performed in conjunction with the heartbeat detection results of the target process; when one or more log messages are received within the time window, it indicates that the program is still generating log output, but the output content itself may exhibit highly repetitive characteristics or contain a large number of erroneous keywords. Furthermore, even if there is log output within the window, the system can still perform a more granular risk assessment of the output status based on the dynamic changing trend of the log content diversity metric. For example, in the process of determining application unresponsiveness, if the log interval metric is not lower than the preset benchmark but the information entropy decreases significantly, the system can choose to wait for a predetermined time and then resample to confirm whether the decrease is a persistent freeze signal or a temporary resource loading delay. It should be noted that the statistical range of the above output status is strictly limited to the specific log sequence after being filtered by a unique identifier, thereby effectively shielding noise interference caused by other concurrent test tasks or system background tasks.

[0075] In an optional implementation, anomaly determination includes: based on the log message output status of the dedicated log sequence within a preset time window and the target process status, determining the primary anomaly category of the target test task. The primary anomaly categories include no log message output and log message output. When the primary anomaly category is no log message output, the anomaly determination determines that the target test task is a crash of the tested program if the target process does not exist, and an unresponsive application if the target process exists. Thus, by first classifying the primary anomaly category according to the log output status and then combining it with process existence detection, it is possible to accurately distinguish between a crash of the tested program and an unresponsive application, significantly reducing the probability of false positives in complex scenarios and improving the accuracy of fault location.

[0076] In one implementation, the test server performs preliminary screening based on the log message output status of a dedicated log sequence within a preset time window: if no log messages are received within the time window, the current test task is classified as having no log message output; conversely, if log messages exist within the window, it is classified as having log message output. For the no-log-message-output category, the system further detects the process heartbeat of the target game instance by sending a process query command (such as a Hypertext Transfer Protocol query request to a health check interface on a specified port) to the device management service. If the heartbeat detection times out or returns to offline status, it indicates that the target process no longer exists, the exception type is determined to be a crash of the tested program, and a process restart is triggered; if the heartbeat detection returns normally, it indicates that the process is still alive but the logic layer is blocked, the exception type is determined to be an unresponsive application, and a soft recovery command is attempted first. Thus, the system can accurately distinguish between process-level failures and application-level deadlocks in log-silent scenarios, avoiding the unnecessary full restart of all no-log-output scenarios by categorizing them all as crashes.

[0077] Optionally, the first-level exception category is used to initially classify exceptions into no-log and log-enabled categories based on log output status. The no-log output category indicates that there are no new log messages in the exclusive log sequence within a preset time window. This could be due to a crash of the tested program causing process termination, or due to application unresponsiveness causing logic layer blocking while the process still exists. The log-enabled output category indicates that log output still exists within the window, requiring further in-depth judgment based on multi-dimensional temporal characteristics such as log content diversity measurement, log interval measurement, and error keyword frequency. It should be noted that the above-mentioned first-level exception category classification is only one example. In actual implementation, it can be extended to more levels according to the complexity of the tested object (for example, further subdividing network interruption and local deadlock under the no-log output category). This disclosure is not intended to limit this. One purpose of this layered architecture is to quickly separate anomalies of drastically different natures (process-level disappearance and application-level stagnation) by first filtering the explicit indicator of log output status. On the one hand, it can directly trigger process status detection in scenarios without logs, shortening the judgment path. On the other hand, it can also preserve sufficient feature calculation space in scenarios with logs, avoiding performing indiscriminate deep analysis on all tasks, thereby reducing computational overhead and improving judgment efficiency.

[0078] Optionally, the determination of the no-log message output class can be specifically executed as follows: within a preset time window, count the number of log entries in the dedicated log sequence; when this number is zero, the class is confirmed. It should be noted that "no log output" here should be understood as no output after task-level filtering, meaning that while there may be logs from other tasks or system background processes in the global log stream, no new messages have been received in the dedicated log buffer corresponding to the current target test task. Considering the potential for brief data transmission delays in the log acquisition channel, as a possible implementation, the above determination can introduce a buffer tolerance mechanism: for example, allowing a data reception waiting period of no more than 2 seconds at the window threshold, or requiring no logs for two consecutive preset time windows before finally confirming the no-log message output class. Furthermore, the determination result of the no-log message output class is not directly equivalent to the final anomaly assessment, but only serves as a trigger for initiating secondary detection (process status check). One purpose of this design is to avoid confusing single-point acquisition failure with actual process failure. On the one hand, it can leave reasonable fault tolerance space for the network layer or transport layer, and on the other hand, it can ensure that the subsequent distinction between the crash of the tested program and the application's unresponsiveness is based on a sufficiently robust premise, thereby improving the reliability of the entire anomaly judgment chain.

[0079] Optionally, the "log message output" category can be specifically defined as a dedicated log buffer receiving at least one log message carrying the unique identifier of the target test task within a preset time window. The judgment logic for this category is relatively straightforward; typically, it only requires checking if the number of log entries within the window is greater than zero. Furthermore, to enhance the rigor of the judgment, the log messages within the window can be required to have a non-extremely concentrated time distribution (e.g., excluding the extreme case where all logs flood in during the last second of the window), to avoid misjudging normal output simply by receiving a log message instantaneously at the end of the window. It should be noted that once the "log message output" category is confirmed, the system will not immediately check the process status. Instead, it will prioritize calculating temporal characteristics such as log content diversity metrics, log interval metrics, and error keyword frequency, using multi-dimensional joint deviation analysis to identify application unresponsiveness, logical anomalies, or normal operation. One purpose of this diversion mechanism is to guide complex scenarios with active log output to feature-rich analysis paths, thereby fully utilizing the program execution semantics contained in the log content and avoiding simply allowing or abruptly restarting processes just because logs are not silent, making anomaly judgment more refined and data-driven.

[0080] Optionally, a crash of the tested program is used to characterize an abnormal state where there is no log output and the target process no longer exists, serving as the basis for triggering a process restart. Optionally, the determination of a crash of the tested program can be specifically executed as follows: when the first-level exception category is confirmed as the no-log message output category, the test server sends a process query command to the device management service (e.g., a Hypertext Transfer Protocol query request to a health check interface on a specified port). If the returned status indicates that the process of the target game instance has terminated or the heartbeat timed out without response, then the crash of the tested program is confirmed. It should be noted that the target of the above process query command can be the device management service running locally on the tested device. This service maintains the running status registry of the game instance, rather than directly enumerating the operating system process list. This method is particularly suitable when multiple game instances are running concurrently on the same physical device, and can accurately locate the process corresponding to a specific task. Furthermore, the confirmation of a crash of the tested program can also include a retry verification mechanism: for example, a final conclusion is reached only if two consecutive process queries return an exception, preventing misjudgment due to a single network packet loss. One purpose of this dual-insurance mechanism is to avoid misjudging instantaneous network fluctuations as process termination. On the one hand, it ensures that heavyweight recovery strategies such as process restart are only triggered in real crash scenarios. On the other hand, it can reduce the impact of unnecessary process interruptions on test continuity, thereby improving the stability and resource utilization efficiency of automated testing processes.

[0081] Optionally, "Application Unresponsive" is used to characterize an abnormal state where there is no log output but the target process is still alive, serving as the basis for triggering a soft recovery command. Optionally, the determination of "Application Unresponsive" can be specifically executed as follows: when the first-level exception category is confirmed as "no log message output," and the target process is confirmed to still exist through the device management service or heartbeat interface, the current state is marked as "Application Unresponsive." It should be noted that there is a fundamental difference between "Application Unresponsive" and process-level crash: a process-level crash means that the tested program's process has completely terminated, and system resources have been reclaimed by the operating system; while "Application Unresponsive" means that the process is still running at the operating system level, but its internal logic (such as the combat system, resource loading pipeline, or physics engine) may be stalled due to deadlock, infinite loop, or synchronization wait, preventing the generation of new log output. To further distinguish between "Application Unresponsive" and extremely slow normal operation, as a possible implementation, an observation waiting period (e.g., 30 to 300 seconds) can be initiated after confirming "Application Unresponsive." If the dedicated log sequence still has not resumed output within the waiting period, the determination is formally solidified, and a soft recovery command is triggered (e.g., forcibly ending the current battle through scripting language (Lua) injection). One purpose of this progressive confirmation mechanism is to avoid misjudging certain normal scenarios that require long-term calculations (such as large map loading and complex settlement calculations) as deadlocks. On the one hand, it preserves a reasonable execution window for normal but time-consuming game logic, and on the other hand, it ensures that the real unresponsive state can be identified in a timely manner and enter the differentiated recovery process, thereby ensuring the accuracy of the judgment while taking into account the tolerance of normal business.

[0082] In an optional implementation, when the primary anomaly category is one with log message output, the anomaly determination is based on the joint deviation of log content diversity measures and log interval measures from the corresponding preset benchmarks, combined with log content anomaly indicators, to comprehensively determine the running status of the target test task. The comprehensive determination is executed according to a preset anomaly priority; when multiple anomaly conditions are met simultaneously, the application is prioritized as unresponsive. Thus, by performing joint deviation detection on diversity and interval measures and prioritizing the determination of application unresponsiveness based on anomaly priority, not only can fuzzy anomaly perception be transformed into objective numerical evaluation, but misjudgments caused by fluctuations in a single indicator can also be avoided, thereby improving the accuracy of anomaly determination.

[0083] Optionally, the preset benchmark is used to provide a quantitative reference for measuring log content diversity and log interval, and can be dynamically adjusted according to the type of the program being tested. Optionally, the preset benchmark may include at least one of a first threshold corresponding to the log content diversity measure, a second threshold corresponding to the log interval measure, and a third threshold corresponding to the log content anomaly indicator. Considering that different game categories have significant differences in log output patterns, for example, fast-paced action games generate far more log entries per unit time than strategy games, and their information entropy values ​​are usually at a higher level, the preset benchmark is not a fixed constant, but a variable that can be dynamically adapted according to the specific type of the tested object and the historical log output frequency. As a possible implementation, the system can collect information entropy samples of the same type of game during its historical stable operation phase, calculate the percentile of its statistical distribution as the initial value of the first threshold; furthermore, this initial value can be fine-tuned by combining the actual average information entropy of the current test task in the previous few time windows, so as to better fit the operating characteristics of the tested version.

[0084] Optionally, the aforementioned preset benchmarks can also be trained and optimized using machine learning methods to achieve adaptive updates of the thresholds. Specifically, the system can periodically collect a large amount of log data from historical test tasks, and testers or automated labeling tools can label each time window as running normally, stuck, crashed, or logically abnormal. Based on this, classification algorithms such as decision trees, logistic regression, or support vector machines are used to automatically learn the threshold combination that maximizes the accuracy of anomaly detection.

[0085] Optionally, the aforementioned joint deviation reflects the overall deviation of log content diversity and log interval metrics from their respective preset benchmarks, rather than isolated anomalies in a single dimension. When a game client gets stuck in an infinite loop but still outputs logs, it often exhibits the dual characteristics of highly similar log content and abnormally dense output frequency. In this case, simply reducing information entropy or shortening the interval is insufficient to accurately characterize the severity of the anomaly. By jointly evaluating the deviation of the two metrics, occasional log jitter or brief loading pauses can be effectively filtered out. In one implementation, the aforementioned joint deviation is calculated using the following quantification model: Joint deviation score S = W1·max(0,(θ_H) H) / θ_H)+W2·max(0,(θ_M M) / θ_M), where H is the log content diversity measure within the current preset time window, M is the log interval measure (median), θ_H and θ_M are their corresponding preset benchmarks, and W1 and W2 are preset weight coefficients with W1+W2=1; when S exceeds the preset joint threshold, the joint deviation is determined to be valid. It should be understood that the above weight allocation can be a preset fixed ratio, or it can be optimized in reverse based on historical false alarm samples, and this disclosure does not limit it in this way.

[0086] Optionally, log content anomaly indicators are used to quantify the occurrence density of abnormal semantic components in a dedicated log sequence, providing fault clues for comprehensive judgment. Optionally, the aforementioned log content anomaly indicators can be expressed as the frequency or density ratio of predefined abnormal keywords within a preset time window. These predefined abnormal keywords may include, but are not limited to, terms indicating runtime errors, terms indicating null references, and terms indicating fatal faults. In one implementation, the system can pre-construct an abnormal keyword dictionary. When a log entry containing any term from the dictionary is detected in the dedicated log sequence, that entry is counted. Subsequently, the number of abnormal entries is divided by the total number of log entries within the window to obtain a ratio, which serves as the specific quantitative result of the aforementioned log content anomaly indicators. It should be noted that the keyword matching can employ either precise string matching or regular expression matching to cover error log formats with variable stack information. Furthermore, to avoid excessive alerts caused by one or two occasional error logs, the aforementioned preset benchmark is typically set to a threshold value of 30% or higher. Logical anomaly judgment is only triggered when the error density increases significantly, thus balancing sensitivity and stability.

[0087] Optionally, a preset exception priority is used to determine the execution order and priority of judgment rules when multiple exception conditions are met simultaneously. Optionally, the preset exception priority can be represented as one or more sequentially executed decision chains. When logs within a window simultaneously meet multiple exception judgment threshold conditions, the system executes only the judgment result of the highest priority rule according to the preset priority level, and does not continue with subsequent lower priority branch judgments. In one implementation, if the information entropy within a certain time window is lower than the corresponding preset benchmark, the average log interval is lower than the corresponding preset benchmark, and the frequency of error keywords is higher than its corresponding preset benchmark, then according to the preset exception priority, the judgment rules for resource deadlock are executed before the judgment rules for logical exceptions, thereby classifying the current state as application unresponsiveness rather than a logical exception. The reason for setting the application unresponsive state as a high priority is that in engineering practice, deadlocks or infinite loops often cause error handling code to be repeatedly triggered, resulting in a large number of error keyword logs. If the current use case is skipped and the logic exception is prioritized, the real deadlock failure that requires process intervention will be left behind. By prioritizing the handling of the application unresponsive state, it can be ensured that the recovery process enters the soft recovery or restart path in a timely manner, effectively reducing the risk of failure propagation.

[0088] In an optional implementation, the comprehensive determination further includes: if log messages are output in the dedicated log sequence within a preset time window, and the log content diversity metric is lower than its corresponding preset benchmark while the log interval metric is not lower than its corresponding preset benchmark, then after waiting for a predetermined time, the log content diversity metric and the log interval metric are recalculated to confirm whether the determination of application unresponsiveness should be maintained. Thus, by introducing a delayed recalculation mechanism when content diversity is low but the log interval is not shortened, misjudgments in scenarios of normal game loading or periodic log sparsity are effectively avoided, making the determination of application unresponsiveness more robust and reliable.

[0089] Optionally, the aforementioned predetermined time can be a fixed duration parameter or an adaptive waiting time that is dynamically adjusted according to the running stage of the program under test. After waiting for the predetermined time, the system recalculates the log content diversity metric and the log interval metric for the exclusive log sequence of the target test task. If the recalculated log content diversity metric is still lower than its corresponding preset benchmark and the log interval metric has not recovered to above its corresponding preset benchmark, the determination that the application is unresponsive will be maintained; otherwise, the abnormal warning will be revoked.

[0090] Optionally, the aforementioned pre-set time can be flexibly set based on the type of program being tested, the current stage of the test task, and the historical log output patterns. For example, in the level resource pre-loading stage of a racing game, the game engine typically only outputs fixed resource loading progress logs during this stage, causing a temporary decrease in the log content diversity metric. However, the time interval between adjacent logs does not show a significant shortening trend. If the application is immediately judged as unresponsive, there is a high risk of misjudgment. Therefore, the pre-set time is set to 30 seconds. After 30 seconds, the calculation is repeated. If the diversity metric rises above the preset benchmark due to the start of the battle, the warning is canceled. If it remains low and the interval metric shortens further, the judgment is maintained. In another example, in the battle settlement stage of a turn-based strategy game, if the diversity metric is temporarily lower than the preset benchmark due to only outputting repetitive settlement score logs in the window, while the log interval remains at a normal level of 2 to 3 seconds, the pre-set time is set to 45 seconds. After the time expires, the calculation is repeated. If a new skill trigger template appears due to entering the next round, the warning is canceled; otherwise, the application unresponsive judgment is maintained. It should be noted that the above numerical ranges are merely examples, and this disclosure is not intended to limit the method for determining the waiting time. In actual implementation, the time can also be dynamically adjusted according to the characteristics of the game category to which the tested program belongs, for example, setting the scheduled time to a value that matches the historical average settlement time.

[0091] Optionally, after the aforementioned waiting period expires, the system will recalculate the log content diversity metric and log interval metric for the specific log sequence of the target test task. The log data source for this recalculation can be a new window of data extended by the same length from the original preset time window, or all log messages generated within the cumulative interval from the first anomaly trigger point to the current moment. Through recalculation, the system can obtain the latest temporal characteristics of the target test task during the delayed observation period, thereby effectively verifying the anomaly indicators obtained in the previous calculation. If the recalculated log content diversity metric is still lower than the corresponding preset benchmark, and the log interval metric shows a further shortening or continued anomaly, the system will maintain the application unresponsiveness judgment; conversely, if the log content diversity metric returns to a normal level, or a new log message template appears, the system will cancel the previous anomaly judgment and restore normal operation monitoring. It should be understood that whether to use an extended window or a cumulative window during recalculation can be determined comprehensively based on the log output density of the tested program and the real-time requirements of detection.

[0092] Optionally, the aforementioned verification mechanism aims to avoid misjudging an application's unresponsive state due to accidental fluctuations in a single sampling window in certain boundary scenarios. Specifically, when the tested program is in a transient silent phase such as combat settlement data synchronization, cross-scenario resource switching, or network handshake waiting, the log output may temporarily exhibit high-frequency repetition of a single template or periodic output of a fixed template, resulting in a low diversity metric for the log content extracted in a single instance. However, the time interval does not reach the extreme density of a typical infinite loop scenario. By recalculating and comparing the changing trends of the two metrics after a predetermined waiting time, the system can add a secondary verification protection before maintaining the judgment of application unresponsiveness, thereby ensuring that subsequent restart or recovery processes are only triggered when the abnormal state has persistent characteristics. Furthermore, if a communication channel anomaly or device offline is detected during the waiting period, regardless of the metric result, it is directly judged as an activation failure and a downgrade recovery is performed. This design effectively improves the robustness of anomaly detection and reduces unnecessary recovery operation overhead.

[0093] In step S250, based on the anomaly determination result, a differentiated recovery is performed on the target test task that is adapted to its anomaly type.

[0094] In one implementation, when the anomaly determination result indicates that the target test task is an application unresponsiveness, the system sends a soft recovery command to the program under test and, if it does not take effect, downgrades the execution process to restart; if the anomaly type is a program crash, the process is directly restarted and the current test case execution is resumed after confirmation of readiness; if it is a logical anomaly, the current test case is skipped and the next test point is entered. For example, in a racing game's minimum completion level detection across 200 levels, the system triggers the aforementioned differentiated recovery based on the anomaly type, enabling freezes, crashes, and logical errors to receive layered self-healing processing at the second or minute level, respectively, completing a full traversal of all levels without manual intervention.

[0095] Optionally, differentiated recovery aims to match different self-healing actions, such as soft recovery commands, process restarts, or test case skipping, based on different anomaly manifestations, such as no log output, repetitive log loops, or code errors. Considering the diversity of anomaly forms in game automated testing, process-level crashes are suitable for direct restarts to rebuild a clean runtime environment; application unresponsiveness (such as infinite loops or resource deadlocks) can first try in-game soft recovery commands (such as script commands to force end combat) to avoid the tens of seconds of initialization time caused by restarting; logical anomalies (such as null pointer errors) usually do not affect process stability, so the strategy of recording and skipping is the most lightweight. It should be noted that the combination and execution order of the above recovery actions are only one example, and this disclosure is not intended to limit the specific actions of differentiated recovery. In actual implementation, recovery strategies can also include rolling back to the most recent checkpoint, reducing graphics settings to release resources, or notifying the scheduler to isolate abnormal devices, etc. The specific action selection can be dynamically configured according to the type of the program under test, the distribution of anomaly frequency, and the priority of the test task.

[0096] Optionally, the anomaly determination result serves as a bridge connecting log time-series feature analysis and recovery strategy execution. Its generation process relies on multi-dimensional quantitative analysis of a specific log sequence within a preset time window. In one implementation, the result can be presented as structured data encompassing anomaly category labels, confidence scores, and key log segments. The anomaly category labels are used to directly map to the corresponding recovery branch, and the confidence scores are used to determine whether to perform a secondary confirmation wait when boundaries are ambiguous. For example, when the log content diversity measure approaches zero and the time interval is abnormally shortened, the determination result can output a high-confidence application unresponsiveness; when the frequency of erroneous keywords exceeds a threshold while the log continues to output, the determination result points to a logical anomaly. It should be understood that the specific data format and number of fields in the anomaly determination result can be flexibly adjusted according to the interface design of the monitoring system, and this disclosure does not limit this.

[0097] Optionally, the exception type is used to characterize the category of the abnormal state that occurs during the execution of the target test task, and it is determined based on the joint judgment result of temporal characteristics and process state. Optionally, the exception type may include, but is not limited to, application unresponsiveness, logical exception, program crash, and normal operation. It should be noted that the above classification method is only one example, and this disclosure is not intended to limit the division of exception types. In actual implementation, exception types can be further subdivided into more specific subcategories such as memory leak-type freezes, graphics rendering deadlocks, and network timeout unresponsiveness, in order to match more refined differentiated recovery actions. Furthermore, the determination of exception types not only depends on temporal characteristics such as information entropy, time interval, and error keyword frequency, but can also be dynamically weighted in conjunction with the game category to which the program under test belongs (such as fast-paced action games or strategy games). To avoid misjudgment caused by accidental fluctuations of a single feature, the exception type determination can adopt a multi-round voting or sliding window consistency check mechanism to ensure the accuracy of the recovery strategy triggering.

[0098] In an optional implementation, differentiated recovery includes: sending a soft recovery command to the program under test when the exception type is "application unresponsive"; monitoring the exception resolution status of the program under test to determine whether the soft recovery command is effective; and if it is determined that it is ineffective, reverting to restarting the program under test. In this way, when the application is unresponsive, a lightweight repair is prioritized using the soft recovery command, and process restart is only triggered after the command fails. This preserves the context of the test task to shorten recovery time and avoids the environmental reconstruction overhead caused by frequent restarts.

[0099] In one implementation, when the target test task is determined to be an unresponsive application, the test server first sends a soft recovery command to the program under test to attempt to resolve the deadlock state. After the command is submitted, the system continuously monitors the changes in the log output of the program under test to determine whether the anomaly has been resolved. If no signs of anomaly resolution are detected within the specified time limit, the degraded execution logic is triggered to directly restart the program under test, thereby restoring the test execution process with minimal environmental cost.

[0100] Optionally, the soft recovery command aims to remove the runtime blockage of the program under test through lightweight software-level operations, avoiding the context loss and resource reconstruction overhead caused by directly restarting the process. Optionally, the soft recovery command can be a Lua script command injected into the script execution environment of the program under test. For example, it can trigger the normal closure of the settlement interface by calling a business function containing panel exit logic, or it can trigger the exit process of a combat scene to end the current abnormal round. This command is sent through a long-lived connection established between the test framework and the program under test, and then executed directly in the script virtual machine within the game process. It should be noted that the form of the above command is not limited to function calls; it can also be setting specific global variables or triggering internal engine events. Its content can be dynamically constructed according to the business scenario in which the freeze occurred. In actual deployment, to prevent unauthorized device access, two-way authentication of token identifiers and device identifiers can be introduced at the test platform level to ensure that the soft recovery command is only initiated by a legitimate test scheduler.

[0101] Optionally, considering that process-level restarts typically involve multiple time-consuming steps such as forced process termination, client re-initialization, reloading of in-game art and logic resources, and re-injection of test scripts, the overall recovery time often reaches more than ten seconds or even longer, and will result in the complete loss of the execution context of the current level or character state, this embodiment sets the soft recovery command as a pre-repair layer before process restart. Furthermore, the execution strategy of the soft recovery command itself can also be multi-layered and progressive: as one possible implementation, it can prioritize calling the regular interface exit logic to ensure data integrity; as another implementation, if the regular exit does not receive log feedback within a predetermined waiting period, a more forceful battle termination signal is directly triggered at the script level. It should be understood that if the above software-level operations fail to restore the normal diverse output of the dedicated log sequence, it is considered a soft recovery failure, thereby triggering downgraded execution. In addition, the semantic content of the soft recovery command can be flexibly adjusted for different game categories, and this disclosure does not limit this.

[0102] Optionally, in the event that the soft recovery command fails to resolve the exception, the test program is forcibly restarted to achieve a final recovery fallback, ensuring that the testing process is not blocked for a long time. Optionally, the fallback restart refers to a fallback operation where, in the case that the soft recovery command fails to resolve the application's unresponsive state as expected, the test server abandons the software-level repair path and instead forcibly terminates and restarts the test program process through the device management service. This process can be implemented by the test server sending a process restart request to the device management service deployed locally on the device under test. Upon receiving the request, the device management service first forcibly terminates the current game process, then restarts the game client via a command-line tool. Subsequently, the test server periodically polls the health check interface of the device management service until it confirms that the game instance is in a ready state. In one possible implementation, if the soft recovery command does not cause any changes in the test program's log stream within a predetermined waiting period, the final process restart is directly triggered. It should be understood that the triggering conditions for the aforementioned downgrade execution are not limited to waiting period expiration, but can also be extended to situations such as detecting that the tested program outputs a specific error flag after receiving a soft recovery instruction, or that the dedicated log sequence continuously maintains a low information entropy state. This disclosure does not limit these situations.

[0103] In an optional implementation, determining whether the soft recovery command has taken effect includes: detecting the status of the communication channel that sent the soft recovery command; if the communication channel throws an exception, the command is deemed to have failed to take effect; if the communication channel does not throw an exception, the test program's exception resolution status is monitored; if exception resolution status is detected, the command is deemed to have taken effect successfully; if exception resolution status is not detected, the dedicated log sequence is checked to see if a completion log indicating the normal completion of the current test task is captured; if a completion log is captured, the test task is deemed to have automatically ended; if no completion log is captured, the command is deemed to have failed to take effect. This not only allows for immediate interception of communication failures during the command issuance phase but also enables fallback confirmation during the silent period through log semantics, effectively preventing situations where a task has automatically ended but is mistakenly judged as a recovery failure.

[0104] In one implementation, when the test server sends a soft recovery command to the program under test to forcibly end the current battle, the system first checks whether the communication channel carrying the command throws an exception. If an exception is thrown, the system immediately determines that the recovery has failed and prepares for a downgraded restart. If the channel is normal, the system proceeds to the next level of judgment, monitoring whether the program under test has resumed log output to indicate that the exception has been resolved. If the recovery is detected, the system determines that the recovery has been successful. If the exception has not been resolved, the system further traverses the dedicated log sequence to check whether a completion log indicating that the current test task has ended normally has been captured. If a completion log is captured, the system determines that the task has ended automatically and no additional recovery is needed. If a completion log is not captured, the system ultimately determines that the recovery has failed and triggers the restart process.

[0105] Optionally, communication channel status detection aims to perform pre-availability diagnostics on the link carrying the soft recovery command, identifying connection-level anomalies before the command is actually delivered. Optionally, the aforementioned communication channel can be a communication link established between the test framework and the program under test (DUT) based on a long-connection protocol. It should be noted that detecting whether the communication channel throws an anomaly essentially captures errors such as device offline or connection interruption when the soft recovery command is submitted to the DUT's test framework. Considering that the target device may experience momentary disconnection or fail to re-register after a process restart in unattended testing scenarios, pre-checking the communication channel status at the moment the command is issued allows for differentiation between two fundamentally different failure modes: command not delivered and command delivered but not effective, at the delivery stage. If the communication channel throws an anomaly, it means the soft recovery command did not successfully reach the DUT. In this case, there is no need to enter the subsequent lengthy anomaly resolution waiting phase; instead, it is directly determined as a failure to take effect and a degraded restart is triggered, thus avoiding the waste of monitoring resources when the connection is broken and significantly reducing the overall fault recovery time.

[0106] Optionally, communication channel status detection, besides relying on exceptions directly thrown by the underlying communication framework, can also manifest as keyword matching of error messages returned by the communication framework, or verification of the return status code of the script submission interface. For example, when the test server sends a soft recovery command in Lua script form to the target device through the test scheduling platform, if the returned message body contains keywords indicating that the device is offline, or if no delivery confirmation is received within the specified timeout window, it is also considered a communication channel exception and the operation fails. Furthermore, this detection can also continuously monitor channel feedback for a very short buffer period after script submission, thus being compatible with edge scenarios where delays and errors occur due to momentary network jitter. It should be understood that the above description of communication channel types is only one example. In actual implementation, communication links based on TCP (Transmission Control Protocol) direct connection, HTTP (Hypertext Transfer Protocol) short polling, or other proprietary protocols can also be used, and this disclosure is not intended to limit this.

[0107] Optionally, the aforementioned completion log refers to key log messages with stage termination semantics captured in a dedicated log sequence. These messages are used to confirm, from a log semantic perspective, that the current test task has naturally terminated when the abnormal resolution state is not directly detected. Considering that after a soft recovery command is issued, the program under test may not represent abnormal resolution in the usual log output manner, but instead directly advance to the final stage of the test task and silently exit, it becomes particularly necessary to identify whether the task has naturally terminated through log content. For example, in an automated combat test scenario, if a soft recovery command successfully allows the stalled combat logic to advance, the program under test may directly output messages such as exiting the settlement panel or combat settlement completion without generating a recovery heartbeat. When the system retrieves such completion logs in the dedicated log sequence, even if no obvious abnormal resolution state is detected, it can determine that the current test task has logically terminated automatically. This prevents unnecessary forced intervention on tasks that have already terminated naturally, avoiding the introduction of additional unstable factors by the recovery action itself. Furthermore, the completion log's fallback capture mechanism provides a reliable basis for judgment for game clients with slow responses or sparse log output, making the decision-making chain of the recovery strategy more complete and comprehensive.

[0108] In an optional implementation, the program under test is re-run and restarted in a degraded manner as follows: when the error type indicates that the target device is offline and the process is currently within a preset recovery window after restarting, a limited number of timed retries are performed on the soft recovery command. If the timed retry fails, the program under test is restarted. When the error type indicates that the target device is offline and the process is not currently within the preset recovery window, or when the error type is not related to device offline, the program under test is restarted. This approach, by differentiating error types and pre-setting a recovery window, provides a limited number of timed retries during the unstable phase immediately after restarting, avoiding invalid process restarts caused by momentary communication disturbances. For non-device offline type substantive anomalies, a direct degraded restart is performed, balancing recovery fault tolerance and test execution efficiency.

[0109] Optionally, "target device offline" does not generally refer to the device under test being powered off or de-energized. Instead, it specifically refers to a communication failure signal caused by network jitter, process initialization delays, or transient unreachability during client long-connection reconstruction when the test server sends control commands to the test framework on the device under test via a full-duplex long-connection protocol. In one implementation, this error type can be determined by capturing connection timeout exceptions thrown by the WebSocket or SocketIO communication layer, unresponsive heartbeat detection, or service unreachability flags in HTTP status codes. Considering that within seconds of a process restarting, the game client may be in the resource loading, Lua virtual machine initialization, or test framework re-registration phase, the communication channel is not yet fully ready, making it highly susceptible to momentary target device offline signals. Directly restarting the process during this phase not only fails to solve the fundamental problem but also increases system overhead and prolongs overall test time. Therefore, treating target device offline as an error type requiring refined processing in conjunction with the time dimension helps to achieve hardware and software layering in the recovery strategy, avoiding overly aggressive process-level interventions for transient and self-healing communication disturbances.

[0110] Optionally, a preset recovery window period is used to define a specific time interval starting from the last restart of the program under test. This serves as a temporal condition for determining whether timed retries are allowed. Its duration can be dynamically configured based on the startup time of different game engines, device hardware performance, and the complexity of level resources. In one implementation, for test hosts equipped with solid-state storage and high-performance CPUs, this window period can be set to a shorter interval of thirty seconds to two minutes; while for mobile test devices that need to load a large amount of art resources from a remote server, this window period can be extended to three to five minutes to ensure that the core subsystems within the game complete initialization. Within this window period, if the soft recovery command fails to be sent due to the target device going offline, the system will determine that the offline status is highly likely to be a transient phenomenon after a restart, thus initiating a limited number of timed retries. Once this window period is exceeded, even if the target device goes offline again, the system will treat it as a persistent communication failure or device disconnection, and will not perform retries but will directly downgrade to a restart. This time-domain layering design can accommodate the natural instability phase at the beginning of the process, while also preventing test resources from being wasted due to invalid retries in scenarios where the device is unreachable for a long time through strict time boundaries.

[0111] Optionally, the limited number of retries is an integer threshold determined based on the overall time requirements of the test task, the concurrency scale of the test equipment, and the historical success rate of the anomaly recovery link. This value is not fixed but can be adaptively adjusted according to the version iteration of the program under test or fluctuations in the network quality of the test environment. In one implementation, this number can be set to three, with a three-minute interval between each retry, ensuring that the entire retry link will not exceed a total waiting limit of ten minutes even in the worst-case scenario. In another implementation, for smoke test scenarios with higher timeliness requirements, this number can be reduced to one and the interval shortened to thirty seconds. It should be noted that the key function of setting a limited number of retries is to prevent the test scheduler from being suspended for a long time while waiting for a response from a certain device, thereby blocking the queuing execution of subsequent test cases. When the number of retries is exhausted and the anomaly is still not successfully resolved, the system immediately terminates the waiting process of that branch and executes the degradation path of process restart, thus ensuring global test throughput and scheduling determinism while also considering the fault tolerance and recovery opportunities for some self-healing anomalies.

[0112] Optionally, timed retries refer to the test server, after detecting that the target device is offline and determining that the retry conditions are met, not immediately and continuously sending soft recovery commands, but instead performing phased recovery attempts at preset time intervals. In one implementation, the test server executes the second, third, and fourth command delivery at the first, fourth, and seventh minutes after the initial failure to send the soft recovery command, respectively. Before each delivery, the reachability of the communication channel is confirmed by a short-pulse heartbeat probe; if the probe is successful, a new round of command transmission is triggered; if the probe fails, it is directly counted as an invalid retry and waits for the next round. In another implementation, the timed retry interval can also adopt an exponential backoff strategy, i.e., the first interval is thirty seconds, the second is one minute, and the third is two minutes, with the exponentially increasing waiting time matching the self-healing cycle of transient faults of different severity. This non-continuous retry method with cooling intervals can, on the one hand, effectively avoid command loss caused by network congestion or client thread blocking, giving the test framework and game client enough time to complete internal state synchronization; on the other hand, it also avoids the additional load impact on the device under test caused by high-frequency retries.

[0113] Optionally, the "non-device offline" type covers all exception categories other than communication link interruption that cause the soft recovery command to fail. Optionally, the non-device offline type includes, but is not limited to, test script execution exceptions, game engine internal logic errors, null pointer references, array out-of-bounds errors, memory overflows, or the soft recovery command itself being intercepted by the game's security mechanisms. In one implementation, when the test server sends a soft recovery command to the program under test, if the communication channel does not throw an offline exception, but the program under test returns a clear Lua script execution error code, or a fatal error keyword indicating an internal engine crash appears in the dedicated log sequence, then the failure is classified as a non-device offline type. Unlike target device offline, this type of error usually indicates that the internal logic of the program under test is substantially damaged or in an unpredictable state. Simply waiting or repeatedly sending the same command often cannot fix the fundamental problem; therefore, when the error type is determined to be a non-device offline type, regardless of whether it is currently within the preset recovery window, the system directly skips the retry phase and performs a downgrade operation to restart the program under test. This strategy of classifying and handling errors based on semantics ensures that the response paths for different types of anomalies match the actual repair needs, avoiding unnecessary retries on logical anomalies.

[0114] In an optional implementation, differentiated recovery further includes: when the exception type is a crash of the program under test, restarting the program under test, and resuming the execution of the current test task after confirming that the program under test is ready. This way, after the program under test restarts due to a crash, resuming task execution only after confirming its readiness avoids premature script activation leading to secondary failures, ensuring the continuity and reliability of long-term unattended testing.

[0115] Optionally, confirming the readiness of the program under test aims to determine whether the restarted client is in a state that can accept test instructions through periodic status probing. Optionally, the implementation of the above readiness confirmation is not limited to a single interface polling mechanism. In one specific implementation, the test server sends a process status query request to the physical machine with the device management service deployed, and the service returns the running status identifier of the target game instance; in another implementation, the test server directly and periodically probes the communication response capability of the program under test through a full-duplex long-connection communication channel. If several consecutive probes successfully return the agreed message, it is considered ready. Considering the significant differences in initialization time between different game engines, this readiness determination can carry multiple layers of verification logic. For example, in addition to verifying the existence of the target executable file in the operating system process list, it further checks whether the core subsystems within the game have been loaded, thereby avoiding triggering subsequent script injections before the screen is fully rendered or the logic is ready, and preventing chain test failures caused by premature driving.

[0116] Optionally, resuming the execution of the current test task aims to re-queue the test task and distribute it to a ready device, maintaining the continuous progress of unattended testing. Optionally, resuming the execution of the current test task does not refer to resuming interrupted tasks at the script bytecode level, but rather to rescheduling the test task while preserving its context boundaries at the business logic level. In one implementation, the test server maintains a queue of tasks to be tested. When the program under test on a device restarts due to a crash and is confirmed to be ready, the system retains the original configuration of key parameters such as the level number, role level, and task identifier, re-queues the complete task, and assigns it to the same recovered device for execution again. In another implementation, the system can also migrate tasks to idle devices in the same batch to balance the overall load. It should be noted that regardless of the redistribution method used, the re-distributed test script must completely contain the entire process from task initialization to entering the target level, rather than resuming from a script breakpoint in the middle. This is because the original script virtual machine context is lost after the process restarts, and there is no intermediate state to connect to. In this way, we can ensure the final coverage of a single test task after a crash, while avoiding the additional burden that complex state persistence mechanisms would place on the system architecture.

[0117] Corresponding to the above method embodiments, this invention provides an automated test anomaly detection and recovery device, see [link to relevant documentation]. Figure 3 According to one embodiment of this disclosure, the device may include: The log embedding and execution module is used to embed a unique identifier corresponding to the test task into the test task and control the execution of the test task so that the program under test outputs log messages carrying the unique identifier. The log routing module is used to establish a communication connection with the program under test, receive log messages, and route the log messages to the independent log buffer of the corresponding test task according to the unique identifier, so as to obtain the exclusive log sequence corresponding to the target test task. The feature extraction module is used to extract at least two types of time-series features from a specific log sequence within a preset time window. The time-series features include log interval measurement and log content diversity measurement. The anomaly detection module is used to determine anomalies in the running status of the target test task based on time-series characteristics, the detected target process status, and the log message output status of the dedicated log sequence. The differentiated recovery module is used to perform differentiated recovery on the target test task according to the anomaly judgment results output by the anomaly judgment module, and the anomaly type should be appropriate.

[0118] In this way, anomaly detection and differentiated recovery based on unique identifier-based traffic splitting and time-series feature extraction achieve precise isolation of multi-task logs, reducing the false alarm rate and recovery time, and improving the efficiency of automated test monitoring. Furthermore, it helps reduce unnecessary restarts triggered by false alarms, lowers computational resource overhead during testing, and improves system resource utilization.

[0119] The specific details of each part of the above-mentioned device have been described in detail in the method section of the implementation plan. For any undisclosed details, please refer to the implementation plan of the method section, and therefore will not be repeated here.

[0120] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0121] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0122] The following is a detailed reference. Figure 4 The diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present disclosure. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 1201, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1202 or a program loaded from memory 1208 into random access memory (RAM) 1203. The RAM 1203 also stores various programs and data required for the operation of the electronic device. The processor 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0123] Typically, the following devices can be connected to I / O interface 1205: input devices 1206 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1207 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 1208 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1209. Communication device 1209 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0124] In particular, according to one embodiment of this disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, one embodiment of this disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication device 1209, or installed from memory 1208, or installed from ROM 1202. When the computer program is executed by processor 1201, it performs the functions defined in the methods described above in various embodiments of this disclosure.

[0125] Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0126] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded over a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0127] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0128] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An automated test anomaly detection and recovery method, characterized in that, The method includes: A unique identifier corresponding to the test task is embedded in the test task, and the execution of the test task is controlled so that the program under test outputs a log message carrying the unique identifier. Establish a communication connection with the program under test, receive the log messages, and distribute the log messages to the independent log buffer of the corresponding test task according to the unique identifier to obtain a unique log sequence corresponding to the target test task; Within a preset time window, at least two types of time-series features are extracted from the dedicated log sequence, including log interval measurement and log content diversity measurement. Based on the aforementioned timing features, the detected target process status, and the log message output status of the exclusive log sequence, anomaly determination is made regarding the running status of the target test task. Based on the anomaly determination results, a differentiated recovery is performed on the target test task that is adapted to its anomaly type.

2. The method according to claim 1, characterized in that, The unique identifier is a globally unique identifier and is embedded in the body of the log message in the form of a structured field.

3. The method according to claim 1, characterized in that, The communication connection is a communication connection established based on a full-duplex long connection protocol. The traffic splitting includes: parsing the text of each log message in the received global log stream, extracting the identification field, and writing the log message whose identification field matches the unique identifier corresponding to the target test task into the independent log buffer corresponding to the target test task.

4. The method according to claim 1, characterized in that, The length of the preset time window is dynamically adjusted according to the type of the object being tested and the historical log output frequency of the target test task, so that the number of log entries in the exclusive log sequence within the preset time window meets the preset statistical quantity requirements.

5. The method according to claim 1, characterized in that, The timing features also include the frequency of occurrence of predefined abnormal keywords; the step of determining anomalies in the running state of the target test task based on the timing features and the detected target process state includes: The proportion of log entries containing the predefined abnormal keywords in the dedicated log sequence within the preset time window is counted. If the proportion of log entries exceeds a preset threshold, it is determined to be a logical abnormality.

6. The method according to claim 1, characterized in that, The log interval metric includes a central tendency indicator based on the sequence of timestamp differences between adjacent log messages in the dedicated log sequence.

7. The method according to claim 6, characterized in that, The central tendency metric includes the median of the time interval sequence between adjacent log messages in the dedicated log sequence.

8. The method according to claim 1, characterized in that, The log content diversity measure includes: Semantic templates are extracted from the log messages in the dedicated log sequence, and log messages that conform to the same semantic template are grouped into the same category; The log messages are normalized to remove variable fields that change with the test instance; The distribution of each category is statistically analyzed, and the weighted concentration index of the occurrence ratio of each template is calculated as a measure of the diversity of the log content.

9. The method according to claim 1, characterized in that, The anomaly determination includes: Based on the log message output status of the dedicated log sequence within the preset time window and the target process status, the first-level anomaly category of the target test task is determined. The first-level anomaly category includes no log message output and log message output. When the first-level exception category is the no-log message output category, the exception determination is that the target test task is a crash of the tested program if the target process does not exist, and that the application is unresponsive if the target process exists.

10. The method according to claim 9, characterized in that, When the first-level anomaly category is the log message output category, the anomaly determination is based on the joint deviation of the log content diversity measure and the log interval measure from the corresponding preset benchmark, and combined with the log content anomaly index, to comprehensively determine the running status of the target test task. The comprehensive judgment is performed according to the preset abnormality priority. When multiple abnormal conditions are met at the same time, the application is given priority to be judged as unresponsive.

11. The method according to claim 10, characterized in that, The comprehensive judgment also includes: If there are log messages output in the dedicated log sequence within the preset time window, and the log content diversity metric is lower than its corresponding preset benchmark and the log interval metric is not lower than its corresponding preset benchmark, the log content diversity metric and the log interval metric are recalculated after a predetermined time to confirm whether to maintain the application unresponsiveness determination.

12. The method according to claim 1, characterized in that, The differential recovery includes: When the exception type is "application unresponsive", a soft recovery command is sent to the program under test. Monitor the abnormal resolution status of the tested program to determine whether the soft recovery command is effective; If the test is determined to be ineffective, the program under test will be downgraded and restarted.

13. The method according to claim 12, characterized in that, The determination of whether the soft recovery command is effective includes: The status of the communication channel that sent the soft recovery command is detected. If the communication channel throws an exception, it is determined that the recovery has failed. When the communication channel does not throw an exception, the exception resolution status of the program under test is monitored. When the exception resolution status is detected, it is determined that the program has been successfully activated. When the abnormal resolution status is not detected, check whether a completion log indicating the normal completion of the current test task is captured in the dedicated log sequence. If the completion log is captured, it is determined that the current test task has automatically ended. If the completion log is not captured, it is determined that the action has failed.

14. The method according to claim 12, characterized in that, The downgrade execution restarts the tested program, as follows: When the error type indicates that the target device is offline and is currently within the preset recovery window period after the process has just restarted, the soft recovery command is retried a limited number of times. If the timed retry fails, the tested program is restarted. When the error type indicates that the target device is offline and is not currently within the preset recovery window, or when the error type is not a device offline type, the tested program is restarted.

15. The method according to claim 1, characterized in that, The differentiated recovery also includes: when the exception type is a crash of the program under test, performing an operation to restart the program under test, and resuming the execution of the current test task after confirming that the program under test is ready.

16. An automated test anomaly detection and recovery device, characterized in that, include: The log embedding and execution module is used to embed a unique identifier corresponding to the test task into the test task and control the execution of the test task so that the program under test outputs a log message carrying the unique identifier. The log routing module is used to establish a communication connection with the program under test, receive the log messages, and route the log messages to the independent log buffer of the corresponding test task according to the unique identifier, so as to obtain a unique log sequence corresponding to the target test task. The feature extraction module is used to extract at least two types of time-series features from the dedicated log sequence within a preset time window. The time-series features include log interval measurement and log content diversity measurement. The anomaly detection module is used to determine the anomaly of the running state of the target test task based on the time-series characteristics, the detected target process state, and the log message output state of the dedicated log sequence. The differentiated recovery module is used to perform differentiated recovery on the target test task according to the anomaly determination result output by the anomaly determination module, which is adapted to the anomaly type.

17. An electronic device, characterized in that, include: Memory stores computer-executable instructions that can be executed by a processor; A processor for executing the computer-executable instructions to implement the method as claimed in any one of claims 1-15.

18. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the method as described in any one of claims 1-15.

Images