An automated testing method, apparatus, system, and medium

Abstract

The application provides an automatic testing method, device, system and medium; comprising: obtaining test requirements, structured test data and front-end code to be tested; task decomposition and structured processing of the test requirements according to test operation types are performed to obtain a structured task tree with execution timing, and the structured task tree comprises sub-tasks of different test operation types; taking the structured task tree as an execution logic framework and the structured test data as case input parameters, an executable Playwright test script of the front-end code is generated by using a staged progressive generation strategy; the executable Playwright test script is run, and a first actual image corresponding to a key time point in a three-dimensional animation scene is collected; a multi-dimensional consistency comparison and analysis is performed on the first actual image and an expected image to obtain a comparison and analysis result, and a visual test report is generated according to the comparison and analysis result; wherein the expected image is a standard three-dimensional animation frame image generated based on a vehicle signal corresponding to the key time point.

Description

Technical Field

[0001] This application relates to the field of computer software testing technology, and in particular to an automated testing method, apparatus, system and medium. Background Technology

[0002] With the development of digital twins and intelligent transportation, WebGL-based 3D visualization front-end systems are widely used in applications such as vehicle operation status monitoring. These systems utilize frameworks like Three.js to construct interactive scenes that include 3D models and animation effects. However, existing automated testing methods struggle to effectively cover the graphics rendering and dynamic behavior verification requirements of such systems.

[0003] Current mainstream automated testing tools such as Selenium and Playwright primarily rely on the DOM structure for manipulation and assertions, only able to control UI elements such as buttons and input boxes. They cannot access or perceive the 3D model state, materials, lighting, and animation logic within the WebGL canvas. Some solutions attempt to verify rendering results through pixel comparison, but are prone to false positives due to rendering differences across devices and browsers. Furthermore, for complex interactive operations (such as rotation and scaling), event listeners and matrix transformation logic must be manually written, resulting in high development and maintenance costs. Therefore, a new automated testing method is urgently needed for the front-end of WebGL-based vehicle 3D visualization animation. Summary of the Invention

[0004] One objective of this application is to provide an automated testing method to solve the problem of inaccurate or failed parking space detection caused by low accuracy of parking space information in related technologies; another objective is to provide an automated testing device; a third objective is to provide a computer device; a fourth objective is to provide a computer-readable storage medium; and a fifth objective is to provide a computer program product.

[0005] To achieve the above objectives, the technical solution of this application embodiment is implemented as follows: This application provides an automated testing method, the method including: Obtain test requirements, structured test data, and front-end code to be tested; wherein, the front-end code is used to construct a 3D animation scene containing a 3D vehicle model based on the vehicle's basic information and the vehicle signals associated with each function of the vehicle, and to drive the 3D vehicle model to perform the corresponding 3D animation effect in response to the state changes of the vehicle signals. The test requirements are broken down and structured according to the test operation type to obtain a structured task tree with execution sequence. The structured task tree includes sub-tasks of different test operation types. The sub-tasks include at least one of the following: DOM operation sub-tasks, Three.js interactive operation sub-tasks, video synchronization operation sub-tasks, and screenshot verification operation sub-tasks. Using the structured task tree as the execution logic framework and the structured test data as test case input parameters, an executable Playwright test script for the front-end code is generated using a phased progressive generation strategy. Run the executable Playwright test script to acquire the first actual image corresponding to the key time point in the 3D animation scene; wherein, the key time point is the moment when the vehicle state of the 3D vehicle model changes; A multi-dimensional consistency comparison analysis is performed on the first actual image and the expected image to obtain the comparison analysis results, and a visualization test report is generated based on the comparison analysis results; wherein, the expected image is a standard three-dimensional animation frame image generated based on the vehicle signal corresponding to the key time point.

[0006] Based on the aforementioned technical means, a large language model is used as the testing agent. By decomposing test requirements into a structured task tree containing multiple types of sub-tasks and with execution sequence, an executable Playwright test script adapted to the front-end 3D visualization scene is automatically generated using a phased progressive generation strategy. This eliminates the need for manual writing of complete automation code, significantly reducing the development threshold and labor costs for automated testing of vehicle 3D animation scenes. By collecting 3D scene rendering images at the moment of vehicle state change as the key time point and combining them with standard 3D animation frame images generated based on vehicle signals to conduct multi-dimensional image consistency comparison, the effectiveness of vehicle signal-driven 3D model animation rendering can be accurately verified, improving the objectivity and reliability of test results. At the same time, it fully covers various test behaviors such as DOM manipulation, Three.js 3D interaction, video synchronization, and screenshot verification, realizing the entire process of requirement analysis, automatic script generation, automated execution, visual verification, and report output. This effectively improves the automation level and test coverage of 3D animation scene testing, and the comparison results are presented intuitively through a visual test report, making it easy for R&D personnel to quickly locate 3D rendering anomalies.

[0007] This application provides an automated testing device, including: The acquisition module is used to acquire test requirements, structured test data, and front-end code to be tested; wherein, the front-end code is used to construct a 3D animation scene containing a 3D vehicle model based on the vehicle's basic information and the vehicle signals associated with each function of the vehicle, and to drive the 3D vehicle model to perform the corresponding 3D animation effect in response to the state changes of the vehicle signals. The processing module is used to decompose and structure the test requirements according to the test operation type to obtain a structured task tree with execution sequence. The structured task tree includes sub-tasks of different test operation types. The sub-tasks include at least one of the following: DOM operation sub-tasks, Three.js interactive operation sub-tasks, video synchronization operation sub-tasks, and screenshot verification operation sub-tasks. The processing module is also used to generate an executable Playwright test script for the front-end code by using the structured task tree as the execution logic framework and the structured test data as test case input parameters, and by adopting a phased progressive generation strategy. The runtime module is used to run the executable Playwright test script; The processing module is further configured to acquire a first actual image corresponding to a key time point in the 3D animation scene; wherein the key time point is the moment when the vehicle state of the 3D vehicle model changes; perform a multi-dimensional consistency comparison analysis on the first actual image and the expected image to obtain the comparison analysis result, and generate a visualization test report based on the comparison analysis result; wherein the expected image is a standard 3D animation frame image generated based on the vehicle signal corresponding to the key time point.

[0008] This application provides an automated testing system, including: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the automated testing method provided in the embodiments of this application.

[0009] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the automated testing method provided in this application.

[0010] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the automated testing method provided in this application. Attached Figure Description

[0011] Figure 1This is a flowchart illustrating the automated testing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an optional automated testing system provided in an embodiment of this application. Figure 1 ; Figure 3 This is a schematic diagram of the behavior flow of an optional test agent provided in an embodiment of this application; Figure 4 This is a flowchart illustrating the automated testing apparatus provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the automated testing system provided in the embodiments of this application. Figure 2 .

[0012] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] This application provides an automated testing method applied to an automated testing system. This method enables the perception of the 3D model state within a WebGL canvas, achieving fully autonomous closed-loop testing of the entire front-end process of WebGL-based vehicle 3D visualization animation, thus improving testing efficiency. The automated testing system may include mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, and ultra-mobile personal computers (Ultra). Mobile personal computers (UMPCs), netbooks or personal digital assistants (PDAs), servers, network attached storage (NAS), personal computers (PCs), etc. (See reference) Figure 1 As shown, Figure 1 This application provides a schematic diagram of the implementation process of an automated testing method, which may include the following steps: Step 101: Obtain test requirements, structured test data, and the front-end code to be tested.

[0015] The front-end code is used to construct a 3D animation scene containing a 3D vehicle model based on the vehicle's basic information and the vehicle signals associated with each of the vehicle's functions. It also responds to changes in the state of the vehicle signals and drives the 3D vehicle model to perform corresponding 3D animation effects.

[0016] In this embodiment, the test requirements can be user requirements input in natural language to test the WebGL-based vehicle 3D animation front-end system. For example, verifying whether the vehicle trajectory animation and the projection of the monitoring video on the 3D road are synchronized, or opening the vehicle visualization interface, unchecking all layers, only checking 'vehicle' and 'grid', then searching for and selecting a specific topic such as / vehicle / trajectory, playing a 30-second animation, and finally comparing the captured actual image with the expected image to determine whether the rendering is correct. The aforementioned monitoring video can be video images of vehicle movement captured by onboard cameras deployed on the vehicle and / or cameras around the vehicle.

[0017] In this embodiment of the application, the structured test data can be a standardized set of test parameters organized in the form of tables (such as Excel or JSON). The structured test data may include, but is not limited to, one or more of the following: functional module name, Uniform Resource Locator (URL) address for visualization data, functional description, test plugin name involved in each functional module, topic name used to characterize vehicle status messages, total playable duration of animation, expected image storage path, actual image storage path, video synchronization offset used for Canvas video alignment, etc. The test structured data is used to control the test process and serve as the context input for the large language model to generate scripts.

[0018] In this embodiment of the application, the front-end code to be tested is a vehicle 3D animation front-end system developed based on WebGL technology. The core logic of the front-end system is implemented through the Three.js library. It can receive basic vehicle information (such as vehicle model, color, size) and vehicle signals associated with various vehicle functions (such as vehicle speed, steering angle, and light status) as input. It dynamically constructs a WebGL 3D rendering scene on the WebGL canvas, that is, a 3D animation scene of the 3D vehicle model. When the signal status changes, it triggers the 3D vehicle model to perform the corresponding 3D animation effects, such as wheel rotation, light flashing, and vehicle posture adjustment.

[0019] In this embodiment, the front-end code implementation may include: creating a scene, camera, and renderer using Three.js; loading 3D model resources; binding vehicle signals to the animation controller; and listening for signal update events to call the animation play or pause methods of the 3D model, thus achieving real-time synchronization with the actual vehicle's operating state. For example, when a signal indicating a vehicle speed of 60 km / h is received, the front-end code will start a wheel rotation animation and adjust the animation rate according to the speed value; when a signal indicating the left turn signal is on is received, the left turn signal flashing animation will be activated. This mechanism enables the 3D animation scene to accurately reflect the actual operating state of the vehicle, providing a verifiable visual output basis for automated testing.

[0020] In this embodiment, the automated testing system obtains test requirements from the user or test management platform, reads structured test data files (such as local Excel or remote API interfaces), and loads the front-end code to be tested.

[0021] Step 102: Decompose and structure the test requirements according to the test operation type to obtain a structured task tree with execution sequence.

[0022] The structured task tree includes subtasks of different test operation types. Each subtask includes at least one of the following: DOM manipulation subtasks, Three.js interactive operation subtasks, video synchronization operation subtasks, and screenshot verification operation subtasks.

[0023] In this embodiment of the application, the test operation type can be different categories of operations involved in the test process, including but not limited to DOM operations, Three.js interactive operations, video synchronization operations, and screenshot verification operations. The test operation type reflects the technical implementation dimension of the test task.

[0024] In this embodiment, DOM manipulation subtasks include, but are not limited to, opening a menu, entering keywords, and selecting layers; Three.js interactive manipulation subtasks include, but are not limited to, starting an animation player, controlling the timeline, and adjusting the camera angle; video synchronization manipulation subtasks may include ensuring that the Canvas video frame and the 3D animation keyframe are aligned at the specified timestamp; screenshot verification manipulation subtasks may include: capturing the actual image displayed on the WebGL canvas at a specified time point, i.e., the displayed three-dimensional (also known as 3D) animation image, and performing image consistency comparison.

[0025] In this embodiment, the structured task tree can be a data structure that decomposes multiple subtasks into multiple nodes with hierarchical relationships and execution order according to the test process. Each node represents a specific subtask (such as opening a menu, playing an animation, or comparing screenshots) and contains metadata such as task type, dependencies, and parameter configuration. There are clear parent-child relationships and sequential orders between nodes in the structured task tree, ensuring that the test script generation process follows correct business logic. For example, in trajectory playback testing, the structured task tree may contain a root node for starting the test. Branches under this root node include sub-nodes for loading the page, deselecting all, searching for a topic, selecting the vehicle, playing an animation, capturing images, and comparison verification. These nodes are connected through dependency chains to form a complete execution path. In this way, multi-step tasks can be accurately decomposed and a reasonable execution sequence can be established, thereby efficiently generating a structured task plan and laying the foundation for subsequent code generation.

[0026] In this embodiment, the task decomposition and timing processing of test requirements can be accomplished by calling a test agent through a large language model. Large language models include, but are not limited to, Qwen3, Claude, and GLM models. The large language model, by calling the test agent, performs semantic parsing and task decomposition of test requirements according to the test operation type, obtaining multiple sub-tasks such as DOM operation sub-tasks, Three.js interactive operation sub-tasks, video synchronization operation sub-tasks, and screenshot operation sub-tasks. These sub-tasks are then structured to obtain a structured task tree with an execution sequence. Thus, task decomposition, through the semantic understanding capabilities of the third large language model, transforms ambiguous and complex natural language instructions into a structured, executable task tree, significantly reducing the workload of manually writing test cases. This achieves automated planning and modular decomposition of the test process, improving the efficiency and accuracy of test design, and is particularly suitable for frequently changing business scenarios, enabling rapid adaptation to new requirements or functions.

[0027] In some embodiments, test requirements are decomposed and structured according to test operation types to obtain a structured task tree with execution sequence. This can be achieved through the following steps: extracting key semantic elements from the test requirements; wherein, the key semantic elements include at least one of test object, execution action, and spatiotemporal constraint relationship; decomposing the test requirements hierarchically according to the test operation type based on the key semantic elements to obtain multiple sub-tasks; processing the multiple sub-tasks according to hierarchical relationship and execution order to generate a structured task tree.

[0028] In this embodiment of the application, key semantic elements can be the core elements parsed from the test requirements, which are used to guide the generation and execution of subsequent test tasks. Key semantic elements constitute the basis for the test agent to understand user intent.

[0029] In this embodiment, key semantic elements include: test object, execution action, and spatiotemporal constraints. The test object can be a specific entity or functional module in the system under test, such as a menu, animation player, surveillance video, or 3D animated images displayed on a WebGL canvas. The test object is the direct target of the test operation. The execution action can be the specific operation to be performed on the test object, such as playing, taking a screenshot, or unchecking. The execution action defines the dynamic instructions in the test process. The spatiotemporal constraints describe the limitations of the test operation in the time dimension (e.g., playing for 30 seconds) and the spatial dimension (e.g., on a 3D road), ensuring that the test behavior conforms to the business logic and scenario context.

[0030] In this embodiment, hierarchical decomposition refers to dividing the overall testing requirements into multiple independent subtasks that can be executed in parallel or sequentially, based on key semantic elements and operation types. Each subtask has a clear goal, input, output, and execution logic, forming a task tree structure.

[0031] In this embodiment, after obtaining the test requirements, a large language model can be used to semantically parse the test requirements. A pre-defined Prompt project guides the large language model to extract one or more of the following from the test requirements: test objects, actions performed on the test objects, and spatiotemporal constraints. This transforms the fuzzy natural language requirements into computable structured information. Furthermore, based on the extracted key semantic elements, the original test requirements are layered and decomposed according to pre-defined test operation types, resulting in subtasks corresponding to different test operation types. This decomposes a single complex requirement into multiple clear, independent, and executable subtasks, thereby improving the accuracy and maintainability of the generated test script and supporting high-precision, high-coverage automated testing in multimodal fusion scenarios. Finally, the multiple subtasks are processed according to hierarchical relationships and execution order to generate a structured task tree. This allows for the establishment of a reasonable execution sequence for multiple tasks based on hierarchical relationships and execution order, enabling efficient generation of structured task planning and laying the foundation for subsequent code generation.

[0032] In one feasible scenario, we will take the test requirement of verifying whether the projection of a vehicle trajectory animation and a surveillance video onto a 3D road is synchronized as an example. A large language model performs semantic parsing of the test requirement, automatically identifying the test objects as the vehicle trajectory animation and the surveillance video projection, the action to be performed is to verify synchronization, and the spatiotemporal constraint relationship is synchronized playback on a 3D road. Further, based on key semantic elements, the test requirement is layered and decomposed according to the test operation type, resulting in three independent executable sub-tasks: controlling the model to move along the trajectory under the Three.js interactive operation; obtaining the timestamp of the video frame and the timestamp of the aforementioned animation frame under the video synchronization operation, and calculating the offset; and capturing actual images at multiple key frames for comparison under the screenshot operation. Finally, these sub-tasks are processed according to hierarchical relationships and execution order to generate a structured task tree.

[0033] In one feasible scenario, taking the test requirement as an example, we open a vehicle visualization interface, deselect all layers, and only select 'Automobile' and 'grid'. Then, we search for and select a specific topic such as / vehicle / trajectory, play a 30-second animation, and finally compare the captured actual image with the expected image to determine if the rendering is correct. The large language model performs semantic parsing on the test requirement, automatically identifying the test objects as menus, animation players, timelines, monitoring video frames, and 3D animation keyframes. The actions to be executed can be deselecting all in the menu, entering keywords, selecting target items, starting the task player, controlling the timeline, and capturing 3D animation images on the WebGL canvas. The spatiotemporal constraints are time alignment between monitoring video frames and 3D animation keyframes. Furthermore, based on key semantic elements, the test requirement is layered and decomposed according to the test operation type, resulting in four independent executable subtasks: DOM operation subtasks: open menu → deselect all → enter keywords → select target items → close menu. The hree.js interactive operation subtasks include: starting the animation player and controlling the timeline to run for 30 seconds; the Canvas video synchronization operation subtasks include: ensuring that the monitoring video frames are aligned with the key frames of the 3D animation in time; the screenshot and verification operation subtasks include: capturing 3D animation images on the WebGL canvas at specified time points and comparing image consistency; and then processing multiple subtasks according to their hierarchical relationship and execution order to generate a structured task tree.

[0034] Step 103: Using a structured task tree as the execution logic framework and structured test data as test case input parameters, a phased progressive generation strategy is adopted to generate an executable Playwright test script for the front-end code.

[0035] In this embodiment of the application, the execution logic framework is used to define the operation content and execution order of the test, and the input parameters are used to define the test case parameters and verification benchmarks.

[0036] In this embodiment of the application, the executable Playwright test script that generates front-end code includes at least four stages: framework code generation stage, business flow test code generation stage, 3D interaction injection stage, and assertion log enhancement stage.

[0037] In this embodiment, a test agent uses a structured task tree as the execution logic framework and structured test data as the input parameter. A phased progressive generation strategy is adopted to obtain a complete executable Playwright test script that can be directly run in a headless browser environment and is fully adapted to the front-end code under test.

[0038] In some embodiments, a phased, progressive generation strategy may include: Step 131: Build the basic structure code according to the Playwright test framework template.

[0039] The basic structure code includes at least one of the following: page load waiting logic, test data reading module, global logging module, and exception handling mechanism.

[0040] The page loading waiting logic can include front-end browser startup configuration, page redirection logic, and 3D scene resource loading waiting logic, ensuring that scene models, textures, and video streams are all loaded before subsequent operations are performed.

[0041] The test data reading module can directly read structured test data in tabular form, extract multiple parameters from the test data in batches, and support batch execution of multiple test cases.

[0042] The global logging module can automatically record the execution time, input parameters, and execution status of each test operation.

[0043] Among them, the global exception handling mechanism supports automatic retries and exception snapshot retention for abnormal scenarios such as scene loading timeout, rendering lag, and missing elements.

[0044] In this embodiment of the application, during the framework code generation stage, the computer device, through a test agent, constructs basic structure code including page loading waiting logic, test data reading module, global log recording module, and exception handling mechanism according to a preset Playwright test framework template.

[0045] Step 132: Based on the pre-built context knowledge base, using the structured task tree as the execution logic framework and the structured test data as the test case input parameters, fill the basic structure code with business logic and expand its functions to generate Playwright business flow test code that matches the business process of the front-end code.

[0046] In this embodiment, the context knowledge base is used to assist the large language model in understanding test requirements and generating accurate test scripts. In some embodiments, the context knowledge base includes: the official Three.js API documentation, project structure information of the front-end code, 3D scene interaction logic description information, and video synchronization verification rules, which can form multi-dimensional constraints on the test code generation process to ensure that engine method calls, business interface connections, and interaction logic implementations all comply with project specifications and technical standards.

[0047] The Three.js official API documentation is a standardized interface specification document provided by the Three.js graphics library, containing all available classes, methods, properties, and usage examples. For instance, the `controls.rotateLeft()` method of the `Camera` class is used to control camera rotation, and the `material` property of the `Mesh` class is used to set the material map. The Three.js official API documentation provides authoritative guidance for AI to generate Three.js interactive code, avoiding script failures or rendering anomalies due to API misuse. In practical applications, when AI needs to generate operations that rotate the camera to a specific viewpoint, it consults the Three.js official API documentation to obtain the correct function call methods and parameter formats, ensuring the correctness and compatibility of the code.

[0048] The project structure information for the front-end code can be the project directory organization of a WebGL visualization project, including the hierarchical relationship of folders such as src, assets, components, and utils, as well as the paths and dependencies of core modules such as App.vue, Scene.js, and AnimationController.js. This project directory organization helps the AI ​​system understand the code's organizational logic, accurately reference target modules, and adhere to project coding standards. For example, when generating a script to load a vehicle model, the system can locate the models / vehicle.glb path based on this project directory organization and import the resource using an import statement, avoiding resource loading failures due to incorrect paths.

[0049] The 3D scene interaction logic description information can be behavioral rules describing how to interact with 3D objects in a WebGL scene, including mouse dragging and rotation, scroll wheel zooming, keyboard shortcuts to control animation playback, and timeline binding. This 3D scene interaction logic description information typically exists in the project as comments or configuration files, such as `onMouseMove: rotateCamera` and `onWheel: zoomScene` defined in `sceneConfig.json`. By parsing this 3D scene interaction logic description information, AI can translate natural language commands such as rotating the viewpoint and zooming in into specific JavaScript event listeners and matrix transformation code, enabling the automated generation of complex interactive behaviors.

[0050] The video synchronization verification rules are used to verify the technical standards and algorithmic processes for ensuring strict temporal and spatial alignment between Canvas video projection and 3D animation. These rules typically include parameters such as timestamp alignment thresholds (e.g., ±50ms) and frame rate matching requirements (e.g., both at 30fps). During the test execution phase, the video frame timestamps and 3D animation keyframe timestamps are compared according to the video synchronization verification rules to calculate the synchronization error and determine whether the expected accuracy is met. For example, if the video frame timestamp is 10.1s, and the corresponding 3D animation keyframe time is 10.08s, the error is within the threshold, indicating successful synchronization.

[0051] In this embodiment, during the business flow test code generation phase, the test agent uses pre-built contextual knowledge as the basis for interface compliance and business logic, a structured task tree as the execution logic and timing framework of the test process, and structured test data as the input parameters for test cases. It then fills in the already built infrastructure code with business logic and expands its functionality, ultimately generating Playwright business flow test code that matches the business flow of the front-end code under test. Thus, by using a unified framework template as the output standard and combining it with a contextual knowledge base for interface constraints, the generated code has a consistent style and accurate interface calls, avoiding issues such as inconsistent standards and incorrect interface calls that occur during manual development. Simultaneously, using a structured task tree as the logical framework and structured test data as input ensures that the generated scripts are highly matched to the business flow under test.

[0052] Step 133: Inject the engine call hook function into the Playwright business flow test code through Playwright's page script execution interface. The engine call hook function is used to call the 3D rendering engine control code in the front-end code in the page running context.

[0053] In this embodiment of the application, the page script execution interface can specifically be the page.evaluate method of the Playwright framework. This method can pass JavaScript code through to the main execution environment of the browser page to run, sharing objects and engine instances in the global scope of the page. It is the core channel connecting external test scripts and the internal 3D rendering capabilities of the page.

[0054] It is understandable that, since Playwright test scripts run in processes outside the browser, they natively only support interactive operations on HTML DOM elements and cannot directly access the initialized 3D rendering engine instance within the Canvas. To overcome the limitations of automation tools, this embodiment of the application, during the 3D interaction injection phase, uses the page script execution interface provided by Playwright to embed an engine call hook function based on JavaScript in the Playwright business flow test code. When the test script runs to the 3D scene control node, this hook function injects the control logic into the native runtime context of the browser page through the page script execution interface, directly calling the 3D rendering engine control code encapsulated in the front-end code under test, ultimately achieving precise low-level control over the entire 3D rendering process from the rendering layer.

[0055] Step 134: Add multi-dimensional image comparison assertion logic and full-link execution log logic to the Playwright business flow test code to obtain an executable Playwright test script.

[0056] In this embodiment, the multi-dimensional image comparison assertion logic can be designed to address the limitation that WebGL 3D scenes cannot verify rendering effects through DOM elements. It incorporates multiple image verification rules and quantified pass / fail thresholds. When the script executes to a key screenshot verification node, it automatically performs a multi-dimensional comparison between the actual rendered image of the WebGL canvas and the standard expected image, and autonomously outputs a clear judgment result indicating whether the test passed or failed.

[0057] In this embodiment, the full-link execution log logic runs through the entire test execution cycle, automatically recording the execution timestamp, input parameters, running status, verification error value and intermediate product storage path of each operation step, so as to realize the traceability and replayability of the entire test process.

[0058] In this embodiment, during the assertion log enhancement stage, as the final engineering completion step of the phased script generation strategy, the test agent integrates multi-dimensional image comparison assertion logic and full-link execution log logic on the Playwright business flow test code, which has already completed business logic filling and 3D engine call hook injection. The final output is an executable Playwright test script with complete automatic verification capabilities. Thus, through multi-dimensional image comparison assertions, automated judgment of 3D rendering images is achieved, eliminating the need for manual visual verification and forming a complete automated testing loop. The full-link execution log retains complete test evidence, facilitating rapid tracing and review of test anomalies and reducing debugging costs.

[0059] It should be noted that step 134 can be executed after or before step 133, or before step 133. Of course, step 134 and step 133 can also be executed simultaneously.

[0060] As can be seen from the above, the embodiments of this application adopt a phased progressive generation mode. From basic structure to business logic, to the injection of underlying capabilities, and finally to verification enhancement, the entire process is automatically completed by the test intelligence agent. Testers do not need to be proficient in three types of technologies: automated testing, 3D rendering engine, and image algorithm. This significantly reduces the development threshold of 3D scene test scripts and greatly shortens the development cycle.

[0061] Step 104: Run the executable Playwright test script to capture the first actual image corresponding to the key time points in the 3D animation scene.

[0062] The key time point is the moment when the vehicle state of the 3D vehicle model changes.

[0063] In this embodiment, the key time point can be a specific moment when the visual changes of the 3D vehicle model are triggered by changes in vehicle signal state during the animation process, such as the time points when events such as vehicle turning, acceleration, braking, light activation, and door lock closure occur. During testing, the automated testing system acquires images at these time points to ensure the correct execution of the animation logic. For example, when the vehicle accelerates from a standstill to 60 km / h, the automated testing system can acquire images at key frames such as t=5s, t=10s, and t=15s to observe whether the wheel rotation speed matches the speed value; when the vehicle turns left, the automated testing system can acquire images at the start, middle, and end of the turn to verify whether the vehicle body posture transitions smoothly.

[0064] In this embodiment, image acquisition can be performed in a headless Chrome environment using Playwright test scripts. The automated testing system can capture the entire page using screenshot functions such as `page.screenshot()`, or precisely capture the WebGL canvas area using `canvas.toDataURL()`, thereby obtaining the first actual rendered image. To ensure that the first actual image is time-aligned with the 3D animation keyframes, a short delay (e.g., 50ms) is waited before each screenshot, or the Three.js rendering loop is monitored via a JavaScript hook, triggering a screenshot only when rendering is complete. Simultaneously, the automated testing system synchronously acquires Canvas video frames, ensuring strict temporal and spatial alignment of multimodal content by comparing the video frame timestamps with the animation frame timestamps. The acquired images are stored in the storage location corresponding to the actual image storage path in the structured test data for subsequent comparative analysis.

[0065] As described above, this embodiment of the application achieves high-fidelity capture of keyframes in 3D animation by precisely controlling the browser rendering process and screenshot timing. This ensures the authenticity and timeliness of the test data, avoids misjudgments caused by incomplete rendering or frame rate fluctuations, and provides a reliable basis for subsequent graphic comparisons.

[0066] Step 105: Perform a multi-dimensional consistency comparison analysis between the first actual image and the expected image to obtain the comparison analysis results, and generate a visualization test report based on the comparison analysis results.

[0067] The expected image is a standard 3D animation frame image generated based on vehicle signals corresponding to key time points.

[0068] In this embodiment, the expected image can be a standard 3D animation frame image generated offline through pre-rendering or an AI image generation model (such as Stable Diffusion + 3D prior) based on key time points and vehicle signal states specified in the test requirements. The expected image can characterize the visual effect that should be presented under ideal conditions and is used for comparison and verification with the actually acquired image. For example, when the vehicle is in a signal state with the left turn signal on and the vehicle speed at 30km / h, the expected image should show features such as the left headlight flashing, the wheels slightly tilted, and the vehicle body shifting to the left.

[0069] In this embodiment of the application, the multi-dimensional consistency comparison analysis between the first actual image and the expected image includes, but is not limited to: structural similarity analysis, feature matching degree analysis, and difference heatmap.

[0070] In this embodiment of the application, the visual test report includes at least one of the following: statistics on the execution results of test cases, execution time and status logs of each test step, screenshot collection records, image comparison and analysis results, analysis of test failure reasons, and information on the accumulation of test experience and knowledge.

[0071] Test case execution result statistics refer to the summary analysis of the execution results of all test cases, including the number and proportion of passed, failed, and skipped cases. Test case execution result statistics help testers quickly grasp the overall test coverage and quality level, and identify high-risk modules or frequently failing test cases.

[0072] The execution time and status logs for each test step record the start time, end time, duration, and execution status (success / failure / exception) of each test operation. These logs are crucial for locating performance bottlenecks and debugging issues. For example, if a Three.js scene loading step takes significantly longer than expected (e.g., more than 5 seconds), it may indicate unoptimized resources or network latency. The system arranges the execution time and status logs for each test step in chronological order and marks key event nodes (such as page loading completion, animation start, successful screenshot, etc.), supporting click-throughs to the corresponding screenshot or video clip, improving problem tracing efficiency.

[0073] The screenshot capture record is used to automatically capture keyframe images during test execution, along with the record's metadata (such as timestamp, screenshot path, and trigger conditions). This record is used to visually verify whether the rendering results meet expectations, and is particularly useful for identifying visual defects in WebGL scenarios that cannot be determined through DOM assertions. For example, in a vehicle trajectory animation test, the system captures images at the 10th, 20th, and 30th seconds of the animation. The screenshot capture record also includes exception information for failed screenshots (such as Canvas not responding), which helps diagnose environment or script issues.

[0074] Among them, the image comparison analysis results can be quantitative indicators and visualization results obtained by comparing the actual image with the expected image through multi-dimensional algorithms.

[0075] Among them, the test failure reason analysis is used when there is inconsistency in the image comparison analysis results. The system calls the large language model to reason about the types of differences identified in the image comparison analysis results, and outputs the possible root causes of the failure by combining context logs and code logic.

[0076] Among them, the test experience knowledge accumulation information refers to storing the problems found in this test, the diagnosis process, the repair solution and the final result in the test experience knowledge accumulation information repository for reuse in subsequent test tasks and AI learning.

[0077] In some embodiments, step 105 performs a multi-dimensional consistency comparison analysis between the first actual image and the expected image to obtain the comparison analysis results, which can be achieved through the following steps: Step 151: Calculate the structural similarity between the first actual image and the expected image to obtain the structural similarity analysis results.

[0078] In this embodiment, structural similarity calculation is used to determine whether the first actual image and the expected image are consistent in macroscopic visual effect, and is used to evaluate the overall image quality rather than local details. Structural similarity calculation can be implemented using the SSIM (Structural Similarity Index Measure) algorithm, which evaluates the overall visual fidelity of the image by comparing the statistical characteristics of pixel values ​​within a local window, such as the total number of pixel values.

[0079] In this embodiment, the automated testing system calls the image processing module built into the testing agent or integrates the OpenCV library to process the first actual image and the expected image in blocks, calculates the SSIM value for each block, and finally obtains the global similarity score by weighted average. In this way, the overall visual consistency of the captured actual rendered image can be objectively evaluated, thereby reducing false alarms caused by color deviation or brightness adjustment, and thus improving the accuracy and stability of the image verification system.

[0080] Step 152: Using the ORB feature matching algorithm, feature points of the first actual image and the expected image are extracted and matched for verification to obtain the feature matching degree analysis results.

[0081] In this embodiment, the ORB (Oriented Fast and Rotated BRIEF) feature matching algorithm is used to detect key feature points (such as vehicle outlines, road marking intersections, building vertices, etc.) in the first actual image to be matched and the preset expected image. The algorithm then determines whether the image is misaligned due to geometric transformations such as rotation, scaling, or translation by matching these key feature points. These key feature points can be pixel locations in the image with unique textures or edge variations, including but not limited to corner points, spots, or edge intersections. These feature points remain stable and are easily identifiable under different viewing angles.

[0082] In this embodiment of the application, the feature matching degree analysis result may be the proportion of matching points of key feature points in the first actual image and the expected image or the matching success rate.

[0083] In this embodiment, the automated testing system performs ORB feature extraction on the first actual image and the expected image respectively to obtain their respective key feature point sets. Then, a matching algorithm is used to calculate the proportion of key feature points with matching relationships, thereby obtaining the feature matching degree analysis result. For example, in a vehicle trajectory playback test, if the vehicle model in the first actual image shifts relative to the reference image, the ORB feature matching algorithm will show a decrease in the number of matching points or uneven distribution, thus locating the shift problem of the vehicle model relative to the reference image. In this way, through feature point matching, the consistency of geometric transformations between images can be effectively detected, thereby identifying rendering anomalies caused by coordinate system offset, camera parameter errors, or model loading failures, and thus improving the ability to detect structural defects.

[0084] Step 153: Generate a pixel-level difference heatmap based on the pixel differences between the first actual image and the expected image.

[0085] The comparative analysis results include at least one of the following: structural similarity analysis results, feature matching degree analysis results, and difference heatmap results.

[0086] Among them, the difference heatmap is used to locate abnormal areas in the rendering results. Abnormal areas include at least one of the following: projection offset areas, texture misalignment areas, and shadow missing areas.

[0087] In this embodiment, the pixel difference can be the difference between the corresponding pixel values ​​of the actual image and the expected image at the same spatial location, which is usually obtained by pixel-by-pixel subtraction.

[0088] In this embodiment, a difference heatmap is used to present a heatmap showing the intensity of differences between pixels at the same location. The degree of difference can be represented by color intensity, such as red for high difference and blue for low difference. This difference heatmap visually displays areas of inconsistency in the image, facilitating rapid location of specific defects.

[0089] In this embodiment, the abnormal region can be an image sub-region that shows significant differences in the difference heatmap. The causes of the abnormal region include, but are not limited to: the causes of projection offset regions (such as misalignment between the Canvas video frame and the 3D model surface), the causes of texture misalignment regions (such as incorrect UV coordinate settings in the texture mapping leading to pattern distortion), and the causes of shadow missing regions (such as improper light source settings or incorrect material opacity configuration resulting in shadows not being rendered). The above-mentioned abnormal regions are direct manifestations of graphics rendering errors and are also the starting point for repair work.

[0090] In this embodiment, pixel-by-pixel difference calculation is performed between the first actual image and the expected image to obtain an absolute pixel difference matrix, which is then converted into a difference heatmap through color mapping. In this way, the method can accurately locate the specific location of rendering defects, thereby guiding developers to quickly troubleshoot the root cause of the problem, and thus shortening the fault repair cycle and improving testing efficiency.

[0091] As can be seen from the above, the embodiments of this application perform multi-dimensional consistency comparison analysis between the first actual image and the reference image through structural similarity calculation, ORB feature matching, and pixel-level difference heatmap generation. In this way, the differences between the images in terms of overall vision, geometric structure, local details, and time synchronization can be fully covered, thereby accurately identifying various types of rendering defects and realizing high-quality automated closed-loop testing of the WebGL visualization system.

[0092] In some embodiments, step 105, which generates a visual test report based on the comparative analysis results, can be achieved through the following steps: Step 154: If the comparative analysis results do not meet the consistency condition in at least one dimension, identify the anomaly type and infer the cause of the anomaly based on the anomaly logs during the test execution process.

[0093] In this embodiment of the application, at least one dimension can be one or more of the following four dimensions: structural similarity, feature point matching, difference heatmap, and spatiotemporal.

[0094] In this embodiment, when multi-dimensional image comparison analysis indicates a visual inconsistency between the current actual image and the expected image, the automated testing system triggers an automatic diagnostic process. Through a testing agent, abnormal logs recorded during test execution are input. The testing agent parses the log semantics, identifies specific anomaly types, such as texture loading failure, coordinate system offset, video frame and animation frame desynchronization, and incorrect lighting parameters, and further infers the causes of the anomalies, such as incorrect UV mapping configuration, incorrect animation timeline binding, and browser rendering pipeline compatibility issues. This semantic transformation process converts raw log data into understandable fault modes, providing accurate basis for subsequent repairs.

[0095] Step 155: Generate repair suggestions for the front-end code based on the exception type and cause.

[0096] In this embodiment, after obtaining the anomaly type and cause, the automated testing system generates targeted front-end code repair suggestions through a testing agent. These suggestions are presented as executable JavaScript / TypeScript code snippets and directly applied to the front-end code. Furthermore, the repair suggestions not only include code changes but also brief explanations, such as ensuring strict alignment of video frames and 3D animation keyframes on the timeline to resolve projection offset issues. Thus, this method achieves a seamless transition from fault diagnosis to specific code correction, thereby lowering the technical barrier for developers and accelerating the speed of defect closure.

[0097] Step 156: Based on the repair suggestions, complete the repair of the front-end code and update the executable Playwright test script simultaneously.

[0098] In this embodiment, after receiving a repair suggestion, the automated testing system injects the suggested code snippet into the front-end code file using automated tools (such as GitHook, local file writing, or remote deployment interfaces) to complete the front-end code repair operation. Simultaneously, to ensure that the executable Playwright test script reflects the latest code state, the automated testing system automatically updates the relevant assertion logic, operation sequences, or screenshot timings in the executable Playwright test script. This achieves coordination between front-end code repair and executable Playwright test script updates, ensuring the consistency of the testing environment, thereby improving overall R&D efficiency and enabling the construction of an efficient continuous integration / continuous delivery (CI / CD) quality assurance closed loop.

[0099] Step 157: Run the updated executable Playwright test script to acquire the second actual image corresponding to the key time points in the 3D animation scene.

[0100] In this embodiment, after the front-end code is repaired and the executable Playwright script is updated, the automated testing system reruns the updated executable Playwright test script. In a headless browser environment, this executable Playwright test script controls page loading, simulates user interactions (such as clicking buttons and dragging sliders), and plays Three.js animations. It also captures the actual rendered images of the 3D animation scene at key time points, i.e., the second actual image. The acquisition process ensures alignment with the same timestamps as the previous acquisition for comparison with the expected image.

[0101] Step 158: Perform a multi-dimensional consistency comparison analysis on the second actual image and the expected image again, and repeat the repair and retest process until the results of the multi-dimensional comparison analysis all meet the consistency threshold requirements, and generate a visual test report of the corresponding vehicle function.

[0102] In this embodiment, the automated testing system performs a multi-dimensional consistency comparison analysis on the newly acquired second actual image and the expected image again, using the same multi-dimensional consistency comparison analysis algorithm (SSIM, ORB, difference heatmap, spatiotemporal alignment verification) to evaluate consistency. If the comparison result between the newly acquired second actual image and the expected image still does not meet the preset consistency threshold (e.g., SSIM < 0.95), steps 154 to 158 are repeated, entering the next round of front-end code repair, executable Playwright test script update, running the updated executable Playwright test script, and acquiring the second actual image. The above loop continues until the multi-dimensional consistency comparison analysis results of all dimensions meet the preset consistency threshold (e.g., SSIM ≥ 0.95, ORB matching rate ≥ 90%, no significantly different heatmap areas). Once the conditions are met, the testing system terminates the loop and generates the above-mentioned visualized test report containing the entire process log, image comparison results, repair history, and final conclusion.

[0103] As described above, the embodiments of this application automatically initiate a closed-loop process when a test fails: analysis of exception logs → generation of repair suggestions → front-end code repair → update of executable Playwright test scripts → running the updated executable Playwright test scripts → acquisition of a second actual image → multi-dimensional consistency comparison analysis. This enables autonomous diagnosis and intelligent repair of defects in the WebGL visualization system, significantly reducing the frequency of manual intervention, and thus enabling the construction of an end-to-end automated testing and quality assurance system.

[0104] The following is an exemplary description of the process of an automated testing method provided in this application embodiment, combined with specific scenarios.

[0105] This application proposes a closed-loop automated testing method for WebGL visualization systems based on intelligent agents (corresponding to the automated testing method mentioned above) and system. It constructs a complete closed-loop process from natural language requirement understanding, test task planning, AI script generation, execution and screenshotting, graphic comparison and verification, automatic diagnosis and repair, to achieve end-to-end automated quality assurance of 3D visualization content.

[0106] Reference Figure 2 and Figure 3As shown, the method uses the 3D visualization test of vehicle operation status in the intelligent transportation digital twin platform as an application scenario to demonstrate the closed loop of the entire process from test requirement input to automated execution, graphical verification and report generation. The method includes the following steps.

[0107] Step S1: Build a data-driven and AI-integrated testing framework.

[0108] To improve the reusability, maintainability, and scalability of test scripts, an automated testing framework that integrates data-driven approaches and Large Language Model (LLM) understanding capabilities is designed.

[0109] Here, the test data is managed using structured files (such as Excel or JSON), and the core fields include: function module name, visualization data URL address, function description (natural language), test plugin name, topic name (corresponding vehicle status message topic), animation playback duration, expected screenshot standard path, actual screenshot output path, and video synchronization time offset (used for Canvas video alignment).

[0110] Here, structured parameters are not only used to control the flow, but also serve as input for AI to understand the context. When data changes (such as topic changes or URL updates), the system automatically detects and adapts without requiring code modifications.

[0111] Here, the framework supports modular plug-in design, with different functional modules (such as trajectory playback, heatmap rendering, and video fusion) being independently encapsulated as pluggable test components, making them easy to combine and call.

[0112] Step S2: Natural Language Testing Requirements Modeling.

[0113] Here, combined Figure 2 The system obtains user input 210, representing natural language testing requirements such as "verify whether the vehicle trajectory animation and the projection of the surveillance video on the 3D road are synchronized," or "open the vehicle visualization interface, uncheck all layers, only check 'vehicle' and 'grid,' then search for and select a specific topic (e.g., / vehicle / trajectory), play a 30-second animation, and finally take a screenshot and compare it with the expected image to determine if the rendering is correct." Then, through the requirement understanding module 202, using large language models (e.g., Qwen3, Claude, GLM) and Natural Language Processing (NLP), the system parses the natural language testing requirements, automatically extracting key semantic elements, including objects, actions, and spatiotemporal relationships. Through the task planning module 203, based on the key semantic elements, the natural language testing requirements are decomposed into multiple sub-tasks.

[0114] Here, refer to Figure 3 As shown, the reception, parsing, task decomposition, and timing processing of test requirements can be completed by calling the test agent through the large language model. The test agent can receive test requirements through the user interaction interface 209, receive and parse the test requirements using the test agent's perception capability 301, and decompose and process the test requirements using the test agent's planning capability 302.

[0115] In one feasible scenario, the natural language testing requirement is: "Open the vehicle visualization interface, uncheck all layers, only select 'Automobile' and 'Grid', then search for and select a specific topic (e.g., / vehicle / trajectory), play a 30-second animation, and finally take a screenshot and compare it with the expected image to determine if the rendering is correct." The testing system, through the requirement understanding module 20 and the task planning module 203, breaks down this natural language testing requirement into the following sub-tasks: 1) DOM manipulation: Open menu → Deselect all → Enter keywords → Check targets → Close menu.

[0116] 2) Three.js Interaction: Launch the animation player and control the timeline to run for 30 seconds; 3) Canvas video synchronization: Ensures that the monitoring video frames are time-aligned with the 3D animation keyframes; 4) Screenshot and verification: Capture the WebGL canvas at a specified time point and perform image similarity comparison.

[0117] It should be noted that during the task decomposition process, the Prompt project guides large language models (such as Qwen3 and Claude) to accurately understand the semantics, obtain multiple sub-tasks, and output a structured task tree with execution sequence. In this way, based on the business scenario, natural language is used to describe the test objectives and execution process, reducing the coding burden on technical personnel.

[0118] Step S3: The AI ​​agent automatically generates the test script.

[0119] Here, the large language model acts as the testing agent. Through the code generation module 204, based on the structured task tree, the expected image obtained by the expected image generation module 205, and the test data obtained by the data parameter loading module 206, combined with the context knowledge base 303 (such as the Three.js API documentation and project code structure), a phased generation strategy is adopted. Utilizing the execution capability 304 of the testing agent, an executable Playwright test script is generated, containing DOM manipulation, Three.js scene interactions (such as onMouseMove and updateCamera), Canvas video synchronization control, and screenshot verification instructions. This executable Playwright test script is used to test the front-end code. It should be noted that this executable Playwright test script supports modeling complex interactive logic, such as camera path animation, model highlighting, and timeline binding, significantly reducing the barrier to entry for writing test scripts.

[0120] Here, the expected image is generated either by pre-rendering an ideal scene offline or by using an AI image generation model (such as StableDiffusion + 3D prior) to generate a standard image.

[0121] Here, the phased generation strategy includes: 1) Framework code generation Here, AI generates a basic code structure based on the test framework template, including: page loading and waiting logic, data reading module (reading URL, Topic, etc. from Excel), and logging and exception handling mechanism.

[0122] 2) Business flow code generation Here, AI combines contextual knowledge base, test data, natural language description, and basic code structure to generate business flow test code that matches the business process of the front-end code.

[0123] 3) Three.js interactive code injection Here, a hook function is injected into the business flow test code to execute Three.js control logic in the browser context via Playwright's page.evaluate().

[0124] 4) Assertions and log enhancement Here, AI automatically adds image comparison assertions and detailed logging logic to the business flow test code, resulting in an executable Playwright test script. Here, AI not only generates code but also proactively adds assertions, logging, and exception handling.

[0125] Step S4: Graphical comparison verification and closed-loop self-repair.

[0126] Traditional methods require manual review of screenshots to determine whether a result passes. This application introduces a graphical feedback closed-loop mechanism to achieve automated verification and intelligent repair.

[0127] (1) Image acquisition and alignment Here, the generated executable Playwright test script is run in the test execution engine 207 environment of Headless Chrome 305, and the Three.js rendering process is precisely controlled through the JavaScript hooks injected by Playwright.

[0128] Here, the screenshot acquisition module 208 uses Playwright to call canvas.toDataURL() at a specified time point to capture the WebGL canvas; simultaneously, it acquires Canvas video frames to ensure alignment with the 3D animation keyframes in terms of timestamps (e.g., both based on performance.now()); it stores the actual screenshots to the corresponding local path; and it supports automatic traversal of multiple perspectives, lighting, and zoom levels according to preset paths to improve test coverage.

[0129] (2) Multi-dimensional graphic comparison Utilizing the feedback capability 306 of the test agent, the image comparison verification module 209 employs a multi-dimensional image comparison algorithm to verify the consistency between the cropped image and the expected image. The multi-dimensional comparison algorithm includes: SSIM, used to judge or evaluate the overall visual fidelity; ORB feature matching, used to detect whether key points are aligned and tolerate slight rotation / scaling; and difference heatmap, used to locate local abnormal regions, using OpenCV to calculate pixel-level differences and highlight abnormal regions (such as projection offset, texture misalignment, and shadow loss).

[0130] Meanwhile, it supports spatiotemporal consistency verification, comparing video frame timestamps with 3D animation keyframe timestamps to ensure that the projection of the Canvas video onto the 3D surface is strictly aligned in time and space.

[0131] (3) Automatic diagnosis and script optimization Here, the feedback capability 306 of the test agent is used to determine whether the image comparison results meet expectations 210, i.e., whether the image comparison results are consistent. Here, when the image comparison results indicate that the two images are inconsistent, the self-repair process is initiated. Using the memory bank 307 of the test agent, the difference analysis and diagnosis module 211 automatically analyzes the difference type (such as Z-fighting, texture not loaded, coordinate offset, projection offset, and lighting anomaly) and infers possible causes, such as video and animation time synchronization issues and UV mapping offsets. The self-repair code generation module 212 calls the large language model to generate repair suggestion code and repairs and updates the front-end code, automatically updates the script, and re-executes the test. This forms an autonomous closed-loop optimization mechanism of "generation → execution → verification → repair", reducing manual intervention.

[0132] Step S5: Generate a visual test report Here, when the image comparison result meets expectations 210, that is, when the image comparison result indicates that the two images are consistent, after the script execution is complete, the learning ability 308 of the test agent is used to automatically generate an interactive HTML test report 210, supporting rapid problem location and process tracing. The test report includes: overall statistics: number of passed / failed / skipped test cases; detailed step logs: time, status, and screenshots for each step; image comparison results: displaying the expected and actual images side-by-side, with SSIM values ​​and difference heatmaps labeled; root cause analysis of failures: automatically extracting error information and repair suggestions; and knowledge accumulation: storing the problem and solution in the test memory 307 for subsequent AI access. Simultaneously, the system supports synchronizing test results to CI / CD pipelines (such as Jenkins and GitHub Actions), achieving automated quality gatekeeping in continuous integration.

[0133] The embodiments of this application have the following beneficial effects: 1) Breaking through the WebGL unpredictability bottleneck: For the first time, it realizes automated verification of WebGL rendered content (such as 3D models, materials, lighting, and animation), solving the fundamental defect of traditional tools (Selenium / Playwright) that can only manipulate the DOM and cannot perceive the visual output.

[0134] 2) Achieve fully autonomous closed-loop testing: Unlike existing AI that only assists in element recognition or code completion, this invention constructs a complete intelligent agent closed loop from requirement to repair, with autonomous planning, execution, judgment and optimization capabilities.

[0135] 3) Support for multimodal fusion scene verification: Innovatively introduces a spatiotemporal consistency detection mechanism between Canvas video and 3D scene, which is applicable to high-value scenarios such as digital twins and intelligent transportation, ensuring visual authenticity.

[0136] 4) Significantly reduces testing threshold and cost: Driven by natural language, it enables non-professional developers to initiate complex 3D testing tasks, allowing technical personnel to focus on business logic rather than coding details.

[0137] 5) The effectiveness has been verified in real-world scenarios: This invention has been applied to the intelligent transportation digital twin platform for regression testing of vehicle operation status monitoring systems, improving test script generation efficiency by more than 80% and significantly increasing defect detection rate.

[0138] This application provides an automated testing device, referring to... Figure 4 As shown, Figure 4 This is a schematic diagram of an automated testing device provided in an embodiment of this application. The automated testing device 4 includes: Module 401 is used to obtain test requirements, structured test data, and front-end code to be tested. The front-end code is used to construct a 3D animation scene containing a 3D vehicle model based on the vehicle's basic information and the vehicle signals associated with each function of the vehicle. In response to changes in the state of the vehicle signals, it drives the 3D vehicle model to perform the corresponding 3D animation effects. The processing module 402 is used to decompose and structure the test requirements according to the test operation type to obtain a structured task tree with execution sequence. The structured task tree includes sub-tasks of different test operation types. The sub-tasks include at least one of the following: DOM operation sub-tasks, Three.js interactive operation sub-tasks, video synchronization operation sub-tasks, and screenshot verification operation sub-tasks. The processing module 402 is also used to generate an executable Playwright test script for the front-end code using a structured task tree as the execution logic framework and structured test data as test case input parameters, and adopting a phased progressive generation strategy. Run module 403, which is used to run executable Playwright test scripts; The processing module 402 is also used to acquire the first actual image corresponding to the key time point in the 3D animation scene; wherein, the key time point is the moment when the vehicle state of the 3D vehicle model changes; to perform a multi-dimensional consistency comparison analysis on the first actual image and the expected image, to obtain the comparison analysis result, and to generate a visualization test report based on the comparison analysis result; wherein, the expected image is a standard 3D animation frame image generated based on the vehicle signal corresponding to the key time point.

[0139] This application provides a hardware entity diagram of an automated testing system, such as... Figure 5As shown, the hardware entity of the automated testing system 5 includes a processor 501 and a memory 502. The memory 502 stores a computer program that can run on the processor 501. When the processor 501 executes the computer program, it implements some or all of the steps in the automated testing method as described in the above embodiments.

[0140] The memory 502 stores computer programs that can run on the processor. The memory 502 is configured to store instructions and applications that can be executed by the processor 501. It can also cache data to be processed or already processed by the processor 501 and the various modules in the automated test system 5 (e.g., image data, audio data, voice communication data and video communication data). It can be implemented by flash memory or random access memory (RAM).

[0141] The processor 501 executes the program to implement the steps of the automated test execution method described above. The processor 501 typically controls the overall operation of the automated test system 5.

[0142] This application provides a computer-readable storage medium storing one or more computer programs, which can be executed by one or more processors to implement some or all of the steps in the above-described method. The storage medium can be transient or non-transient.

[0143] This application provides a computer program including computer-readable code, wherein when the computer-readable code is run in a vehicle, a processor in the vehicle executes some or all of the steps in the above-described method.

[0144] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0145] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0146] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0147] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0148] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0149] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0151] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0152] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0153] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0154] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an in-vehicle terminal (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0155] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An automated testing method, characterized in that, The method includes: Obtain test requirements, structured test data, and front-end code to be tested; wherein, the front-end code is used to construct a 3D animation scene containing a 3D vehicle model based on the vehicle's basic information and the vehicle signals associated with each function of the vehicle, and to drive the 3D vehicle model to perform the corresponding 3D animation effect in response to the state changes of the vehicle signals. The test requirements are broken down and structured according to the test operation type to obtain a structured task tree with execution sequence. The structured task tree includes sub-tasks of different test operation types. The sub-tasks include at least one of the following: DOM operation sub-tasks, Three.js interactive operation sub-tasks, video synchronization operation sub-tasks, and screenshot verification operation sub-tasks. Using the structured task tree as the execution logic framework and the structured test data as test case input parameters, an executable Playwright test script for the front-end code is generated using a phased progressive generation strategy. Run the executable Playwright test script to acquire the first actual image corresponding to the key time point in the 3D animation scene; wherein, the key time point is the moment when the vehicle state of the 3D vehicle model changes; A multi-dimensional consistency comparison analysis is performed on the first actual image and the expected image to obtain the comparison analysis results, and a visualization test report is generated based on the comparison analysis results; wherein, the expected image is a standard three-dimensional animation frame image generated based on the vehicle signal corresponding to the key time point.

2. The method according to claim 1, characterized in that, The test requirements are broken down and structured according to test operation types to obtain a structured task tree with execution sequence, including: Extract the key semantic elements from the test requirements; wherein, the key semantic elements include at least one of the following: test object, execution action, and spatiotemporal constraint relationship; Based on the key semantic elements, the test requirements are broken down into multiple sub-tasks according to the test operation type. The multiple subtasks are processed according to their hierarchical relationship and execution order to generate the structured task tree.

3. The method according to claim 1, characterized in that, The phased, progressive generation strategy includes: Based on the Playwright testing framework template, construct the basic structure code; wherein, the basic structure code includes at least one of the following: page load waiting logic, test data reading module, global logging module, and exception handling mechanism; Based on a pre-built contextual knowledge base, using the structured task tree as the execution logic framework, and using the structured test data as test case input parameters, the basic structure code is populated with business logic and its functions are extended to generate Playwright business flow test code that matches the business process of the front-end code. Through Playwright's page script execution interface, an engine call hook function is injected into the Playwright business flow test code. The engine call hook function is used to call the 3D rendering engine control code in the front-end code in the page running context. Add multi-dimensional image comparison assertion logic and full-link execution log logic to the Playwright service flow test code to obtain the executable Playwright test script.

4. The method according to claim 3, characterized in that, The context knowledge base includes: the official Three.js application programming interface (API) documentation, the project structure information of the front-end code, the description information of the 3D scene interaction logic, and the video synchronization verification rules.

5. The method according to claim 1, characterized in that, The step of performing a multi-dimensional consistency comparison analysis between the first actual image and the expected image to obtain the comparison analysis results includes: The structural similarity between the first actual image and the expected image is calculated to obtain the structural similarity analysis results; The ORB feature matching algorithm is used to extract feature points from the first actual image and the expected image respectively and perform matching verification to obtain the feature matching degree analysis results. Based on the pixel differences between the first actual image and the expected image, a pixel-level difference heatmap is generated; wherein, the difference heatmap is used to locate abnormal regions in the rendering result, and the abnormal regions include at least one of projection offset regions, texture misalignment regions, and shadow missing regions; wherein, the comparison analysis results include at least one of the structural similarity analysis results, the feature matching degree analysis results, and the difference heatmap.

6. The method according to any one of claims 1 to 5, characterized in that, The process of generating a visual test report based on the comparative analysis results includes: If the comparative analysis results do not meet the consistency condition in at least one dimension, the anomaly type is identified and the cause of the anomaly is inferred based on the anomaly logs during the test execution process. Based on the exception type and the exception cause, generate repair suggestions for the front-end code; Based on the repair suggestions, the front-end code was repaired, and the executable Playwright test script was updated synchronously. Run the updated executable Playwright test script to acquire the second actual image corresponding to the key time points in the 3D animation scene; The second actual image and the expected image are subjected to a multi-dimensional consistency comparison analysis again. The repair and retest process is repeated until the results of the multi-dimensional comparison analysis meet the consistency threshold requirements, and a visual test report of the corresponding vehicle function is generated.

7. The method according to any one of claims 1 to 5, characterized in that, The visualized test report includes at least one of the following: test case execution status statistics, execution time and status logs of each test step, screenshot collection records, image comparison and analysis results, test failure reason analysis, and test experience and knowledge accumulation information.

8. An automated testing device, characterized in that, include: The acquisition module is used to acquire test requirements, structured test data, and front-end code to be tested; wherein, the front-end code is used to construct a 3D animation scene containing a 3D vehicle model based on the vehicle's basic information and the vehicle signals associated with each function of the vehicle, and to drive the 3D vehicle model to perform the corresponding 3D animation effect in response to the state changes of the vehicle signals. The processing module is used to decompose and structure the test requirements according to the test operation type to obtain a structured task tree with execution sequence. The structured task tree includes sub-tasks of different test operation types. The sub-tasks include at least one of the following: DOM operation sub-tasks, Three.js interactive operation sub-tasks, video synchronization operation sub-tasks, and screenshot verification operation sub-tasks. The processing module is also used to generate an executable Playwright test script for the front-end code by using the structured task tree as the execution logic framework and the structured test data as test case input parameters, and by adopting a phased progressive generation strategy. The runtime module is used to run the executable Playwright test script; The processing module is further configured to acquire a first actual image corresponding to a key time point in the 3D animation scene; wherein the key time point is the moment when the vehicle state of the 3D vehicle model changes; perform a multi-dimensional consistency comparison analysis on the first actual image and the expected image to obtain the comparison analysis result, and generate a visualization test report based on the comparison analysis result; wherein the expected image is a standard 3D animation frame image generated based on the vehicle signal corresponding to the key time point.

9. An automated testing system, characterized in that, include: Memory is used to store executable instructions or computer programs. A processor for implementing the automated testing method as described in any one of claims 1 to 7 when executing computer-executable instructions or computer programs stored in the memory.

10. A computer-readable storage medium, characterized in that, It stores a computer program or computer-executable instructions for implementing the automated testing method as described in any one of claims 1 to 7 when executed by a processor.

Images