Front-end regression test method, program product, electronic equipment and storage medium

Abstract

The invention provides a front-end regression test method, a program product, electronic equipment and a storage medium. The method comprises the following steps: acquiring user behavior data of a user in a to-be-tested application program; based on the element positioning information, performing semantic annotation on the user behavior data through a predefined rule and / or a semantic understanding model to obtain semantic information; based on the network request data, grouping the labeled user behavior data, and screening out a test case corresponding to a single page path for a user behavior data group; restoring the test environment to an initial state; determining a target element corresponding to the interactive operation data in the test case by adopting an element positioning algorithm; and in the initial state, executing a corresponding interaction operation based on the target element, and completing test playback. Therefore, the test script has higher adaptability to local change of the front-end page, the large-scale failure risk of the test case caused by the change of the page structure is reduced, and the stability and reliability of the test are enhanced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a front-end regression testing method, a program product, an electronic device, and a storage medium. Background Technology

[0002] Current front-end testing heavily relies on manual script recording and maintenance based on testing tools. Testers need to manually manipulate the page to generate test cases. In the context of agile development and rapid iteration, page structure and code change frequently, and any modification can cause test cases to fail, resulting in a large maintenance workload and low efficiency of regression testing. Summary of the Invention

[0003] The purpose of this application is to provide a front-end regression testing method, program product, electronic device, and storage medium to improve the above-mentioned problems.

[0004] In a first aspect, embodiments of this application provide a front-end regression testing method, comprising: acquiring user behavior data in an application to be tested; the user behavior data includes interaction operation data, network request data, environmental context data, and element location information; based on the element location information, semantically labeling the user behavior data using predefined rules and / or a semantic understanding model to obtain semantic information of the user behavior data; the semantic information is used to characterize the functions triggered in the user behavior data; based on the network request data, grouping the labeled user behavior data to obtain a user behavior data group for a single page path; for the user behavior data group, selecting test cases corresponding to the single page path based on the semantic information; restoring the test environment to its initial state based on the environmental context data in the test cases; using an element location algorithm to determine the target element corresponding to the interaction operation data in the test cases; and in the initial state, performing the corresponding interaction operation based on the target element to complete the test replay.

[0005] In the aforementioned implementation process, test case materials can be automatically and continuously collected and intelligently filtered from real online traffic, saving a significant amount of cost and time spent on manually writing and maintaining test scripts, and improving the efficiency of test case construction and maintenance. By introducing multi-strategy element positioning and AI-assisted matching, the test scripts have a stronger ability to adapt to local changes in the front-end page, reducing the risk of large-scale test case failures caused by page structure changes, and enhancing the stability and reliability of automated testing. Since the test cases originate from user behavior and undergo semantic-based intelligent filtering, they focus on covering core and high-frequency business scenarios, improving the accuracy and business relevance of test coverage.

[0006] Optionally, in this embodiment of the application, based on element location information, semantic annotation of user behavior data is performed through predefined rules and / or a semantic understanding model to obtain semantic information of the user behavior data, including: Based on a predefined component rule mapping table, the element tags, class names, or attribute keywords in the element location information are matched with the functional components in the component rule mapping table. Based on the successfully matched functional components, semantic annotation is performed on the user behavior data to obtain the semantic information of the user behavior data; and / or, the element location information, interaction operation data, and network request data in the user behavior data are input into a trained semantic understanding model to obtain the semantic description of the functions triggered in the user behavior data output by the semantic understanding model, and the user behavior data is semantically annotated to obtain the semantic information of the user behavior data.

[0007] In the above implementation process, user behavior data is semantically annotated using at least one method, enabling machines to understand the business intent of user operations like testers. This lays a high-quality semantic foundation for subsequent test case selection and aggregation, greatly improving the automation and semantic accuracy of test case generation. Furthermore, different annotation methods enhance the solution's adaptability to different technologies and page styles; rule mapping can efficiently handle standardized components, while the semantic understanding model can flexibly handle customized and dynamic content.

[0008] Optionally, in this embodiment of the application, based on element location information, semantic annotation of user behavior data is performed through predefined rules and / or a semantic understanding model to obtain semantic information of user behavior data. This includes: matching element tags, class names, or attribute keywords in the element location information with functional components in the component rule mapping table based on a predefined component rule mapping table; performing semantic annotation on user behavior data based on successfully matched functional components to obtain semantic information of user behavior data; obtaining user behavior data that fails to match; inputting element location information, interaction operation data, and network request data from the user behavior data that fails to match into a trained semantic understanding model; obtaining a semantic description of the functions triggered in the user behavior data output by the semantic understanding model; and performing semantic annotation on the user behavior data to obtain semantic information of user behavior data.

[0009] In the above implementation process, the two annotation methods work in a collaborative mode of "rule-first, AI as a backup." First, efficient rule mapping is used for rapid annotation; for behavioral data where rule matching fails or the matching confidence is low, the data is automatically transferred to a semantic understanding model for in-depth analysis and annotation. This hybrid strategy ensures processing speed while also improving annotation coverage and accuracy.

[0010] Optionally, in this embodiment of the application, for a user behavior data group, test cases corresponding to a single page path are selected based on semantic information, including: for a user behavior data group, determining the type, total number of types, and number of times each type of function is triggered by the user behavior data based on semantic information; generating a score for the user behavior data based on at least one of the type, total number of types, and number of times each type of trigger is selected; and selecting multiple test cases corresponding to a single page path based on the score; the multiple test cases cover all functions in the single page path.

[0011] In the above implementation process, by considering multiple aspects such as the type of triggered functions, the total number of types, and the number of times each type is triggered, comprehensive test scenarios are automatically and quickly identified and selected from massive amounts of user behavior. This improves upon the oversights or subjective biases that may exist in traditional manual test case design, and enhances the efficiency and representativeness of test case set construction. Quantitative analysis based on function type and trigger count ensures that the final generated test case set can systematically cover all functionalities of the page, especially high-priority core functions, thereby effectively improving the completeness and reliability of testing.

[0012] Optionally, in this embodiment, the element location algorithm includes at least one of a main strategy location algorithm, a hash value matching algorithm, an artificial intelligence similarity algorithm, and a backup strategy algorithm; using the main strategy location algorithm to determine the target element corresponding to the interactive operation data in the test case includes: obtaining a structured location sequence generated for the interactive operation data during the user behavior data collection process, the structured location sequence including the hierarchical path of the interactive operation from the root node to the target element; parsing the hierarchical path to determine the target element; using the hash value matching algorithm to determine the target element corresponding to the interactive operation data in the test case includes: matching the unique hash value generated by the element attributes in the interactive operation data with a list of hash values ​​corresponding to multiple preset element attributes, and selecting the element that matches successfully. The elements are identified as target elements corresponding to the interactive operation data. An artificial intelligence similarity algorithm is used to determine the target elements corresponding to the interactive operation data in the test cases, including: generating target feature vectors corresponding to element attributes in the interactive operation data using the artificial intelligence similarity algorithm; calculating the similarity between the target feature vectors and multiple preset feature vector sets corresponding to element attributes; and determining the target elements corresponding to the interactive operation data based on the similarity results. A backup strategy algorithm is used to determine the target elements corresponding to the interactive operation data in the test cases, including: determining the target elements corresponding to the interactive operation data based on element location information in the user behavior data; the element location information includes at least one of Cascading Style Sheet selectors, document path language selectors, and location coordinates.

[0013] In the aforementioned implementation process, the multi-strategy composite element localization algorithm, by constructing a multi-layered, progressive localization solution ranging from precise to fuzzy and from intelligent to traditional methods, ensures the stability of front-end automated testing. It significantly improves the robustness and success rate of automated test scripts, enabling test cases to reliably execute through hash verification or similarity matching even when facing non-disruptive changes such as common front-end page style adjustments and partial component reconstructions. By introducing AI models for semantic and visual feature similarity judgment, the adaptability and intelligence level of automated testing are enhanced. Finally, a reliable backup plan is provided, achieving a balance between localization success rate, execution efficiency, and system resource consumption, thereby comprehensively improving the convenience, reliability, and sustainability of regression testing.

[0014] Optionally, in this embodiment, the priority order of different element location algorithms from high to low is as follows: main strategy location algorithm, hash value matching algorithm, artificial intelligence similarity algorithm, and backup strategy algorithm; in the process of using element location algorithms to determine the target element corresponding to the interactive operation data in the test case, the element location algorithms are used in sequence according to the priority order from high to low to determine the target element, until any element location algorithm successfully determines the target element, and the subsequent element location algorithms are terminated.

[0015] In the above implementation process, Optionally, in this embodiment of the application, in the initial state, the corresponding interactive operation is performed based on the target element to complete the test replay, including: configuring network request interception rules according to network request data so that the interface request initiated during the test replay is intercepted and the response data in the record is returned; after locating the target element, the corresponding simulation operation is performed according to the corresponding interactive operation data to complete the test replay.

[0016] In the aforementioned implementation process, the intelligent interception of network requests and precise simulation of operations implemented during the replay phase together constitute a stable, reliable, and efficient automated testing environment. By intercepting network requests and returning preset responses, test failures caused by backend service instability, data changes, or network latency are reduced, allowing test results to focus on the frontend logic itself, thus improving the stability and execution efficiency of automated testing. Since the exact same request and response data is used for each replay, a high degree of consistency in test conditions is ensured, enhancing the controllability and repeatability of the testing process.

[0017] Optionally, in this embodiment of the application, after completing the test replay, the method further includes: generating a visual video corresponding to the user's original operation by restoring user behavior data; the visual video is a simulated screen recording or element recording sequence of the user during the operation process; simultaneously recording the replay operation process video during the execution of the replay test cases; comparing and analyzing the replay operation process video and the visual video to identify differences in operation steps or page responses; generating a test report based on the replay operation process video, the visual video, and the differences, the test report displaying the replay operation process video and the visual video, and showing the differences; the report also includes execution logs, exception information, and exception information based on artificial intelligence analysis.

[0018] In the above implementation process, the efficiency of automated test result analysis was improved by introducing a visual comparison and intelligent report generation mechanism. This transforms failure localization into an intuitive visual difference comparison, enabling developers and testers to quickly and intuitively understand where and why deviations occurred during playback, much like reviewing surveillance footage. This improves the efficiency of identifying discrepancies and enhances report readability.

[0019] Optionally, in this embodiment, obtaining user behavior data in the application under test includes: listening to DOM events by integrating a collection SDK into the application under test, and obtaining user interaction operation data and environment context data in the application under test; intercepting network request data through a request proxy; locating elements in the listened DOM events using an element positioning algorithm, and obtaining element positioning information; the element positioning information includes at least one of a Cascading Style Sheet selector, a document path language selector, position coordinates, and a unique hash value; the unique hash value is generated based on element attributes.

[0020] In the aforementioned implementation process, the integrated automated data collection and intelligent location information generation scheme lays a high-quality and highly available data foundation for front-end automated testing. By automatically capturing real user operation flows and their complete context through a non-intrusive SDK, subjective omissions and errors inherent in manual recording are avoided, improving the realism and diversity of test scenarios and enhancing the automation, comprehensiveness, and accuracy of raw test data acquisition.

[0021] Secondly, embodiments of this application also provide a front-end regression testing device, comprising: The data acquisition module is used to acquire user behavior data in the application under test; user behavior data includes interaction data, network request data, environmental context data, and element location information. The annotation module is used to semantically annotate user behavior data based on element location information, through predefined rules and / or semantic understanding models, to obtain semantic information of user behavior data; the semantic information is used to characterize the functions triggered in the user behavior data. The grouping module is used to group labeled user behavior data based on network request data to obtain user behavior data groups for a single page path. The cleaning module is used to filter out test cases corresponding to a single page path based on semantic information from user behavior data sets. The replay module is used to restore the test environment to its initial state based on the environmental context data in the test cases; it uses an element location algorithm to determine the target element corresponding to the interactive operation data in the test cases; and in the initial state, it executes the corresponding interactive operation based on the target element to complete the test replay.

[0022] Thirdly, embodiments of this application also provide a computer program product, including computer program instructions, which are executed by a processor to perform the method provided in the first aspect or any implementation thereof.

[0023] Fourthly, embodiments of this application also provide an electronic device, including: a processor and a memory, the memory storing computer program instructions, which are executed by the processor to perform the method provided in the first aspect or any implementation thereof.

[0024] Fifthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, perform the method provided in the first aspect or any implementation thereof.

[0025] The front-end regression testing method, program product, electronic device, and storage medium provided in this application can automatically and continuously collect test case materials from real online traffic and intelligently filter them, saving a significant amount of cost and time spent on manually writing and maintaining test scripts, and improving the efficiency of test case construction and maintenance. By introducing multi-strategy element positioning and AI-assisted matching, the test scripts have a stronger ability to adapt to local changes in the front-end page, reducing the risk of large-scale test case failures caused by page structure changes, and enhancing the stability and reliability of automated testing. Since the test cases originate from user behavior and undergo semantic-based intelligent filtering, they focus on covering core and high-frequency business scenarios, improving the accuracy and business relevance of test coverage. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating a front-end regression testing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the front-end regression testing device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application.

[0030] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0031] Traditional test cases are tightly coupled with the page structure. Even minor changes to the page can cause a large number of test cases to become invalid, requiring manual repair and re-entry, resulting in extremely high test case maintenance costs and difficulty in maintaining the freshness and effectiveness of the test cases. This invention uses an intelligent selector algorithm and an element attribute hashing algorithm to simultaneously and imperceptibly collect online user behavior. Through dual intelligent labeling and AI judgment, it automatically cleans and aggregates massive amounts of test cases, significantly reducing test case maintenance costs.

[0032] Traditional regression testing for middle and back-end systems relies heavily on manual execution. With the increasing number of pages and the growing complexity of business scenarios, manual regression testing requires significant time and manpower, resulting in low efficiency, missed critical functionalities, and frequent online issues. This invention utilizes intelligent selector generators and event collectors to automate the recording of user actions. Combined with intelligent playback technology based on multi-strategy element location algorithms, it compresses the traditionally days-long manual regression testing process into just a few hours.

[0033] Traditional automated testing relies primarily on a single element location strategy, which can easily lead to element location failures when the page structure changes, resulting in poor test stability. This invention addresses this by employing a multi-strategy element location algorithm that combines CSS selectors, XPath selectors, position coordinates, and other location methods, while also incorporating AI similarity matching technology, significantly improving the success rate and stability of element location.

[0034] When common components or core modules are refactored, traditional testing methods cannot accurately assess the impact on related page functions. Relying on manual experience is prone to omissions, leading to increasingly high regression testing costs after refactoring, and impact assessment becomes a technical black box. This invention uses dependency test case management algorithms and URL path analysis technology to automatically establish dependencies between pages, accurately identify the impact of code changes, and verify them.

[0035] Currently, most end-to-end automated testing solutions for middle and back-end systems on the market are traditional testing solutions based on manual recording of test cases. They rely solely on testers manually operating the pages to record test scripts. For middle and back-end scenarios with a large number of pages or rapid iterations (such as agile development iteration models), it is impossible to guarantee the coverage and timeliness of test cases, and it is difficult to continuously ensure test quality.

[0036] Furthermore, most solutions currently available on the market are element positioning strategies based on single selectors. In real-world enterprise development, even minor changes to the page structure can cause a large number of test cases to fail, resulting in extremely high maintenance costs. It is also impossible to measure whether the impact of a certain functional module or a code change on the entire system has been adequately tested and verified.

[0037] Furthermore, many solutions currently available only support static test case management. However, in many scenarios, such as rapid business iteration or code refactoring, static test cases not only fail to achieve the desired regression testing results, but also lead to a decrease in test coverage due to test case failures and maintenance delays, making it difficult to maintain the freshness and effectiveness of test cases.

[0038] This application provides a front-end regression testing method that automatically collects real user operation behaviors through an SDK, and combines multi-strategy element positioning algorithms and AI similarity matching technology to achieve automatic generation, intelligent maintenance, and accurate replay of test cases. This improves the technical problems of high test case maintenance costs, inaccurate test coverage, and difficulty in impact assessment in traditional testing solutions, and significantly enhances the testing efficiency and quality assurance capabilities of the back-end system.

[0039] Before introducing the front-end regression testing method of this application, let's first introduce the concepts mentioned in this application: Smart Selector Generator: A multi-strategy element positioning technology that can generate various types of selectors for DOM elements, including CSS (Cascading Style Sheets) selectors, XPath (XML Path Language) selectors, attribute selectors, and position selectors, solving the problem of traditional single selectors becoming ineffective after page changes.

[0040] Event Collector: A user behavior capture component based on the browser's native event API, supporting real-time collection of various event types such as clicks, input, scrolling, and keyboard input, and enabling page-level event isolation and management.

[0041] Request Proxy: Based on XMLHttpRequest and Fetch API network request interception technology, it immediately intercepts network requests when the page loads, intelligently filters static resources, and distinguishes between requests from the main application and sub-applications.

[0042] Data compression processor: Based on the pako library, this test data compression and storage technology supports the compression, storage, decompression, and restoration of action and request data, significantly reducing storage space and network transmission costs.

[0043] Element attribute hash algorithm: A hash algorithm that generates unique identifiers based on elements and their parent attributes. It combines element tags, text content, and attribute information to generate SHA-256 hash values, improving the accuracy and stability of element location.

[0044] Page state management mode: An event and request isolation management mode based on page ID, which maintains an independent event queue and request queue for each page, solving the data chaos problem when switching pages in SPA applications.

[0045] Route change monitoring mode: Based on the History API proxy, this technology monitors pushState, replaceState, popstate, and hashchange events to achieve compatibility with various routing frameworks.

[0046] Playwright Test Execution Engine: An automated test execution engine based on Playwright, supporting concurrent execution of test cases in multi-browser environments, and providing complete test lifecycle management and error handling mechanisms.

[0047] Element positioning algorithm: Combining multiple positioning strategies such as codegen, CSS, XPath, and position coordinates, the algorithm has a fault-tolerant mechanism that tries different positioning methods in order of priority, which greatly improves the success rate of element positioning.

[0048] AI (Artificial Intelligence) Similarity Matching Algorithm: Based on AI model, this element similarity calculation technology intelligently judges the element matching degree by comparing element features during recording and playback, thus solving the problem of positioning failure caused by changes in page structure.

[0049] Click effect visualization: Dynamically generate click effect indicators during test replay, including coordinate crosshairs, click animations, and coordinate labels, providing an intuitive visualization of the test execution process.

[0050] Browser Context Manager: Based on Playwright's browser context lifecycle management technology, it supports concurrent execution of multiple instances, automatic resource cleanup, and exception recovery, ensuring the stability of the test environment.

[0051] Dynamic resource interception: Intercepts and simulates network requests during test replay, supports static resource replacement, API response simulation, and error injection, improving the controllability and stability of the test.

[0052] Please see Figure 1 The illustration shows a flowchart of a front-end regression testing method provided in an embodiment of this application. The front-end regression testing method provided in this application can be applied to electronic devices, which may include physical devices such as servers, PCs, tablets, or smartphones, or virtual devices such as virtual machines or containers. The electronic device can be a single device, a combination of multiple devices, or a cluster of a large number of devices. The front-end regression testing method may include: Step S110: Obtain user behavior data in the application to be tested; user behavior data includes interaction operation data, network request data, environmental context data, and element location information.

[0053] Step S120: Based on element location information, semantically annotate user behavior data through predefined rules and / or semantic understanding models to obtain semantic information of user behavior data; semantic information is used to characterize the functions triggered in user behavior data.

[0054] Step S130: Based on the network request data, group the labeled user behavior data to obtain user behavior data groups for a single page path.

[0055] Step S140: For the user behavior data group, based on semantic information, filter out the test cases corresponding to the single page path.

[0056] Step S150: Based on the environment context data in the test case, restore the test environment to its initial state; use the element location algorithm to determine the target element corresponding to the interactive operation data in the test case; in the initial state, perform the corresponding interactive operation based on the target element to complete the test replay.

[0057] In step S110: First, the data collection SDK (Software Development Kit) needs to be integrated into the application to be tested. The data collection SDK runs on the browser side and is responsible for collecting multi-dimensional data. With the user's explicit consent and in compliance with relevant regulations, user behavior data is collected through the data collection SDK. Interaction data can be obtained by listening to the browser's DOM (Document Object Model) events, such as listening to click events to record click operations, listening to input events to record text input, and listening to scroll events to record page scrolling, thereby accurately recording the user's interaction data with page elements.

[0058] The collection of network request data is achieved by proxying (or rewriting) the browser's native XMLHttpRequest and FetchAPI. The proxy layer intercepts all sent HTTP / HTTPS requests and synchronously records their complete URL (Uniform Resource Locator), request method, request parameters, response headers, response body, and timestamp of the request, thereby collecting the network request data behind the operation.

[0059] Contextual data refers to the application's running state when an operation occurs. The collection SDK actively records the current page's URL, routing history, and key content in the browser's local storage mechanism. This data collectively defines the initial scenario of the user session.

[0060] Element location information is a set of multi-dimensional "addresses" generated for each manipulated DOM element, enabling it to be found again in future page changes. For example, a smart selector generator component can synchronously calculate the target element's CSS (Cascading Style Sheets) selector, XPath (XML Path Language) expression, and coordinates relative to the viewport. Alternatively, a hash algorithm (such as SHA-256) can be used to combine element tags, text, attributes, and features of its parent element to generate a unique attribute hash value, forming a redundant and robust set of location information. It should be noted that all user behavior data collected in this embodiment has been explicitly consented to by the user and complies with relevant regulations.

[0061] In step S120: First, the predefined rule annotation is based on a pre-built "component-function" component rule mapping table. For example, the mapping rule may specify that elements with the class name containing "ant-btn-primary" correspond to the "primary button", and "role="searchbox" corresponds to the "search box". The system will match the element location information (such as class name, attribute, and tag name) obtained in step S110 with the "component-function" component rule mapping table. If the match is successful, the matched function tag will be used as semantic information (such as "click [submit button]").

[0062] For complex, custom, or dynamically generated component operations that cannot be covered by rule bases, semantic understanding models can be used for annotation. A semantic understanding model is a trained machine learning model, such as one based on the Transformer architecture. The input to a semantic understanding model includes the element location information of the current operation, the context of adjacent operation sequences, and possible network request characteristics. By analyzing these comprehensive features, the model outputs a semantic description of the operation's intent. This "rule-first, AI-backup" approach balances processing efficiency with the generalization ability of annotation, enabling massive amounts of user behavior to be accurately converted into semantic information representing their triggering functions.

[0063] In step S130, the semantically labeled, discrete user behavior data stream is effectively organized. The core basis for grouping is the page path information carried in the network request data. In the application under test, user interaction or page switching is usually accompanied by sending network requests to the server or triggering front-end route updates. The URLs of these requests (such as the front-end route / dashboard corresponding to https: / / example.com / dashboard or / api / user / profile) clearly identify the functional page currently occupied by the user.

[0064] By analyzing the network requests associated with each user behavior record, the key URLs representing page paths can be extracted (usually filtering out query parameters and hash fragments). Subsequently, the system organizes all user behavior data sharing the same key page path along with semantic information into a single user behavior data group. This step can be understood as organizing massive amounts of disorganized behavior logs into structured data sets corresponding to specific functional interfaces, using page URLs as units.

[0065] The goal of step S140 is to filter out the most representative and comprehensive operation sequences from the massive, potentially repetitive behavior records of each page (i.e., each user behavior data group) and solidify them into standard test cases for that page. The filtering process is based on the semantic information generated in step S120.

[0066] By analyzing the semantic information of all user behavior data within a user behavior data set, we can statistically identify all functional types involved on the page, such as "login," "search," "filter form," and "submit form." We can also statistically analyze the frequency with which each type of function is triggered by different user actions. Based on this, we apply a quality assessment algorithm. The core logic of this algorithm is to prioritize user behavior data that covers a wider range of functionalities, i.e., has more semantic tags and covers high-frequency core functions.

[0067] For example, a sequence of operations that includes "search," "advanced filtering," and "export data" typically has a higher coverage value than user behavior data that only includes multiple "searches." In this way, a concise but comprehensive set of test cases can be automatically filtered from real user behavior, forming the basis of automated scripts for regression testing of the page.

[0068] In step S150: Using the environment context data saved in the test case, the automated script initializes a clean browser instance and accurately restores the initial state at the time of recording, including injecting the same cookies, setting data in LocalStorage / SessionStorage, and navigating to the specified initial URL, thereby ensuring that the test start point is consistent with the real user operation.

[0069] The execution engine then processes each interaction in the test cases sequentially. The element location algorithm can employ a multi-strategy, fault-tolerant approach. For example, for each interaction, it might attempt to use the most accurate locator generated during recording (such as a structured selector); if this fails due to page redesign, a backup strategy is used, such as calculating the hash value of the current candidate element and comparing it with the recorded value; if still unsuccessful, an AI similarity matching model based on computer vision or DOM features can be invoked to find the most similar element; finally, basic methods such as coordinate clicks can be used as a fallback. Once the target element is successfully located, the engine executes the corresponding interaction on it, such as simulating clicks or text input, strictly adhering to the timing of the operations. Regression testing is conducted in an isolated environment where network requests can be intercepted and the recorded response returned, thereby improving the stability and repeatability of the tests and automating the replay verification of the entire test case flow.

[0070] In the implementation of the above embodiments: Test case materials can be automatically and continuously collected and intelligently filtered from real online traffic, saving a significant amount of cost and time spent on manually writing and maintaining test scripts, thus improving the efficiency of test case construction and maintenance. By introducing multi-strategy element positioning and AI-assisted matching, the test scripts have a stronger ability to adapt to local changes in the front-end page, reducing the risk of large-scale test case failures caused by page structure changes, and enhancing the stability and reliability of automated testing. Because the test cases originate from user behavior and undergo semantic-based intelligent filtering, they focus on covering core, high-frequency business scenarios, improving the accuracy and business relevance of test coverage.

[0071] Optionally, in this embodiment of the application, based on element location information, semantic annotation of user behavior data is performed through predefined rules and / or a semantic understanding model to obtain semantic information of the user behavior data, including: The first semantic annotation method is based on a predefined component rule mapping table. The element tags, class names, or attribute keywords in the element location information are matched with the functional components in the component rule mapping table. Based on the successfully matched functional components, semantic annotation is performed on the user behavior data to obtain the semantic information of the user behavior data.

[0072] The component rule mapping table is a structured knowledge base that stores the association rules between UI (user interface) component patterns and their corresponding business function semantics. For example, in the component rule mapping, "Button" corresponds to button operations, "Input" corresponds to text input operations, "Select" corresponds to dropdown lists, and so on. The element location information contains the features required to achieve this matching, such as element tags (e.g., ...). <button>Features include class name, various attributes (such as ID, date, etc.). These features of the target element can be extracted from the current user behavior data and matched against each rule entry in the component rule mapping table. For example, various matching methods can be used, such as exact matching, keyword inclusion matching, or regular expression matching. After a successful match, the functional semantics corresponding to the rule (such as "click the query button") are marked on the user behavior data entry. This method is suitable for applications developed using standard UI component libraries, is highly efficient, and yields deterministic results.

[0073] The second semantic annotation method involves inputting element location information, interaction operation data, and network request data from user behavior data into a trained semantic understanding model. This yields a semantic description of the functions triggered in the user behavior data, which is output by the semantic understanding model. The user behavior data is then semantically annotated to obtain its semantic information.

[0074] The training method for semantic understanding models can be supervised learning. The training data for semantic understanding models consists of front-end element features, user operation sequences, network request contexts, and their corresponding manually annotated semantics. During annotation, the user behavior data to be annotated is provided to the model as a multimodal input package. The multimodal input package includes element location information, used to describe "what was operated on"; interaction operation data, such as operation type: click, input, and operation value, used to describe "what operation was performed"; and network request data, such as API endpoints triggered before and after the operation and request parameters, used to provide the business context in which the operation occurred.

[0075] Semantic understanding models acquire this information, perform calculations and analysis through their internal neural networks, understand and analyze it, and ultimately output a semantic description in natural language format, such as "the user entered information into the form" or "the user clicked the 'Add to Cart' button." The advantage of this approach lies in leveraging the powerful generalization and contextual understanding capabilities of semantic understanding models, enabling them to handle complex scenarios where rules cannot be defined, ensuring that all user actions receive a reasonable semantic interpretation.

[0076] In the implementation of the above embodiments: user behavior data is semantically annotated using at least one method, enabling the machine to understand the business intent of user operations like a tester. This lays a high-quality semantic foundation for subsequent test case selection and aggregation, improving the automation and semantic accuracy of test case generation. Furthermore, different annotation methods enhance the solution's adaptability to different page styles, rule mapping efficiently handles standardized components, and the semantic understanding model flexibly responds to custom and dynamic content.

[0077] It is understood that the first method for semantic annotation based on a predefined component rule mapping table and the second method for semantic annotation based on a semantic understanding model provided in this application embodiment can be used selectively or in combination. The strategy and order of using the two semantic annotation methods simultaneously are explained below.

[0078] Optionally, in this embodiment, based on element location information, semantic annotation of user behavior data is performed through predefined rules and / or a semantic understanding model to obtain semantic information of the user behavior data. This includes: matching element tags, class names, or attribute keywords in the element location information with functional components in the component rule mapping table based on a predefined component rule mapping table; semantically annotating the user behavior data based on the successfully matched functional components to obtain semantic information of the user behavior data; acquiring user behavior data that fails to match; inputting the element location information, interaction operation data, and network request data in the user behavior data that fails to match into a trained semantic understanding model to obtain a semantic description of the functions triggered in the user behavior data output by the semantic understanding model; and semantically annotating the user behavior data to obtain semantic information of the user behavior data.

[0079] First, semantic annotation is performed using predefined rules. For user behavior data that is successfully matched through the predefined component rule mapping table, the matching functional components are semantically annotated as user behavior data to obtain semantic information of user behavior data.

[0080] Subsequently, for user behavior data that fails to match through the predefined component rule mapping table, semantic understanding models are used for semantic annotation. For example, for complex interactive elements that cannot be covered by the rule mapping table, are custom-defined, or are dynamically generated, and for which no matching functional component can be successfully found in the pre-set component rule mapping table, a trained semantic understanding model can be used to complete automatic annotation.

[0081] In the implementation of the above embodiments: the two annotation methods work in a collaborative mode of "rule first, AI as a backup". First, efficient rule mapping is used for fast annotation; for behavioral data where rule matching fails or the matching confidence is low, a semantic understanding model is automatically used for deep analysis and annotation. This hybrid strategy improves the coverage and accuracy of annotation while ensuring processing speed.

[0082] Optionally, in this embodiment of the application, for the user behavior data group, test cases corresponding to a single page path are selected based on semantic information, including: For user behavior data sets, based on semantic information, determine the types of functions triggered by the user behavior data, the total number of types, and the number of times each type is triggered.

[0083] Classifying and parsing this semantic information allows us to determine the function type by identifying the core actions and objects in the semantic descriptions. For example, function types include "click the submit button," "click the save button," and "enter in the search box," etc. Alternatively, the semantic tags themselves can be used as types, such as "search," "filter," and "submit form." Next, the total number of types is calculated, that is, the total number of different function types appearing on this page. This total number reflects the richness of functions triggered by the user behavior data. For example, the more interactive operations a user performs on this page, the more types are counted. Simultaneously, the trigger count for each type is accumulated, that is, the total frequency of each specific function being triggered by all user behavior records. The trigger count reflects the activity level of each function. This step, through automated data aggregation and statistical analysis, transforms semantic information into quantitative function profiles, providing a data foundation for subsequent intelligent filtering.

[0084] Based on at least one of the following: type, total number of types, and number of triggers for each type, a score is generated for user behavior data. Multiple test cases are then selected based on the score for a single page path. These multiple test cases cover all the functions within the single page path.

[0085] The scoring can take into account multiple dimensions, such as the breadth of functional coverage, which is how many different functional types the behavior sequence covers. The more types the sequence covers, the higher the score, thus obtaining a basic score.

[0086] As one implementation method, the dimension can also include core function coverage, that is, whether the sequence covers the core functions or the preset functions to be tested. Covering the core functions or the preset functions to be tested can get extra points.

[0087] As one implementation method, the simplicity and typicality of the operation path can be considered to reduce the selection of user behavior data that is too convoluted or contains a large number of redundant operations. For user behavior data that meets the criteria of simplicity and typicality, additional points are awarded. Finally, a comprehensive score is calculated for each candidate user behavior data within the group. The selection strategy can choose one or more test cases with the highest scores, or it can adopt methods such as "Top N" or "setting a minimum score" to select a set of high-scoring user behavior data as multiple test cases for that page. These selected test case sets, as a whole, need to ensure that they can cover all functional types in the single page path, thereby achieving the most comprehensive functional regression verification with fewer test cases.

[0088] In the implementation of the above embodiments: by considering multiple aspects such as the type of triggered function, the total number of types, and the number of times each type is triggered, comprehensive test scenarios are automatically and quickly identified and selected from massive amounts of user behavior. This improves upon the potential omissions or subjective biases that may exist in traditional manual test case design, and enhances the efficiency and representativeness of test case set construction. Quantitative analysis based on function type and trigger count ensures that the final generated test case set systematically covers all functional points of the page, especially high-priority core functions or functions under test, thereby effectively improving the completeness and reliability of the test.

[0089] Optionally, in this embodiment, the element location algorithm includes at least one of the following: a primary strategy location algorithm, a hash value matching algorithm, an artificial intelligence similarity algorithm, and a backup strategy algorithm; wherein, the priority order of the different element location algorithms from high to low is: primary strategy location algorithm, hash value matching algorithm, artificial intelligence similarity algorithm, and backup strategy algorithm; in determining the target element corresponding to the interactive operation data in the test case using the element location algorithm, the element location algorithm is used sequentially according to the priority order from high to low until any element location algorithm successfully determines the target element, at which point the subsequent element location algorithm is terminated.

[0090] For example, once the target element is determined by the main strategy positioning algorithm, there is no need to use the subsequent hash value matching algorithm, artificial intelligence similarity algorithm, and backup strategy algorithm. If the target element cannot be determined by the main strategy positioning algorithm, the hash value matching algorithm is used to determine the target element. If the target element still cannot be determined, the artificial intelligence similarity algorithm and the backup strategy algorithm are used in sequence.

[0091] The following explains the various element location algorithms mentioned above: The main strategy localization algorithm is used to determine the target element corresponding to the interactive operation data in the test case, including: obtaining the structured localization sequence generated for the interactive operation data during the user behavior data collection process, the structured localization sequence including the hierarchical path of the interactive operation from the root node to the target element; parsing the hierarchical path to determine the target element.

[0092] The core of the main strategy localization algorithm lies in utilizing a structured localization sequence pre-generated during the user behavior data collection phase. This structured localization sequence is essentially a unique path description, recording the complete chain from the root node of the Document Object Model (DOM) tree (i.e., the document object), through multiple child node hierarchies, to the final target element. This path information is recorded as a localization identifier during data collection, for example, an enhanced XPath or a dedicated selector generated by the recording tool.

[0093] During the playback phase, when performing location tracking, the structured location sequence stored in the test cases is first retrieved. This sequence is then parsed, translating the instructions into navigation actions within the current real-time DOM tree. For example, based on the path's index, tag name, or ID, child nodes are traversed downwards from the root node. If the endpoint element is successfully found along the path, it is identified as the target element. This method offers extremely high speed and accuracy when the page structure remains unchanged.

[0094] The target element corresponding to the interactive operation data in the test case is determined by a hash value matching algorithm. This includes: matching the unique hash value generated by the element attribute in the interactive operation data with a list of hash values ​​corresponding to multiple preset element attributes, and determining the element that matches successfully as the target element corresponding to the interactive operation data.

[0095] Hash matching algorithms rely on calculating and storing a unique hash value for each element being manipulated during the acquisition phase. For example, tag names, text content, class names, key attribute values, and any one or more of their hierarchical features in the DOM tree can be used as element attributes in the interactive data to generate unique hash values. The diversity of element attributes reduces the probability of collisions. The hash matching algorithm can be a cryptographic hash function (such as SHA-256).

[0096] During the playback and localization process, one or more candidate elements can be obtained through traversal. For each candidate element, the same hash algorithm is used to calculate a unique hash value in real time based on its current element attributes. Subsequently, this real-time calculated unique hash value is compared with the original unique hash value generated during data collection and stored in the test cases. When the two hash values ​​are completely identical during the matching process, the candidate element is determined to be the target element being searched, thus completing accurate verification.

[0097] The method of using artificial intelligence similarity algorithm to determine the target element corresponding to the interactive operation data in the test case includes: using artificial intelligence similarity algorithm to generate target feature vectors corresponding to the element attributes in the interactive operation data, calculating the similarity between the target feature vectors and the feature vector sets corresponding to multiple preset element attributes respectively, and determining the target element corresponding to the interactive operation data based on the similarity results.

[0098] Artificial intelligence similarity algorithms are suitable for intelligent fuzzy matching scenarios where changes in page structure render the aforementioned precise matching methods ineffective. An AI similarity algorithm can be a deep learning-based neural network. During playback localization, the AI ​​similarity algorithm first processes the original element attributes and other information recorded in the test cases, extracting and generating a high-dimensional target feature vector. This vector comprehensively encodes multiple features of the element, such as its visual features (color, size, position), semantic features (nearby text, ARIA tags), and structural features.

[0099] The model calculates the similarity between the target feature vector and each candidate feature vector in the set, using methods such as cosine similarity or distance. It selects the candidate element with the highest similarity score, provided that this similarity exceeds a preset confidence threshold; otherwise, the match is considered unsuccessful. This AI-powered similarity algorithm improves the test script's adaptability to non-drastic changes in the page interface.

[0100] The backup strategy algorithm is used to determine the target element corresponding to the interactive operation data in the test case, including: determining the target element corresponding to the interactive operation data based on the element location information in the user behavior data; the element location information includes at least one of Cascading Style Sheet selector, document path language selector and position coordinates.

[0101] The backup strategy algorithm relies on various traditional element positioning information recorded synchronously during the acquisition phase. Element positioning information can include: 1) Cascading Style Sheets (CSS) Selectors, positioning strings constructed using element IDs, classes, attributes, etc.; 2) Document Path Language (XPath) Selectors, positioning expressions that navigate within the document using XML paths; and 3) Position coordinates, i.e., the absolute or relative coordinates of the element relative to the viewport at the time of recording. During playback implementation, when this strategy is required, this pre-stored element positioning information is read from the test cases and tried sequentially. For example, first, a CSS selector is used to search the DOM; if this fails, an XPath expression is used to search; if this still fails, the recorded coordinates of a mouse click may be directly simulated. This progressive positioning search identifies the target element, improving the stability and accuracy of the determination.

[0102] As one implementation method, any one of the element location algorithms can be selected to determine the target element, reducing the consumption of computing resources and improving the location accuracy. Alternatively, multiple element location algorithms can be used without following a priority order. Even if one algorithm has determined the target element, other element location algorithms can still be executed to verify the target element and improve the accuracy.

[0103] In the implementation of the above embodiments: the multi-strategy composite element localization algorithm, by constructing a multi-layered, progressive localization solution ranging from precise to fuzzy and from intelligent to traditional methods, ensures the stability of front-end automated testing. It significantly improves the robustness and success rate of automated test scripts, enabling test cases to reliably execute through hash verification or similarity matching even when facing non-disruptive changes such as common front-end page style adjustments and partial component reconstructions. By introducing AI models for semantic and visual feature similarity judgment, the adaptability and intelligence level of automated testing are enhanced. Finally, a reliable backup plan is provided, achieving a balance between localization success rate, execution efficiency, and system resource consumption, thereby comprehensively improving the convenience, reliability, and sustainability of regression testing.

[0104] Optionally, in this embodiment of the application, in the initial state, performing corresponding interactive operations based on the target element to complete the test replay includes: Configure network request interception rules based on network request data to intercept interface requests initiated during test replay and return the recorded response data. For example, this can be achieved by utilizing the request interception API provided by modern browser automation frameworks or by setting up a global network proxy in the test runtime environment. A network request interception rule is a mapping table from "request characteristics" to "pre-stored responses." For example, a rule could be set to return a login success response recorded in the test case data when a preset request is intercepted.

[0105] Network request interception rules are used to intercept API requests initiated during test replay. When the test script drives the browser to perform an operation and triggers the application code to initiate a network request, this request is captured by a pre-configured interception layer. The interception layer then parses the key characteristics of the request (such as the URL and request method) and matches them against a rule mapping table. If a match is successful, the corresponding response data is read from the record and constructed into a standard HTTP response object, which is then returned to the application code that initiated the request. In this way, the data received by the application during testing is completely consistent with that during recording, achieving controllability of the test environment.

[0106] After locating the target element, the corresponding simulated operation is executed based on the corresponding interaction operation data to complete the test replay.

[0107] After successfully locating the target element on the interactive interface, the system simulates the user's original interaction to drive the application state transition and verify the functionality. The input for this step is interaction operation data, which explicitly records the type of operation and related values. Technically, executing the corresponding simulated operation involves calling the element manipulation APIs provided by the browser's automation tools. For example, for a "click" operation, after locating the target button element, the element's `.click()` method is called, or a composite mouse event is dispatched at the corresponding coordinates; for an "input" operation, the `.fill()` method is first called to clear and fill the text, or a sequence of `keydown`, `input`, and `keyup` events is dispatched to simulate real keyboard input.

[0108] The execution engine drives these actions according to the order and timing of the operations recorded in the test cases. Throughout the sequence, each successful operation propels the application forward, just as a real user would. Once all the steps defined in the test cases have been executed sequentially, a complete test replay is complete. The system then enters the result verification phase, including determining the final state of the page, checking specific outputs, or confirming that no exceptions were thrown, thus completing the automated verification of a functional scenario.

[0109] In the implementation of the above embodiments: the intelligent interception of network requests and the precise execution of simulated operations implemented during the replay phase together constitute a stable, reliable, and efficient automated testing environment. By intercepting network requests and returning preset responses, test failures caused by backend service instability, data changes, or network latency are reduced, allowing test results to focus on the frontend logic itself, thus improving the stability and execution efficiency of automated testing. Since the exact same request and response data is used for each replay, a high degree of consistency in test conditions is ensured, enhancing the controllability and repeatability of the testing process.

[0110] Optionally, in this embodiment of the application, after completing the test playback, the method further includes: By reconstructing user behavior data, a visual video corresponding to the user's original operation is generated; the visual video is a simulated screen recording or a sequence of elements recorded during the user's operation.

[0111] Methods for generating visual videos include simulated screen recording and generating element recording sequences. Simulated screen recording uses a front-end recording and playback library to record serialized snapshots of DOM structure changes, mouse movement paths, and click coordinates at a very high frequency during the capture phase. When video generation is needed, the engine re-parses this serialized data and reconstructs the dynamic changes of the page frame by frame in a "player," ultimately synthesizing a video file or an interactive playback segment.

[0112] Generating element record sequences is a more lightweight visualization method. Key actions (such as clicks and input) from user behavior data are extracted along a timeline, and a visualization frame containing key information is generated for each action. For example, the clicked element is highlighted and the action type is labeled, or the entered text floats next to the input box. These frames are then combined sequentially to form a walkthrough sequence. Both methods generate visualizations that reproduce the visual process and context of the original user actions.

[0113] During the execution of replay test cases, the replay process is recorded synchronously. Synchronous recording means that a parallel recording process is started at the same time as the replay script drives the browser to perform each operation (such as clicking or typing). The recording process usually directly utilizes the video recording API provided by the browser's automation box to record the actual screen of the browser tab or the entire window used for testing.

[0114] The start and end points of the recording are synchronized with the execution cycle of the test cases. The generated playback video objectively records how the application interface responds to the automated script in the test environment, including the rendering status of page elements, the appearance of pop-ups, data loading animations, and all other visual feedback.

[0115] The replay video of the operation process is compared and analyzed with the visual video to identify differences in operation steps or page responses. This comparative analysis is used to discover deviations between the expected behavior (original operation) and the actual behavior (replay operation). The two videos are time-axis aligned to ensure the starting points of the operations correspond. Then, at each key operation step's time point, two frames are extracted and compared. Methods for identifying differences include: comparing pixel or layout changes in the same area of ​​the two frames to identify missing elements, positional shifts, drastic style changes, etc.; or comparing the key states of the page after the operation; and checking whether the expected pop-ups appear or page transitions occur correctly in the replay video. All detected deviations are marked as differences and associated with specific test steps.

[0116] A test report is generated based on the replay video, visualization video, and differences. The test report displays the replay video and visualization video, and shows the differences. The report also includes execution logs, exception information, and exception information based on artificial intelligence analysis.

[0117] The main body of the report uses a side-by-side layout, with a visualization video embedded or linked on the left and a replay video of the operation process embedded or linked on the right, facilitating direct visual comparison. All identified differences are marked on the timeline, for example, by marking points on the video progress bar, accompanied by the caption, "At the 3-second mark, the expected button was blue, but it was actually rendered as gray." The report integrates complete execution logs (including detailed output of each step of location and operation) and runtime exception information. It may also incorporate AI-based analysis, such as inputting differences, error logs, and code change information into a diagnostic model that could output predictions about the cause of the anomaly, such as "this style difference may be related to CSS selector priority adjustments" or "this element failed to be located."

[0118] In the implementation of the above embodiments: by introducing a visual comparison and intelligent report generation mechanism, the efficiency of automated test result analysis is improved. The failure to locate the problem is transformed into an intuitive visual difference comparison, enabling developers and testers to quickly and intuitively understand where and why the deviation occurred during playback, much like watching surveillance footage. This improves the efficiency of locating differences and enhances the readability of the report.

[0119] Optionally, in this embodiment of the application, obtaining user behavior data in the application to be tested includes: By integrating the data collection SDK into the application under test, we can listen to DOM events and obtain user interaction data and environment context data in the application under test; and intercept network request data through a request proxy.

[0120] A lightweight data collection SDK is integrated into the application under test (typically a web front-end application). This SDK usually exists as a JavaScript library, loaded and run during application initialization. One of its main functions is to listen for DOM events. The DOM (Document Object Model) is the programming interface for web page content; any user interaction (such as clicking, typing, or hovering) triggers corresponding DOM events. The data collection SDK uses methods such as `addEventListener` to listen for events on global or specific elements on the target page, thereby accurately obtaining user interaction data. Simultaneously, the SDK collects environmental context data, including but not limited to: the complete URL (Uniform Resource Locator) of the current page, the browser's user agent string, and local storage state readable through the browser API.

[0121] On the other hand, to capture the data stream behind the operations, the SDK integrates a request proxy. This proxy intercepts the network request the instant it is initiated by rewriting or wrapping the browser's native XMLHttpRequest object and Fetch API, thus recording complete network request data without omission, including the request URL, method, request headers and body, response status code, response headers, and response body. Through this listening and proxying mechanism, the SDK achieves panoramic and non-intrusive collection of user behavior.

[0122] An element positioning algorithm is used to locate elements in the monitored DOM events and obtain element positioning information. The element positioning information includes at least one of the following: Cascading Style Sheet selector, Document Path Language selector, position coordinates, and unique hash value. The unique hash value is generated based on element attributes.

[0123] Whenever the collection SDK listens to a DOM event and captures its target element, the system immediately starts the element positioning algorithm to calculate and generate a set of redundant element positioning information for that element. This set of information is intended to provide multiple possible means to reposition the element when the page may change in the future.

[0124] Cascading Style Sheets (CSS) selectors: Analyze an element's id, class, attributes, and its structure in the DOM tree to generate a concise and unique CSS selector string. Document Path Language Selector: Generates an XPath expression describing the complete path from the document root node to the element. Position Coordinates: Records the precise pixel coordinates of the element relative to the browser viewport at the time of recording, typically including information about the element's bounding rectangle. Unique Hash Value: Extracts a set of attributes from an element, including tag name, text content, and key attributes. A cryptographic hash function is used to process this string of attribute values ​​to generate a fixed-length, unique hash value.

[0125] Hash values, acting as a "digital fingerprint" of an element, are independent of the DOM structure and strongly correlated only with the element's own characteristics. By synchronously generating and storing at least one of the above-mentioned positioning information, the system ensures that even if the page structure, style, or attributes change partially during subsequent playback, the success rate of relocating the element can still be greatly improved through algorithmic matching (such as hash comparison) or strategy degradation (such as using coordinates).

[0126] In the implementation of the above embodiments: the integrated automated data collection and intelligent location information generation scheme lays a high-quality and highly available data foundation for front-end automated testing. By automatically capturing real user operation flows and their complete context through a non-intrusive SDK, subjective omissions and errors from manual recording are avoided, improving the realism and diversity of test scenarios and enhancing the automation, comprehensiveness, and accuracy of raw test data acquisition.

[0127] Please see Figure 2 The diagram shown is a structural schematic of the front-end regression testing device provided in an embodiment of this application; this embodiment of the application provides a front-end regression testing device 200, including: The data acquisition module 210 is used to acquire user behavior data in the application under test; the user behavior data includes interaction operation data, network request data, environmental context data, and element location information; The annotation module 220 is used to semantically annotate user behavior data based on element location information, through predefined rules and / or a semantic understanding model, to obtain semantic information of the user behavior data; the semantic information is used to characterize the functions triggered in the user behavior data. Grouping module 230 is used to group labeled user behavior data based on network request data to obtain user behavior data groups for a single page path. The cleaning module 240 is used to filter out test cases corresponding to a single page path based on semantic information from the user behavior data group. The replay module 250 is used to restore the test environment to its initial state based on the environmental context data in the test cases; it uses an element location algorithm to determine the target element corresponding to the interactive operation data in the test cases; and in the initial state, it executes the corresponding interactive operation based on the target element to complete the test replay.

[0128] It should be understood that this device corresponds to the aforementioned front-end regression testing method embodiment and is capable of performing the various steps involved in the aforementioned method embodiment. The specific functions of this device can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0129] Please see Figure 3 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. An electronic device 300 provided in this application includes a processor 310 and a memory 320. The memory 320 stores machine-readable instructions executable by the processor 310. When the machine-readable instructions are executed by the processor 310, the method described above is performed.

[0130] Figure 3 The components shown can be implemented using hardware, software, or a combination thereof. Electronic device 300 may be a physical device, such as a server or PC, or a virtual device, such as a virtual machine or virtualization container. Furthermore, electronic device 300 is not limited to a single device; it can be a combination of multiple devices or a cluster of numerous devices.

[0131] This application also provides a storage medium storing a computer program, which is executed by a processor to perform the above-described method.

[0132] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0133] This application also provides a computer program product, including computer program instructions, which are executed by a processor to perform the method described above.

[0134] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, given the several embodiments provided in this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0135] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0136] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.< / button>

Claims

1. A front-end regression testing method, characterized in that, include: Obtain user behavior data in the application under test; The user behavior data includes interaction operation data, network request data, environmental context data, and element location information; Based on the element location information, the user behavior data is semantically annotated using predefined rules and / or a semantic understanding model to obtain semantic information of the user behavior data; the semantic information is used to characterize the functions triggered in the user behavior data. Based on the network request data, the labeled user behavior data is grouped to obtain user behavior data groups for a single page path. For the user behavior data set, test cases corresponding to the single page path are selected based on the semantic information; Based on the environmental context data in the test cases, the test environment is restored to its initial state; An element location algorithm is used to determine the target element corresponding to the interactive operation data in the test case. In the initial state, the corresponding interactive operation is performed based on the target element to complete the test playback.

2. The method according to claim 1, characterized in that, Based on the element location information, semantic annotation is performed on the user behavior data using predefined rules and / or a semantic understanding model to obtain the semantic information of the user behavior data, including: Based on a predefined component rule mapping table, the element tags, class names, or attribute keywords in the element location information are matched with the functional components in the component rule mapping table. Based on the successfully matched functional components, the user behavior data is semantically annotated to obtain the semantic information of the user behavior data. and / or; The element location information, interaction operation data, and network request data in the user behavior data are input into the trained semantic understanding model to obtain the semantic description of the function triggered in the user behavior data output by the semantic understanding model, and the user behavior data is semantically annotated to obtain the semantic information of the user behavior data.

3. The method according to claim 1, characterized in that, Based on the element location information, semantic annotation is performed on the user behavior data using predefined rules and / or a semantic understanding model to obtain the semantic information of the user behavior data, including: Based on a predefined component rule mapping table, the element tags, class names, or attribute keywords in the element location information are matched with the functional components in the component rule mapping table. Based on the successfully matched functional components, the user behavior data is semantically annotated to obtain the semantic information of the user behavior data. The user behavior data that failed to match is obtained. The element location information, interaction operation data and network request data in the user behavior data that failed to match are input into the trained semantic understanding model. The semantic understanding model outputs a semantic description representing the function triggered in the user behavior data. The user behavior data is then semantically annotated to obtain the semantic information of the user behavior data.

4. The method according to claim 1, characterized in that, For the user behavior data set, based on the semantic information, test cases corresponding to the single page path are selected, including: For the user behavior data group, based on the semantic information, determine the type of function triggered by the user behavior data, the total number of types, and the number of times each type is triggered; A score is generated for the user behavior data based on at least one of the type, the total number of types, and the number of times each type is triggered. Multiple test cases are then selected based on the score for the single page path. The multiple test cases cover all the functions in the single page path.

5. The method according to claim 1, characterized in that, The element location algorithm includes at least one of the following: primary strategy location algorithm, hash value matching algorithm, artificial intelligence similarity algorithm, and backup strategy algorithm; The method of using the main strategy localization algorithm to determine the target element corresponding to the interactive operation data in the test case includes: obtaining a structured localization sequence generated for the interactive operation data during the user behavior data collection process, wherein the structured localization sequence includes the hierarchical path of the interactive operation from the root node to the target element; parsing the hierarchical path to determine the target element; The method of using the hash value matching algorithm to determine the target element corresponding to the interactive operation data in the test case includes: matching the unique hash value generated by the element attribute in the interactive operation data with a list of hash values ​​corresponding to multiple preset element attributes, and determining the element that successfully matches as the target element corresponding to the interactive operation data. The method of using the artificial intelligence similarity algorithm to determine the target element corresponding to the interactive operation data in the test case includes: using the artificial intelligence similarity algorithm to generate a target feature vector corresponding to the element attribute in the interactive operation data; calculating the similarity between the target feature vector and a set of feature vectors corresponding to multiple preset element attributes respectively; and determining the target element corresponding to the interactive operation data based on the similarity results. The backup strategy algorithm is used to determine the target element corresponding to the interactive operation data in the test case, including: determining the target element corresponding to the interactive operation data based on the element location information in the user behavior data; the element location information includes at least one of Cascading Style Sheets selector, document path language selector, and position coordinates.

6. The method according to claim 1, characterized in that, in, The priority order of the different element location algorithms, from high to low, is as follows: main strategy location algorithm, hash value matching algorithm, artificial intelligence similarity algorithm, and backup strategy algorithm. In the process of using element location algorithms to determine the target element corresponding to the interactive operation data in the test case, the element location algorithms are used in sequence according to the priority order from high to low to determine the target element, until any element location algorithm successfully determines the target element, and the subsequent element location algorithms are terminated.

7. The method according to claim 1, characterized in that, In the initial state, based on the target element, corresponding interactive operations are performed to complete the test replay, including: Configure network request interception rules based on the network request data, so that the interface requests initiated during the test replay are intercepted and the response data in the record is returned; After locating the target element, the corresponding simulated operation is executed based on the corresponding interactive operation data to complete the test playback.

8. The method according to claim 1, characterized in that, After completing the test replay, the method further includes: By reconstructing the user behavior data, a visual video corresponding to the user's original operation is generated; the visual video is a simulated screen recording or a sequence of element records of the user during the operation process. During the execution and playback of the test cases, the playback process is recorded simultaneously. The playback operation process video is compared and analyzed with the visualization video to identify differences in operation steps or page responses; A test report is generated based on the replay operation process video, the visualization video, and the differences. The test report displays the replay operation process video and the visualization video, and shows the differences. The report also includes execution logs, exception information, and exception information based on artificial intelligence analysis.

9. The method according to claim 1, characterized in that, The acquisition of user behavior data in the application to be tested includes: By integrating the data collection SDK into the application under test, DOM events are monitored to obtain the user's interaction data and environment context data in the application under test; the network request data is intercepted through a request proxy. The element positioning algorithm is used to locate elements in the monitored DOM events and obtain the element positioning information; the element positioning information includes at least one of Cascading Style Sheets selector, Document Path Language selector, position coordinates and unique hash value; the unique hash value is generated based on element attributes.

10. A computer program product, characterized in that, It includes computer program instructions that are executed by a processor to perform the method as described in any one of claims 1 to 9.

11. An electronic device, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, perform the method as described in any one of claims 1 to 9.

Images