Interface test method and device, electronic equipment and storage medium

Abstract

This disclosure provides an interface testing method, apparatus, electronic device, and storage medium, belonging to the field of computer technology. The method includes: storing a scenario identifier and data protocol of a business scenario in a sandbox environment of a business server; sending test instructions to multiple types of terminals, each test instruction instructing a terminal to access the business server using a weak login state identifier and returning a test interface image, which is obtained by rendering and screenshotting the business scenario based on the data protocol of the business scenario in the sandbox environment; and judging the test interface images of multiple terminals based on a reference interface image of the business scenario to obtain interface test results, which are used to indicate the degree of consistency of the business scenario on different types of terminals. This method can promptly detect whether the interfaces displayed on multiple terminals are consistent, solving the technical problem of difficulty in timely detection of cross-terminal interface differences during the testing phase, and improving the consistency and testing efficiency of multi-terminal products.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an interface testing method, apparatus, electronic device, and storage medium. Background Technology

[0002] User interface testing refers to testing whether the functional modules of the user interface meet the requirements. The goal is to ensure that these modules provide users with the necessary access or browsing capabilities. Its core principle is to use automated scripts to simulate real user operation paths, performing actions such as clicking, inputting, and swiping on the front-end interface, and verifying whether UI elements, text, and interaction logic meet expectations, thereby achieving automated verification of application functionality.

[0003] However, the front-end interface of modern application systems has expanded from a single form to a multi-terminal coexistence system, including web pages, iOS applications, Android applications, and mini-programs. While the interaction logic of the same business process may be similar across different terminals, the UI structure, style, and data rendering methods can differ significantly. For example, the order placement process may exhibit differences in button placement, discount presentation, and dynamic component loading order across different terminals. These cross-platform interface inconsistencies, as a typical functional anomaly, are often difficult to detect in a timely manner during the testing phase. Summary of the Invention

[0004] This disclosure provides an interface testing method, apparatus, electronic device, and storage medium that can promptly detect inconsistencies in interfaces displayed across multiple platforms. It solves the technical problem of difficulty in timely detecting cross-platform interface differences during the testing phase, improving the consistency and testing efficiency of multi-platform products. The technical solution of this disclosure is as follows: According to one aspect of the embodiments of this disclosure, an interface testing method is provided, comprising: The scenario identifier and data protocol of the business scenario are stored in the sandbox environment of the business server. The data protocol includes the business data and business rules of the business scenario. The business server is used to provide backend support for the business scenario. The sandbox environment is isolated from the production environment. Test instructions are sent to multiple types of terminals respectively. Each test instruction contains the scenario identifier. Different types of terminals have different operating environments. Each test instruction is used to instruct the terminal to access the business server using a weak login state identifier and return a test interface image. The test interface image is obtained by rendering and screenshotting the business scenario based on the data protocol of the business scenario in the sandbox environment. Based on the reference interface image of the business scenario, the test interface images of multiple terminals are judged to obtain the interface test results. The interface test results are used to indicate the degree of consistency of the business scenario on different types of terminals.

[0005] According to another aspect of the embodiments of this disclosure, an interface testing apparatus is provided, comprising: The storage unit is configured to store the scenario identifier and data protocol of the business scenario in a sandbox environment of the business server. The data protocol includes the business data and business rules of the business scenario. The business server is used to provide backend support for the business scenario. The sandbox environment is isolated from the production environment. The sending unit is configured to send test instructions to multiple types of terminals respectively. Each test instruction contains the scenario identifier. Different types of terminals have different operating environments. Each test instruction is used to instruct the terminal to access the business server using a weak login state identifier and return a test interface image. The test interface image is obtained by rendering and screenshotting the business scenario based on the data protocol of the business scenario in the sandbox environment. The discrimination unit is configured to execute a reference interface image based on the business scenario, discriminate test interface images of multiple terminals, and obtain interface test results. The interface test results are used to indicate the degree of consistency of the business scenario on different types of terminals.

[0006] In some embodiments, the apparatus further includes: The first processing unit is configured to perform atomic-level decomposition of the business process to obtain at least one atomic scenario for each business stage in the business process. An atomic scenario is the smallest indivisible business unit obtained after layer-by-layer decomposition of the business process. The unit combines the atomic scenarios in multiple business stages of the business process to obtain multiple business scenarios. Each business scenario includes one atomic scenario from each business stage of the business process. For any business scenario among the multiple business scenarios, the unit constructs a scenario identifier and data protocol for the business scenario.

[0007] In some embodiments, the apparatus further includes: The second processing unit is configured to perform at least one of the following: Based on the target combination among atomic scenarios in the multiple business stages, business scenarios that do not contain the target combination are removed from the multiple business scenarios. The target combination includes at least two atomic scenarios, and the number of times the at least two atomic scenarios appear together is higher than a threshold. Based on business constraint rules, scenarios that do not conform to the business constraint rules are removed from the multiple business scenarios. The business constraint rules are used to indicate the restrictive conditions on the logical relationship between parameters in the business scenario when the business logic is conforming. Based on the weights of the multiple business scenarios, business scenarios with weights lower than the weight threshold are removed from the multiple business scenarios. The weight of each business scenario is used to indicate the importance of the business scenario. Based on the interface issues that occurred within a historical time period, the business scenarios containing the interfaces where the interface issues occurred are removed from the multiple business scenarios.

[0008] In some embodiments, the discrimination unit is configured to perform the following actions for any of the plurality of terminals: determining the structural similarity between the reference interface image and the test interface image based on the reference interface image and the test interface image of the business scenario; determining the difference region between the reference interface image and the test interface image based on the structural similarity; and performing semantic recognition on the difference region between the reference interface image and the test interface image using a large language model to obtain the interface test result.

[0009] In some embodiments, the discrimination unit is configured to perform the following: determine a difference region image based on the difference region between the reference interface image and the test interface image; input the difference region image and context information into the large language model to obtain the interface test result; wherein the context information is used to describe the semantic identity and scene information of the difference region shown in the difference region image.

[0010] In some embodiments, the difference region image includes any of the following: The test interface image contains a region image of the difference area; The test interface image includes a region image of the difference region and a region image of the reference interface image corresponding to the difference region; The image obtained by annotating the difference regions in the test interface image, wherein the annotations are used to indicate the difference regions; The image obtained by annotating the difference regions in the test interface image and the image obtained by annotating the regions in the reference interface image that correspond to the difference regions.

[0011] In some embodiments, the discrimination unit is configured to perform the following: using the large language model, based on a preset dynamic element identifier, to remove dynamic elements from the difference region image; and using the large language model to process the difference region image after removing dynamic elements to obtain the interface test result.

[0012] In some embodiments, the apparatus further includes: The third processing unit is configured to execute the large language model and output the confidence level of the interface test results; for interface test results with a confidence level exceeding a preset value, to annotate the difference region images based on the interface test results and store the annotated difference region images and corresponding context information; and to periodically adjust the parameters in the large language model based on the newly annotated difference region images and corresponding context information.

[0013] In some embodiments, the interface test results include difference description text and category labels. The difference description text is used to describe inconsistencies in the interface of the business scenario using natural language, and the category labels are used to indicate the category to which the inconsistency belongs.

[0014] According to another aspect of the embodiments of this disclosure, an electronic device is provided, the electronic device comprising: One or more processors; Memory used to store the executable program code of the processor; The processor is configured to execute the program code to implement the aforementioned interface testing method.

[0015] According to another aspect of the present disclosure, a computer-readable storage medium is provided that, when the program code in the computer-readable storage medium is executed by the processor of an electronic device, enables the electronic device to perform the above-described interface testing method.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the above-described interface testing method.

[0017] This disclosure provides an interface testing method. By storing the scene identifiers and data protocols of the business scenarios to be tested in a sandbox environment of the business server, the test data has version management and precise reproduction capabilities, eliminating differences in test data across multiple terminals and making multi-terminal interface rendering comparable. Then, through the cooperation of weak login state identifiers and scene identifiers, concurrent driving of multiple types of terminals is achieved. All terminals render based on the same data protocol, which not only eliminates comparison noise caused by differences in backend data and establishes a clean experimental environment for interface consistency verification, but also enables complete business process simulation without relying on production login state, reducing the coupling of the test environment and enabling simultaneous verification across multiple terminals. Using a reference interface image as a benchmark, the rendering results of multiple terminals are systematically judged, directly identifying interface inconsistencies. This not only accurately locates real defects caused by front-end code or compatibility issues, improving the stability of interface testing, but also quickly detects whether the interfaces displayed on multiple terminals are consistent, solving the technical problem of difficulty in timely detection of cross-terminal interface differences during the testing phase, and improving the consistency and testing efficiency of multi-terminal products.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0020] Figure 1 This is a schematic diagram illustrating the implementation environment of an interface testing method according to an exemplary embodiment.

[0021] Figure 2 This is a flowchart illustrating an interface testing method according to an exemplary embodiment.

[0022] Figure 3 This is a flowchart illustrating another interface testing method according to an exemplary embodiment.

[0023] Figure 4 This is a schematic diagram illustrating an atomic scene split according to an exemplary embodiment.

[0024] Figure 5 This is a flowchart illustrating another interface testing method according to an exemplary embodiment.

[0025] Figure 6 This is a block diagram illustrating an interface testing apparatus according to an exemplary embodiment.

[0026] Figure 7This is a block diagram illustrating a terminal according to an exemplary embodiment.

[0027] Figure 8 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0029] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0030] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this disclosure are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the scene identifiers, interface images, and other related data involved in various business scenarios in this disclosure were obtained with full authorization.

[0031] Figure 1 This is a schematic diagram illustrating an implementation environment for an interface testing method according to an exemplary embodiment. Taking an electronic device provided as a test server as an example, see [link to example]. Figure 1 The implementation environment specifically includes: terminal 101, business server 102, and test server 103.

[0032] In some embodiments, terminal 101 is at least one of devices such as a smartphone, smartwatch, desktop computer, laptop, MP3 player, MP4 player, and laptop computer. An application requiring interface testing is installed and runs on terminal 101. This application can be a multimedia application, social application, game application, browser, or embedded application, etc., and this disclosure does not limit this. Users can log in to the application through terminal 101 to obtain the services provided by the application. Terminal 101 can connect to service server 102 and test server 103 respectively via wireless or wired network, receive test commands sent by test server 103, and respond to the test commands by retrieving service data from service server 102 and displaying the corresponding interface. Then, terminal 101 captures an interface image and returns it to test server 103, which determines whether the interface image is accurate.

[0033] Terminal 101 generally refers to one of a plurality of terminals; this embodiment uses terminal 101 as an example. Those skilled in the art will understand that the number of terminals can be more or less. For example, there may be several terminals, or dozens or hundreds of terminals, or even more. This disclosure does not limit the number of terminals or the type of device.

[0034] In some embodiments, the business server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), big data, and artificial intelligence platforms. The business server 102 provides background services for applications. In some embodiments, the business server 102 undertakes the primary computing work, and the terminal 101 undertakes the secondary computing work; or, the business server 102 undertakes the secondary computing work, and the terminal 101 undertakes the primary computing work; or, the business server 102 and the terminal 101 collaborate on computing using a distributed computing architecture.

[0035] In some embodiments, the test server 103 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), big data, and artificial intelligence platforms. The test server 103 is used to test the accuracy of the application's interface. In some embodiments, the test server 103 undertakes the primary computing work, and the terminal 101 undertakes the secondary computing work; or, the test server 103 undertakes the secondary computing work, and the terminal 101 undertakes the primary computing work; or, the test server 103 and the terminal 101 collaborate on computing using a distributed computing architecture.

[0036] Figure 2 This is a flowchart illustrating an interface testing method according to an exemplary embodiment, see [link to flowchart]. Figure 2 The interface testing method is applied to the test server and includes the following steps: In step 201, the test server stores the scenario identifier and data protocol of the business scenario in the sandbox environment of the business server. The data protocol includes the business data and business rules of the business scenario. The business server is used to provide backend support for the business scenario. The sandbox environment is isolated from the production environment.

[0037] In this disclosure, a business scenario refers to a series of coherent operations and interactions performed by a user to achieve a specific goal within a particular business context. In the field of interface testing, a business scenario includes a complete user task flow (i.e., a complete sequence of operations from start to finish), the specific business rules and data states involved, and a clearly defined expected result. A business scenario could be a scenario such as "using a platform coupon to complete an order" or "a new user placing an order with a stacked discount for the first time," etc., and this disclosure does not limit this to any particular scenario.

[0038] A scenario identifier (e.g., `data_id`) is a globally unique ID used to uniquely identify a specific business scenario that needs to be tested. A data protocol is a structured data file (e.g., JSON / YAML) that defines the business data (e.g., product price, coupon amount) and business rules (e.g., discount calculation logic, inventory status) required to render the interface of a business scenario. A business server refers to the backend service that provides data interfaces and business logic calculations for the frontend application. A sandbox environment refers to an isolated storage space or special mode for the business server. It is completely isolated from the production environment, does not operate on the real database, and is specifically used for testing. Its core function is to return a pre-defined data protocol based on the scenario identifier, rather than real data.

[0039] In other words, this solution first designs the business scenario (such as "placing an order using a platform discount coupon") on the test server. The test server then "deploys" or "registers" the scenario identifier and corresponding data protocol to the sandbox environment of the business server. At this point, the sandbox environment is ready: when it receives a request with a specific scenario identifier, it can accurately return the preset test data so that the terminal can display the corresponding interface.

[0040] In step 202, the test server sends test instructions to multiple types of terminals respectively. Each test instruction contains a scenario identifier. Different types of terminals have different operating environments. Each test instruction is used to instruct the terminal to access the business server using a weak login state identifier and return a test interface image. The test interface image is obtained by rendering and screenshotting the business scenario based on the data protocol of the business scenario in the sandbox environment.

[0041] In this embodiment, "terminal type" refers to different front-end runtime environments (i.e., the runtime environment of the interface to be tested), such as iOS App, Android App, mini-programs embedded in an application, web browsers, etc. The test instruction is a command issued by the test server, such as "Please [terminal name] access [URL / page], and include this [scenario identifier]." The weak login state identifier is a generic, sufficiently authorized test account token or cookie. It is only used for login verification through the business server and is not bound to any specific user's real data, thus achieving "decoupling of login state and test data." The test interface image refers to a standardized screenshot automatically captured after the terminal completes page rendering.

[0042] The test server simultaneously sends test commands to multiple terminals, including iOS devices, Android emulators, mini-program debuggers, and browsers. Each terminal, upon receiving the command, logs in using a weak login identifier and accesses the target page, including a scene identifier (such as data_id) in the request. Then, the sandbox environment of the business server receives the request, parses the scene identifier, looks up the corresponding data protocol locally, and returns it as an interface response to the front-end (terminal). The applications on each terminal use this identical data protocol for rendering, ultimately presenting the corresponding business status interface. Finally, each terminal automatically takes a screenshot and uploads this test interface image back to the test server.

[0043] In step 203, the test server judges the test interface images of multiple terminals based on the reference interface image of the business scenario to obtain the interface test results. The interface test results are used to indicate the degree of consistency of the business scenario on different types of terminals.

[0044] In this embodiment, the reference interface image refers to a screenshot of the interface deemed "benchmark" or "correct". Typically, the rendering result of a specific terminal (such as a browser) can be specified as the benchmark, or a screenshot that passed the previous test can be used as the benchmark; this embodiment does not limit this. Then, the test server performs similarity detection on the reference interface image and test interface images from multiple terminals. If the test interface image of a terminal and the reference interface image are within the allowable error range, the test interface image is determined to be consistent with the reference interface image; otherwise, the test interface image of the terminal has differences. In this case, a test report can be generated, including the location and type of the difference (such as layout misalignment, inconsistent colors, or missing elements) and specific terminal information, to notify the testers of the detection results.

[0045] This disclosure provides an interface testing method. By storing the scene identifiers and data protocols of the business scenarios to be tested in a sandbox environment of the business server, the test data has version management and precise reproduction capabilities, eliminating differences in test data across multiple terminals and making multi-terminal interface rendering comparable. Then, through the cooperation of weak login state identifiers and scene identifiers, concurrent driving of multiple types of terminals is achieved. All terminals render based on the same data protocol, which not only eliminates comparison noise caused by differences in backend data and establishes a clean experimental environment for interface consistency verification, but also enables complete business process simulation without relying on production login state, reducing the coupling of the test environment and enabling simultaneous verification across multiple terminals. Using a reference interface image as a benchmark, the rendering results of multiple terminals are systematically judged, directly identifying interface inconsistencies. This not only accurately locates real defects caused by front-end code or compatibility issues, improving the stability of interface testing, but also quickly detects whether the interfaces displayed on multiple terminals are consistent, solving the technical problem of difficulty in timely detection of cross-terminal interface differences during the testing phase, and improving the consistency and testing efficiency of multi-terminal products.

[0046] In some embodiments, the method further includes: The business process is atomically decomposed to obtain at least one atomic scenario for each business stage in the business process. An atomic scenario is the smallest indivisible business unit obtained after the business process is decomposed layer by layer. Combining atomic scenarios from multiple business stages in a business process yields multiple business scenarios, each of which includes one atomic scenario from each business stage in the business process. For any business scenario among multiple business scenarios, construct the scenario identifier and data protocol for the business scenario.

[0047] The solution provided in this disclosure constructs a scientific scenario generation system by decomposing business processes into indivisible minimum business units and then systematically combining them to generate complete business scenarios. Atomized decomposition ensures the completeness and comprehensive coverage of business scenarios, with each test point corresponding to a specific business function, avoiding the scenario omission problem common in traditional testing. Secondly, the rule-based scenario generation method can automatically generate massive amounts of test cases, improving efficiency by tens of times compared to manual scenario design, making it particularly suitable for complex business processes. Finally, structured scenario identifiers and data protocols provide standardized input for subsequent automated testing, enabling the entire testing system to be scalable and maintainable, laying a solid foundation for large-scale automated interface testing.

[0048] In some embodiments, the method further includes at least one of the following: Based on the target combination between atomic scenarios in multiple business stages, remove business scenarios that do not contain the target combination from multiple business scenarios. The target combination includes at least two atomic scenarios, and the number of times the at least two atomic scenarios appear together is higher than the number threshold. Based on business constraint rules, scenarios that do not conform to the business constraint rules are removed from multiple business scenarios. Business constraint rules are used to indicate the restrictions on the logical relationship between parameters in a business scenario when the business logic is met. Based on the weights of multiple business scenarios, business scenarios with weights below the weight threshold are removed from the multiple business scenarios. The weight of each business scenario is used to indicate the importance of the business scenario. Based on the UI issues that occurred within a historical time period, we removed business scenarios that contained the UI issues from multiple business scenarios.

[0049] The solution provided in this disclosure employs a multi-dimensional scenario tailoring mechanism, including intelligent filtering strategies based on combination frequency, business constraint rules, weight evaluation, and historical issues. This step optimizes the process from "full generation" to "precise filtering." Firstly, by removing non-critical scenarios and illegal combinations, the number of generated scenarios is reduced to a reasonable range, avoiding test case explosion and significantly reducing test execution costs while maintaining coverage. Secondly, intelligent filtering based on business rules and historical data ensures precise allocation of test resources, prioritizing the verification of high-frequency, important scenarios and known problem scenarios, thus improving test effectiveness and issue discovery rate. Thirdly, the weighting mechanism and association with historical issues enable the testing system to have adaptive evolution capabilities, dynamically adjusting test priorities based on business importance and historical quality data, achieving intelligent configuration of test resources.

[0050] In some embodiments, based on a reference interface image of a business scenario, test interface images of multiple terminals are judged to obtain interface test results, including: For any terminal among multiple terminals, determine the structural similarity between the reference interface image and the test interface image based on the business scenario. Based on structural similarity, the regions of difference between the reference interface image and the test interface image are determined. Using a large language model, semantic recognition is performed on the regions of difference between the reference interface image and the test interface image to obtain the interface test results.

[0051] The solution provided in this disclosure constructs a three-step analysis framework: "structural similarity calculation → difference region localization → large-scale model semantic recognition." First, a structural similarity algorithm is used to quickly compare the reference image and the test image as a whole, accurately calculating the similarity value between the two images and automatically locating the boundary regions where visual differences exist. This step solves the problems of low efficiency and easy omission of subtle differences in manual comparison. Then, based on the located difference regions, a large-scale language model is introduced for in-depth semantic analysis, enabling the system to understand "what the difference is," rather than just "where the difference is." This conversion from pixel-level to semantic-level gives the test results real business meaning, allowing testers to directly fix problems based on semantic descriptions without having to spend time interpreting abstract pixel difference maps. Simultaneously, the entire analysis process is fully automated, compressing the analysis work that originally required tens of minutes or even hours from human experts to seconds, enabling rapid feedback in continuous integration environments.

[0052] In some embodiments, semantic recognition is performed on the difference regions between the reference interface image and the test interface image using a large language model to obtain interface test results, including: Based on the difference regions between the reference interface image and the test interface image, determine the difference region image; Inputting difference region images and contextual information into a large language model yields interface test results. The contextual information is used to describe the semantic identity and scene information of the difference regions shown in the difference region images.

[0053] The solution provided in this disclosure further clarifies the specific form of the difference region image and the input format of the large language model, specifying a dual input mode of "difference region image + contextual information." This step significantly improves the accuracy and practicality of semantic recognition in large models. Specifically, the accurate extraction of the difference region image provides focused visual material for model analysis, avoiding computational redundancy and information interference caused by inputting the entire screenshot into the model. This allows the model to concentrate on analyzing key change areas, which is particularly important when dealing with complex interfaces. Secondly, the introduction of contextual information is a key innovation for achieving accurate semantic understanding. By providing the model with the "semantic identity" and "scene information" of the difference region, the model can perform difference analysis in conjunction with the business context. This context-enhanced analysis method significantly reduces the misjudgment rate, enabling AI to understand business semantics rather than just visual patterns.

[0054] In some embodiments, the difference region image includes any of the following: The test interface image contains regions with differences. The test interface image contains the region image of the difference area and the reference interface image contains the region image of the region corresponding to the difference area; The image obtained after annotating the difference regions in the test interface image; the annotations are used to indicate the difference regions. The image obtained by annotating the difference regions in the test interface image and the image obtained by annotating the regions corresponding to the difference regions in the reference interface image.

[0055] The solution provided in this disclosure offers multiple processing modes for images of differing regions, including flexible image input methods such as single images, comparison images, and labeled images. This step provides large-scale model analysis with sufficient contextual information. Different image input modes adapt to different analysis needs: single image mode facilitates rapid identification of specific differences, comparison image mode is beneficial for analyzing trends, and labeled image mode can accurately locate the range of differences. This flexibility ensures that the system can provide the most suitable analysis method regardless of whether it is a simple text change or a complex layout adjustment. At the same time, the diverse input formats provide rich data support for subsequent model training and optimization, which helps to improve the generalization and accuracy of image recognition capabilities.

[0056] In some embodiments, the difference region image and contextual information are input into a large language model to obtain interface test results, including: By using a large language model and based on preset dynamic element identifiers, dynamic elements in the difference region image are removed; By using a large language model, the difference region image after removing dynamic elements is processed to obtain the interface test results.

[0057] The solution provided in this disclosure proposes a noise filtering mechanism based on dynamic element identification. By automatically filtering non-functional differences during the semantic analysis stage using preset dynamic element templates, the accuracy of testing can be improved. This not only effectively solves the long-standing problem of dynamic content interference in automated interface testing, preventing non-fixed elements such as timestamps, carousels, and user avatars from being misjudged as defects; secondly, by combining dynamic element identification with business scenarios, it can distinguish between "expected dynamic changes" and "abnormal content changes," achieving intelligent false alarm filtering; finally, this mechanism reduces reliance on manual review, making automated test results more reliable and providing technical support for fully automated quality access control in continuous integration environments.

[0058] In some embodiments, the method further includes: The confidence level of the interface test results is output using a large language model. For interface test results with a confidence level exceeding a preset value, the difference region images are labeled based on the interface test results, and the labeled difference region images and corresponding context information are stored. The parameters in the large language model are periodically adjusted based on newly annotated difference region images and corresponding contextual information.

[0059] The solution provided in this disclosure employs a closed-loop model evolution mechanism of "confidence assessment + automatic annotation + periodic training." Training samples are formed through automatic annotation of high-confidence results, and the model is periodically fine-tuned, constructing a self-learning testing system. Specifically, the confidence mechanism enables intelligent hierarchical processing of test results; high-confidence results directly enter the alarm process, while low-confidence results trigger manual review, optimizing human-machine collaboration efficiency. Secondly, automated data annotation and storage create a continuously growing high-quality training dataset, providing a constant source of fuel for model optimization. Finally, periodic incremental training allows the system's recognition capability to continuously improve with usage time, achieving self-adaptation and self-evolution of the testing system. In the long run, this will significantly reduce maintenance costs and improve testing accuracy.

[0060] In some embodiments, the interface test results include discrepancy description text and category labels. The discrepancy description text is used to describe inconsistencies in the interface of a business scenario using natural language, and the category labels are used to indicate the category to which the inconsistency belongs.

[0061] The solution provided in this disclosure specifies a dual output format for interface test results—a combination of natural language description and classification labels—ensuring that the test results are both human-readable and easy to automate. Specifically, the natural language description allows personnel in different roles, such as development, testing, and product, to intuitively understand the problem, enabling them to quickly locate and fix defects without specialized training. Simultaneously, standardized classification labels provide structured data support for problem management, statistical analysis, and quality measurement, facilitating the establishment of defect trend analysis and quality early warning mechanisms. This dual output format bridges the gap between automated test results and manual workflows, maintaining the intelligent advantages of AI analysis while being compatible with the collaborative habits of existing R&D systems, significantly improving the practical value and implementation effectiveness of the test results.

[0062] The above Figure 2 The diagram shown is merely the basic process of this disclosure. The following section will further elaborate on the solution provided in this disclosure based on a specific implementation method. Figure 3 This is a flowchart illustrating another interface testing method according to an exemplary embodiment. Taking an electronic device provided as a test server as an example, see [link to example]. Figure 3 The interface testing method includes: In step 301, the test server constructs data for multiple business scenarios, with each business scenario's data including a scenario identifier and a data protocol.

[0063] In this embodiment, the test server performs atomic-level decomposition of the business process to obtain at least one atomic scenario for each business stage within the business process. An atomic scenario is the smallest indivisible business unit obtained after layer-by-layer decomposition of the business process. Then, the test server combines the atomic scenarios from multiple business stages to obtain multiple business scenarios, each including one atomic scenario from each business stage of the business process. For any business scenario among the multiple business scenarios, the test server constructs a scenario identifier and data protocol for that business scenario.

[0064] The solution provided in this disclosure constructs a scientific scenario generation system by decomposing business processes into indivisible minimum business units and then systematically combining them to generate complete business scenarios. Atomized decomposition ensures the completeness and comprehensive coverage of business scenarios, with each test point corresponding to a specific business function, avoiding the scenario omission problem common in traditional testing. Secondly, the rule-based scenario generation method can automatically generate massive amounts of test cases, improving efficiency by tens of times compared to manual scenario design, making it particularly suitable for complex business processes. Finally, structured scenario identifiers and data protocols provide standardized input for subsequent automated testing, enabling the entire testing system to be scalable and maintainable, laying a solid foundation for large-scale automated interface testing.

[0065] For example, Figure 4 This is a schematic diagram illustrating an atomic scene splitting according to an exemplary embodiment. See also Figure 4 The process is broken down into atomic scenarios based on business processes. Marketing types in the order placement process include live streamer red envelopes, cross-store discounts, and platform instant discounts; among them, platform coupons can be further enumerated into different discount types (such as platform discount coupons and discount coupons), and each specific type is an atomic scenario; in addition to discount types, other business variables (client type, product type, payment method, etc.) can also be broken down into atomic scenarios.

[0066] In some embodiments, the test server can also filter the combined business scenarios according to certain rules to avoid data explosion. Four filtering methods are exemplified below, but are by no means limited to these.

[0067] The first filtering method involves the test server identifying and focusing on high-frequency atomic scenario combinations. Based on target combinations between atomic scenarios across multiple business stages, the server removes business scenarios that do not contain a target combination. A target combination consists of at least two atomic scenarios, and the frequency of these two atomic scenarios occurring together exceeds a preset threshold. This method identifies and focuses on high-frequency atomic scenario combinations. The system statistically analyzes the co-occurrence frequency of atomic scenarios from historical test data or business logs. When certain atomic scenario combinations (e.g., "using platform coupons" and "choosing cash on delivery") co-occur more frequently than a preset "frequency threshold," they are defined as "target combinations." After generating Cartesian product scenarios, the test server retains only those business scenarios containing at least one "target combination" from the full scenario pool, filtering out scenarios that do not contain any high-frequency combinations. This ensures that the test case set prioritizes covering business paths that most closely resemble real user behavior patterns, investing limited test resources (such as cloud real devices and computing time) in core process combinations that have the greatest impact on the business and are most likely to be encountered by users, avoiding wasting resources on a large number of low-frequency or even theoretically possible but rarely occurring marginal combinations.

[0068] The second filtering method involves the test server using business constraint rules to remove scenarios from multiple business scenarios that do not conform to these rules. Business constraint rules indicate the restrictions on the logical relationships between parameters within a business scenario when the business logic is followed. For example, a business constraint rule might state that if a user's level is 'non-member,' the discount type cannot be 'exclusive member coupon'; or, if the delivery method is 'in-store pickup,' the address information must be 'empty' or 'designated store,' etc. This embodiment does not impose any limitations on these rules. This method ensures that the generated scenarios conform to basic business logic and rules. Business constraint rules are predefined logical conditions describing the legal relationships between business parameters. After generating scenarios, the system acts like a "business rule validator," checking whether the parameter combinations for each scenario violate these hard constraints and eliminating all non-compliant scenarios. It removes invalid scenarios from the source that are logically impossible, have no testing value, or may even cause front-end or server-side anomalies (for example, testing a non-member using a member-exclusive function will inevitably fail and has no testing significance). This ensures that every executed test case is a theoretically viable and verifiable business path, making the test results more valuable and avoiding test environment anomalies caused by invalid requests.

[0069] The third filtering method involves the test server assigning weights to multiple business scenarios. Scenarios with weights below a certain threshold are removed from the pool of scenarios. Each scenario's weight indicates its importance. This method prioritizes test resources based on business importance. Each business scenario is assigned a "weight" value during generation or configuration, reflecting factors such as its business criticality, the scope of its impact on users, and the amount of money involved. The system sets a "weight threshold," and after generating a large number of scenarios, only those with weights above this threshold are retained for testing. The weights can be automatically calculated by the rule engine or manually configured. For example, core main processes (such as normal payment and order placement for ordinary users), high-value processes (such as purchasing luxury goods), and global functions affecting all users (such as login and homepage loading) have higher weights, while peripheral functions (such as feedback pages), functions for specific user groups (such as a rarely used invoice type), and experimental functions (small-scale functions in A / B testing) have lower weights. When testing resources (time, environment) are absolutely limited (e.g., rapid regression testing after each code commit), this approach ensures that the most critical functions, those that would result in the greatest loss if they fail, are 100% covered and verified. This is a typical quality assurance strategy that prioritizes ensuring the "core path is absolutely solid," and then considers the comprehensiveness of coverage on that basis.

[0070] The fourth filtering method involves the test server removing business scenarios containing interfaces with UI issues from multiple business scenarios based on historical UI problems encountered within a given time period. This method leverages historical quality data to optimize future testing focus. The system records and analyzes UI issues (bugs) discovered during historical testing cycles, marking the specific interfaces or UI components corresponding to these issues. Before the start of a new testing cycle, the system temporarily removes (or downgrades) business scenarios containing interfaces in "high-frequency known issue areas." For example, in the last two months of testing, the discount information calculation area of ​​the "order confirmation page" accumulated 15 cross-platform inconsistencies due to dynamic data splicing issues; the system marked the "order confirmation page - discount information area" as a "high-frequency historical issue interface"; after a new round of full scenario generation, the system identifies all business scenarios that will redirect to the "order confirmation page" (e.g., the final step in all order payment processes); these scenarios may be temporarily filtered out and not included in the list for this automated execution. The reason is that this area has many known issues, many of which are currently being fixed. Repeated testing on this area has a low ROI (Return on Investment) and may generate numerous invalid alerts due to unresolved old issues, interfering with the discovery of new problems. This approach makes testing activities no longer static and repetitive, but dynamically adjustable based on the system's quality status. By avoiding repeated investment in "known, resolving quality gaps," valuable testing resources can be saved for exploring new business scenarios, verifying new code modifications, or conducting targeted confirmation testing on fixed issues, making the entire testing activity more intelligent and efficient.

[0071] The above-mentioned filtering methods can be applied individually or in combination freely, and this disclosure does not limit this.

[0072] The solution provided in this disclosure employs a multi-dimensional scenario tailoring mechanism, including intelligent filtering strategies based on combination frequency, business constraint rules, weight evaluation, and historical issues. This step optimizes the process from "full generation" to "precise filtering." Firstly, by removing non-critical scenarios and illegal combinations, the number of generated scenarios is reduced to a reasonable range, avoiding test case explosion and significantly reducing test execution costs while maintaining coverage. Secondly, intelligent filtering based on business rules and historical data ensures precise allocation of test resources, prioritizing the verification of high-frequency, important scenarios and known problem scenarios, thus improving test effectiveness and issue discovery rate. Thirdly, the weighting mechanism and association with historical issues enable the testing system to have adaptive evolution capabilities, dynamically adjusting test priorities based on business importance and historical quality data, achieving intelligent configuration of test resources.

[0073] In step 302, the test server stores the scenario identifier and data protocol of the business scenario in the sandbox environment of the business server. The data protocol includes the business data and business rules of the business scenario. The business server is used to provide backend support for the business scenario. The sandbox environment is isolated from the production environment.

[0074] In this embodiment, the designed test scenario data is securely and completely injected into an isolated sandbox environment, establishing a stable and controllable backend data foundation for multi-terminal rendering. This process begins with the test server encapsulating the atomically split and combined business scenarios. Each scenario is assigned a globally unique scenario identifier (data_id) and a structured data protocol. The protocol content covers complete business data (such as user information, product details, and discount rules) and business logic rules (such as inventory deduction strategies and discount stacking logic). This data is then transmitted in batches through an encrypted channel to a sandbox server completely isolated from the production environment. This server has the same service architecture as the production environment but uses an independent database and middleware, ensuring that testing activities do not interfere with real business data.

[0075] After receiving data, the sandbox environment performs intelligent storage and mapping operations. The system uses the scenario identifier as the primary key to persistently store the data protocol in a dedicated database, while simultaneously building a fast index in the memory cache to support high-frequency queries. More importantly, the sandbox environment parses the business rules in the data protocol, loads them into the built-in business rule engine, and configures the corresponding API interface response templates. For example, when the data protocol contains the rule "coupons can be combined with points," the sandbox environment's order calculation service automatically enables the corresponding combination logic. When a terminal requests the order details interface, the sandbox service layer retrieves the corresponding data protocol from storage based on the scenario identifier carried in the request and encapsulates and returns it according to the production interface format. To ensure data readiness, the test server initiates a verification request. Only after confirming that the data stored in the sandbox environment is completely consistent with the original protocol is the scenario marked as testable, thus completing a full closed loop from test data definition to sandbox service readiness.

[0076] In step 303, the test server sends test instructions to multiple types of terminals respectively. Each test instruction contains a scenario identifier. Different types of terminals have different operating environments. Each test instruction is used to instruct the terminal to access the business server using a weak login state identifier and return a test interface image. The test interface image is obtained by rendering and screenshotting the business scenario based on the data protocol of the business scenario in the sandbox environment.

[0077] In this embodiment, the process is the execution phase of multi-terminal concurrent rendering and standardized image acquisition, which constructs a highly automated pipeline from instruction distribution to image generation. This phase starts with a standardized instruction package constructed by the test server. The instruction explicitly specifies the target terminal type (such as iOS simulator, Android real device, Chrome browser), scene identifier, weak login token, and specific operation commands (such as opening a specific deep link or accessing a specified URL). These instructions are distributed in parallel to various terminals in the device pool through a distributed task queue. The scheduling system intelligently allocates device resources and monitors the execution status to ensure reliability and efficiency in high-concurrency scenarios.

[0078] Upon receiving the instruction, the terminal device first completes authentication in the sandbox environment using a weak login token, establishing a test session decoupled from real user data. The terminal then launches the application and navigates to the target page; all initiated network requests carry a scene identifier parameter. When a request arrives in the sandbox environment, the request interception module accurately extracts the identifier, retrieves the corresponding data protocol from local storage, and dynamically injects the preset test data into the API response. The application uses this controlled data to execute rendering logic completely consistent with actual production, generating a UI interface that matches the expected business state. After rendering, the system triggers a standardized screenshot operation, ensuring that key elements are loaded, network requests are idle, and animations are stable: unifying the window size, clearing interfering elements, capturing full-screen and key area images, and finally compressing the image, embedding metadata, and uploading it to cloud storage to generate a globally accessible image URL. The entire process is equipped with real-time monitoring and fault tolerance mechanisms; any terminal failure triggers automatic retry or alarms, ensuring the successful completion of large-scale multi-terminal testing tasks.

[0079] In step 304, the test server judges the test interface images of multiple terminals based on the reference interface image of the business scenario to obtain the interface test results. The interface test results are used to indicate the degree of consistency of the business scenario on different types of terminals.

[0080] In this embodiment of the disclosure, for any one of multiple terminals, the test server determines the structural similarity between the reference interface image and the test interface image based on the business scenario. Based on the structural similarity, the test server identifies the difference regions between the reference interface image and the test interface image. The test server then uses a large language model to perform semantic recognition on the difference regions between the reference interface image and the test interface image to obtain the interface test results.

[0081] The solution provided in this disclosure constructs a three-step analysis framework: "structural similarity calculation → difference region localization → large-scale model semantic recognition." First, a structural similarity algorithm is used to quickly compare the reference image and the test image as a whole, accurately calculating the similarity value between the two images and automatically locating the boundary regions where visual differences exist. This step solves the problems of low efficiency and easy omission of subtle differences in manual comparison. Then, based on the located difference regions, a large-scale language model is introduced for in-depth semantic analysis, enabling the system to understand "what the difference is," rather than just "where the difference is." This conversion from pixel-level to semantic-level gives the test results real business meaning, allowing testers to directly fix problems based on semantic descriptions without having to spend time interpreting abstract pixel difference maps. Simultaneously, the entire analysis process is fully automated, compressing the analysis work that originally required tens of minutes or even hours from human experts to seconds, enabling rapid feedback in continuous integration environments.

[0082] In the process of recognizing test results through a large language model, the test server determines the difference region image based on the difference regions between the reference interface image and the test interface image. Then, the test server inputs the difference region image and contextual information into the large language model to obtain the interface test results. The contextual information is used to describe the semantic identity and scene information of the difference region shown in the difference region image. The solution provided by this disclosure further clarifies the specific form of the difference region image and the input format of the large language model, and specifies a dual input mode of "difference region image + contextual information". This step design significantly improves the accuracy and practicality of semantic recognition of the large model. That is, the accurate extraction of the difference region image provides focused visual material for model analysis, avoiding the computational redundancy and information interference caused by inputting the entire screenshot into the model, enabling the model to focus on analyzing key change areas, which is especially important when dealing with complex interfaces. Secondly, the introduction of contextual information is a key innovation for achieving accurate semantic understanding. By providing the model with the "semantic identity" and "scene information" of the difference region, the model can perform difference analysis in combination with the business background. This context-enhanced analysis method greatly reduces the misjudgment rate, enabling AI to understand business semantics rather than just visual patterns.

[0083] The difference region image may include any of the following: The first item is to test the image of the region containing the differences in the interface image; The second item is to test the image of the region containing the difference in the interface image and the image of the region corresponding to the difference in the reference interface image. The third item is the image obtained after annotating the difference areas in the test interface image. The annotations are used to indicate the difference areas. The fourth item is the image obtained by annotating the difference areas in the test interface image and the image obtained by annotating the areas corresponding to the difference areas in the reference interface image.

[0084] The solution provided in this disclosure offers multiple processing modes for images of differing regions, including flexible image input methods such as single images, comparison images, and labeled images. This step provides large-scale model analysis with sufficient contextual information. Different image input modes adapt to different analysis needs: single image mode facilitates rapid identification of specific differences, comparison image mode is beneficial for analyzing trends, and labeled image mode can accurately locate the range of differences. This flexibility ensures that the system can provide the most suitable analysis method regardless of whether it is a simple text change or a complex layout adjustment. At the same time, the diverse input formats provide rich data support for subsequent model training and optimization, which helps to improve the generalization and accuracy of image recognition capabilities.

[0085] Then, the process of the test server inputting the difference region image and context information into the large language model to obtain the interface test results includes: the test server removing dynamic elements from the difference region image based on preset dynamic element identifiers using the large language model. The test server then processes the difference region image after removing dynamic elements using the large language model to obtain the interface test results. The solution provided in this disclosure proposes a noise filtering mechanism based on dynamic element identifiers. By automatically filtering non-functional differences during the semantic analysis stage using preset dynamic element templates, the accuracy of testing can be improved. This not only effectively solves the long-standing problem of dynamic content interference in automated interface testing, preventing non-fixed elements such as timestamps, carousels, and user avatars from being misjudged as defects; secondly, by combining dynamic element recognition with business scenarios, it can distinguish between "desirable dynamic changes" and "abnormal content changes," achieving intelligent false alarm filtering; finally, this mechanism reduces the reliance on manual review, making automated test results more reliable and providing technical support for fully automated quality access control in continuous integration environments.

[0086] In some embodiments, interface test results include discrepancy description text and category tags. The discrepancy description text is used to describe inconsistencies in the interface of a business scenario using natural language, and the category tags are used to indicate the category to which the inconsistency belongs. The solution provided in this disclosure specifies a dual output format for interface test results—a combination of natural language description and category tags—ensuring that the test results are both human-readable and easy to automate. That is, the natural language description allows personnel in different roles such as development, testing, and product to intuitively understand the problem and quickly locate and fix defects without special training; at the same time, the standardized category tags provide structured data support for problem management, statistical analysis, and quality measurement, facilitating the establishment of defect trend analysis and quality early warning mechanisms. This dual output format bridges the gap between automated test results and manual workflows, maintaining the intelligent advantages of AI analysis while being compatible with the collaborative habits of existing R&D systems, greatly improving the practical value and implementation effect of test results.

[0087] In some embodiments, the test server can also output the confidence level of the interface test results through a large language model. For interface test results with a confidence level exceeding a preset value, the test server annotates the difference region images based on the interface test results and stores the annotated difference region images and corresponding context information. Then, the test server periodically adjusts the parameters in the large language model based on the newly annotated difference region images and corresponding context information. The solution provided in this disclosure embodiment designs a model evolution closed loop of "confidence assessment + automatic annotation + periodic training," forming training samples through automatic annotation of high-confidence results and periodically fine-tuning the model, thus constructing a self-learning test system. Among these features, the confidence mechanism enables intelligent hierarchical processing of test results. High-confidence results can directly enter the alarm process, while low-confidence results trigger manual review, thus optimizing the efficiency of human-machine collaboration. Secondly, automated data annotation and storage form a continuously growing high-quality training dataset, providing a constant source of fuel for model optimization. Finally, periodic incremental training enables the recognition capability of the entire system to continuously improve with the increase of usage time, realizing the self-adaptation and self-evolution of the testing system. In the long run, this will significantly reduce maintenance costs and improve testing accuracy.

[0088] To more clearly describe the interface testing method provided in the embodiments of this disclosure, the interface testing process is further described below with reference to the accompanying drawings. For example, Figure 5This is a flowchart illustrating another interface testing method according to an exemplary embodiment. First, atomic scenarios are decomposed based on business processes. Then, these atomic scenarios are combined and superimposed on the business processes (Cartesian product, which can be pruned according to rules to avoid explosion), automatically generating a large-scale set of scenario data protocols. Each protocol is stored in a structured form (such as JSON / YAML) and uses a globally unique identifier (such as data_id). The data_id runs throughout the entire end-to-end testing chain and is used for full monitoring in the rendering, screenshot, difference detection, and data annotation chains. The scheduling center (test server) controls multiple environments (cloud real devices, mini-programs, browsers, etc.) to initiate page requests, carrying the data_id. To bypass strict server-side login verification, the system logs in using any user context (not a specific real user). The business server (business system) receives the user session and returns page data. The business server uses the data_id to look up the corresponding data protocol in the sandbox environment and injects the interface response to ensure that the front end obtains complete rendering data. The front-end rendering engine constructs the page according to the protocol data and completes multi-terminal rendering. After rendering is complete, standardized screenshots are automatically captured and uploaded to the CDN for storage. The same data_id can be rendered in parallel across multiple platforms including Web, iOS, Android, HarmonyOS, and WeChat Mini Programs, achieving cross-platform consistency verification. Then, by comparing screenshots corresponding to the same data_id, a fast overall similarity calculation is performed using the Structural Similarity Index (SSIM) to locate visually dissimilar regions (producing bounding boxes). The marked dissimilar regions are then normalized in size, enhanced in contrast, and denoised to generate multimodal input (image + structured context). The image regions and structured context are input into a large multimodal model, which outputs semantic descriptions and classification labels for the dissimilarities (e.g., "text change," "color / style change," "dynamic content noise," "missing button," etc.). The model combines known dynamic element templates (e.g., carousels, timestamps, user avatars) for dynamic noise filtering to avoid misclassifying non-functional dynamic changes as problems. The model's initial judgment and confidence level are recorded. High-confidence results are automatically labeled and entered into the statistics and alert stream; low-confidence results or key information (such as potential functional abnormalities) trigger manual review. The results of manual review are merged with the automatically labeled data to form new labeled samples, which are then stored in the training data warehouse. The newly labeled data is used periodically to perform incremental training / fine-tuning on the multimodal model, improving overall recognition accuracy and recall. The final results are output in a structured report format, including data_id, comparison of main body and swimlane screenshots, highlighted differences, model-determined categories and confidence levels, and manual review records, for rapid problem localization and fixing.

[0089] This disclosure provides an interface testing method. By storing the scene identifiers and data protocols of the business scenarios to be tested in a sandbox environment of the business server, the test data has version management and precise reproduction capabilities, eliminating differences in test data across multiple terminals and making multi-terminal interface rendering comparable. Then, through the cooperation of weak login state identifiers and scene identifiers, concurrent driving of multiple types of terminals is achieved. All terminals render based on the same data protocol, which not only eliminates comparison noise caused by differences in backend data and establishes a clean experimental environment for interface consistency verification, but also enables complete business process simulation without relying on production login state, reducing the coupling of the test environment and enabling simultaneous verification across multiple terminals. Using a reference interface image as a benchmark, the rendering results of multiple terminals are systematically judged, directly identifying interface inconsistencies. This not only accurately locates real defects caused by front-end code or compatibility issues, improving the stability of interface testing, but also quickly detects whether the interfaces displayed on multiple terminals are consistent, solving the technical problem of difficulty in timely detection of cross-terminal interface differences during the testing phase, and improving the consistency and testing efficiency of multi-terminal products.

[0090] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0091] Figure 6 This is a block diagram illustrating an interface testing apparatus according to an exemplary embodiment. See also... Figure 6 The interface testing device includes a storage unit 601, a sending unit 602, and a discrimination unit 603.

[0092] Storage unit 601 is configured to store the scenario identifier and data protocol of the business scenario in the sandbox environment of the business server. The data protocol includes the business data and business rules of the business scenario. The business server is used to provide backend support for the business scenario. The sandbox environment is isolated from the production environment. The sending unit 602 is configured to send test instructions to multiple types of terminals respectively. Each test instruction contains a scenario identifier. Different types of terminals have different operating environments. Each test instruction is used to instruct the terminal to access the business server using a weak login state identifier and return a test interface image. The test interface image is obtained by rendering and screenshotting the business scenario based on the data protocol of the business scenario in the sandbox environment. The discrimination unit 603 is configured to execute a reference interface image based on the business scenario, discriminate the test interface images of multiple terminals, and obtain the interface test results. The interface test results are used to indicate the degree of consistency of the business scenario on different types of terminals.

[0093] In some embodiments, the apparatus further includes: The first processing unit is configured to perform atomic-level decomposition of the business process to obtain at least one atomic scenario in each business stage of the business process. An atomic scenario is the smallest indivisible business unit obtained after layer-by-layer decomposition of the business process. The atomic scenarios in multiple business stages of the business process are combined to obtain multiple business scenarios. Each business scenario includes one atomic scenario from each business stage of the business process. For any business scenario in the multiple business scenarios, a scenario identifier and data protocol for the business scenario are constructed.

[0094] In some embodiments, the apparatus further includes: The second processing unit is configured to perform at least one of the following: Based on the target combination between atomic scenarios in multiple business stages, remove business scenarios that do not contain the target combination from multiple business scenarios. The target combination includes at least two atomic scenarios, and the number of times the at least two atomic scenarios appear together is higher than the number threshold. Based on business constraint rules, scenarios that do not conform to the business constraint rules are removed from multiple business scenarios. Business constraint rules are used to indicate the restrictions on the logical relationship between parameters in a business scenario when the business logic is met. Based on the weights of multiple business scenarios, business scenarios with weights below the weight threshold are removed from the multiple business scenarios. The weight of each business scenario is used to indicate the importance of the business scenario. Based on the UI issues that occurred within a historical time period, we removed business scenarios that contained the UI issues from multiple business scenarios.

[0095] In some embodiments, the discrimination unit 603 is configured to perform the following actions for any of a plurality of terminals: determining the structural similarity between a reference interface image and a test interface image based on a business scenario; determining the difference region between the reference interface image and the test interface image based on the structural similarity; and performing semantic recognition on the difference region between the reference interface image and the test interface image using a large language model to obtain the interface test result.

[0096] In some embodiments, the discrimination unit 603 is configured to perform a process based on the difference region between the reference interface image and the test interface image to determine the difference region image; input the difference region image and context information into the large language model to obtain the interface test result, wherein the context information is used to describe the semantic identity and scene information of the difference region shown in the difference region image.

[0097] In some embodiments, the difference region image includes any of the following: The test interface image contains regions with differences. The test interface image contains the region image of the difference area and the reference interface image contains the region image of the region corresponding to the difference area; The image obtained after annotating the difference regions in the test interface image; the annotations are used to indicate the difference regions. The image obtained by annotating the difference regions in the test interface image and the image obtained by annotating the regions corresponding to the difference regions in the reference interface image.

[0098] In some embodiments, the discrimination unit 603 is configured to perform the following: using a large language model, based on a preset dynamic element identifier, to remove dynamic elements from the difference region image; and using the large language model to process the difference region image after removing dynamic elements to obtain the interface test result.

[0099] In some embodiments, the apparatus further includes: The third processing unit is configured to execute the large language model and output the confidence score of the interface test results; for interface test results with a confidence score exceeding a preset value, the unit annotates the difference region images based on the interface test results and stores the annotated difference region images and corresponding context information; and periodically adjusts the parameters in the large language model based on the newly annotated difference region images and corresponding context information.

[0100] In some embodiments, the interface test results include discrepancy description text and category labels. The discrepancy description text is used to describe inconsistencies in the interface of a business scenario using natural language, and the category labels are used to indicate the category to which the inconsistency belongs.

[0101] This disclosure provides an interface testing device. By storing the scene identifiers and data protocols of the business scenarios to be tested in a sandbox environment of the business server, the test data has version management and accurate reproduction capabilities, eliminating differences in test data across multiple terminals and making multi-terminal interface rendering comparable. Then, through the cooperation of weak login state identifiers and scene identifiers, concurrent driving of multiple types of terminals is realized. All terminals render based on the same data protocol, which not only eliminates comparison noise caused by differences in backend data and establishes a clean experimental environment for interface consistency verification, but also enables complete business process simulation without relying on production login state, reducing the coupling of the test environment and enabling simultaneous verification of multiple terminals. Using a reference interface image as a benchmark, the rendering results of multiple terminals are systematically judged, directly identifying interface inconsistencies. This not only accurately locates real defects caused by front-end code or compatibility issues, improving the stability of interface testing, but also quickly detects whether the interfaces displayed on multiple terminals are consistent, solving the technical problem of difficulty in timely detection of cross-terminal interface differences during the testing phase, and improving the consistency and testing efficiency of multi-terminal products.

[0102] It should be noted that the interface testing device provided in the above embodiments is only illustrated by the division of the above functional units during interface testing. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the electronic device can be divided into different functional units to complete all or part of the functions described above. In addition, the interface testing device and the interface testing method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0103] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0104] When an electronic device is provided as a terminal, Figure 7 This is a block diagram illustrating a terminal 700 according to an exemplary embodiment. The terminal... Figure 7 A structural block diagram of a terminal 700 provided in an exemplary embodiment of this disclosure is shown. The terminal 700 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 700 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0105] Typically, terminal 700 includes a processor 701 and a memory 702.

[0106] Processor 701 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 701 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 701 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 701 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 701 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0107] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 702 are used to store at least one computer program, which is executed by the processor 701 to implement the interface testing method provided in the method embodiments of this application.

[0108] In some embodiments, the terminal 700 may also optionally include a peripheral device interface 703 and at least one peripheral device. The processor 701, memory 702, and peripheral device interface 703 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, and a power supply 708.

[0109] Peripheral device interface 703 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 701 and memory 702. In some embodiments, processor 701, memory 702 and peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 701, memory 702 and peripheral device interface 703 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0110] The radio frequency (RF) circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 704 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. In some embodiments, the RF circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 704 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 704 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0111] Display screen 705 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 705 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 701 for processing. In this case, display screen 705 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 705, disposed on the front panel of terminal 700; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal 700 or in a folded design; in other embodiments, display screen 705 may be a flexible display screen, disposed on a curved or folded surface of terminal 700. Furthermore, display screen 705 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 705 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0112] The camera assembly 706 is used to acquire images or videos. In some embodiments, the camera assembly 706 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash is a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0113] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 701 for processing, or input to the radio frequency circuit 704 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal 700. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0114] Power supply 708 is used to power the various components in terminal 700. Power supply 708 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 708 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0115] Those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on terminal 700, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0116] When electronic devices are provided as servers, Figure 8 This is a block diagram illustrating a server 800 according to an exemplary embodiment. The server 800 can vary significantly due to differences in configuration or performance. It may include one or more Central Processing Units (CPUs) 801 and one or more memories 802. The memory 802 stores at least one line of program code, which is loaded and executed by the processor 801 to implement the interface testing methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 800 may also include other components for implementing device functions, which will not be elaborated upon here.

[0117] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 702 or memory 802 including instructions, which can be executed by the processor 701 of the terminal 700 or the processor 801 of the server 800 to complete the interface testing method described above. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0118] A computer program product includes a computer program / instructions that, when executed by a processor, implement the aforementioned interface testing method.

[0119] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0120] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for testing user interfaces, characterized in that, The method includes: The scenario identifier and data protocol of the business scenario are stored in the sandbox environment of the business server. The data protocol includes the business data and business rules of the business scenario. The business server is used to provide backend support for the business scenario. The sandbox environment is isolated from the production environment. Test instructions are sent to multiple types of terminals respectively. Each test instruction contains the scenario identifier. Different types of terminals have different operating environments. Each test instruction is used to instruct the terminal to access the business server using a weak login state identifier and return a test interface image. The test interface image is obtained by rendering and screenshotting the business scenario based on the data protocol of the business scenario in the sandbox environment. Based on the reference interface image of the business scenario, the test interface images of multiple terminals are judged to obtain the interface test results. The interface test results are used to indicate the degree of consistency of the business scenario on different types of terminals.

2. The interface testing method according to claim 1, characterized in that, The method further includes: The business process is atomically decomposed to obtain at least one atomic scenario for each business stage in the business process. An atomic scenario is the smallest indivisible business unit obtained after the business process is decomposed layer by layer. The atomic scenarios in multiple business stages of the business process are combined to obtain multiple business scenarios, each of which includes one atomic scenario from each business stage of the business process. For any of the multiple business scenarios, construct the scenario identifier and data protocol for that business scenario.

3. The interface testing method according to claim 2, characterized in that, The method further includes at least one of the following: Based on the target combination among atomic scenarios in the multiple business stages, business scenarios that do not contain the target combination are removed from the multiple business scenarios. The target combination includes at least two atomic scenarios, and the number of times the at least two atomic scenarios appear together is higher than a threshold. Based on business constraint rules, scenarios that do not conform to the business constraint rules are removed from the multiple business scenarios. The business constraint rules are used to indicate the restrictive conditions on the logical relationship between parameters in the business scenario when the business logic is conforming. Based on the weights of the multiple business scenarios, business scenarios with weights lower than the weight threshold are removed from the multiple business scenarios. The weight of each business scenario is used to indicate the importance of the business scenario. Based on the interface issues that occurred within a historical time period, the business scenarios containing the interfaces where the interface issues occurred are removed from the multiple business scenarios.

4. The interface testing method according to claim 1, characterized in that, The reference interface image based on the business scenario is used to judge the test interface images of multiple terminals to obtain interface test results, including: For any of the plurality of terminals, based on the reference interface image and the test interface image of the business scenario, determine the structural similarity between the reference interface image and the test interface image; Based on the structural similarity, the difference regions between the reference interface image and the test interface image are determined; Using a large language model, semantic recognition is performed on the difference regions between the reference interface image and the test interface image to obtain the interface test results.

5. The interface testing method according to claim 4, characterized in that, The step of using a large language model to perform semantic recognition on the difference regions between the reference interface image and the test interface image to obtain the interface test results includes: Based on the difference region between the reference interface image and the test interface image, determine the difference region image; The difference region image and context information are input into the large language model to obtain the interface test results. The context information is used to describe the semantic identity and scene information of the difference region shown in the difference region image.

6. The interface testing method according to claim 5, characterized in that, The difference region image includes any of the following: The test interface image contains a region image of the difference area; The test interface image includes a region image of the difference region and a region image of the reference interface image corresponding to the difference region; The image obtained by annotating the difference regions in the test interface image, wherein the annotations are used to indicate the difference regions; The image obtained by annotating the difference regions in the test interface image and the image obtained by annotating the regions in the reference interface image that correspond to the difference regions.

7. The interface testing method according to claim 5, characterized in that, The step of inputting the difference region image and context information into the large language model to obtain the interface test results includes: Using the large language model, dynamic elements in the difference region image are removed based on preset dynamic element identifiers; The interface test results are obtained by processing the difference region image after removing dynamic elements using the large language model.

8. The interface testing method according to claim 5, characterized in that, The method further includes: The confidence level of the interface test results is output using the large language model. For interface test results with a confidence level exceeding a preset value, the difference region images are labeled based on the interface test results, and the labeled difference region images and corresponding context information are stored. The parameters in the large language model are periodically adjusted based on newly annotated difference region images and corresponding contextual information.

9. The interface testing method according to any one of claims 1-8, characterized in that, The interface test results include difference description text and category labels. The difference description text is used to describe the inconsistencies in the interface of the business scenario using natural language, and the category labels are used to indicate the category to which the inconsistency problem belongs.

10. An interface testing device, characterized in that, The device includes: The storage unit is configured to store the scenario identifier and data protocol of the business scenario in a sandbox environment of the business server. The data protocol includes the business data and business rules of the business scenario. The business server is used to provide backend support for the business scenario. The sandbox environment is isolated from the production environment. The sending unit is configured to send test instructions to multiple types of terminals respectively. Each test instruction contains the scenario identifier. Different types of terminals have different operating environments. Each test instruction is used to instruct the terminal to access the business server using a weak login state identifier and return a test interface image. The test interface image is obtained by rendering and screenshotting the business scenario based on the data protocol of the business scenario in the sandbox environment. The discrimination unit is configured to execute a reference interface image based on the business scenario, discriminate test interface images of multiple terminals, and obtain interface test results. The interface test results are used to indicate the degree of consistency of the business scenario on different types of terminals.

11. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory used to store the executable program code of the processor; The processor is configured to execute the program code to implement the interface testing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the interface testing method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the interface testing method according to any one of claims 1 to 9.

Images