Multi-scenario front-end inspection methods, program products, electronic devices and storage media

Abstract

This application provides a multi-scenario front-end inspection method, program product, electronic device, and storage medium. The method includes: acquiring inspection scenarios of front-end pages from multiple data sources; responding to a task creation instruction, determining at least one target inspection scenario from the multiple inspection scenarios and obtaining the detection rules to be executed for the target inspection scenario, generating an inspection task; according to the scenario end type of the target inspection scenario, using the inspector corresponding to the scenario end type to perform page inspection operations and collect page operation data; inputting the page operation data into a multimodal analysis model, whereby the multimodal analysis model, in conjunction with the detection rules, detects the page operation data, identifies, and outputs inspection anomaly information. Automatically acquiring inspection scenarios and identifying end types from multiple data sources eliminates the need to write complex automation scripts for each page, improving the efficiency of detection in different scenarios. The introduction of a multimodal analysis model enhances the comprehensiveness and accuracy of problem discovery.

Description

Technical Field

[0001] This application relates to the field of front-end testing technology, and more specifically, to a multi-scenario front-end inspection method, program product, electronic device, and storage medium. Background Technology

[0002] In current internet application development, front-end page quality assurance primarily relies on automated testing. A common practice is for testers to use automated testing frameworks and write automated scripts for core functionalities. These scripts are executed before code deployment to verify the functionality of core features, or run periodically in the production environment to monitor for issues on core pages. However, this traditional method of front-end automated testing requires significant manpower for writing and maintaining these scripts. Frequent front-end page iterations or changes in the environment necessitate constant script modifications for continued use, resulting in low efficiency. Summary of the Invention

[0003] The purpose of this application is to provide a multi-scenario front-end inspection method, program product, electronic device and storage medium to improve the above-mentioned problems.

[0004] In a first aspect, embodiments of this application provide a multi-scenario front-end inspection method, comprising: obtaining inspection scenarios of front-end pages from multiple data sources, wherein the inspection scenarios include access entry information and scenario type; responding to a task creation instruction, determining at least one target inspection scenario from the multiple inspection scenarios, obtaining the detection rules to be executed for the target inspection scenario, and generating an inspection task; performing page inspection operations using an inspector corresponding to the scenario type according to the scenario type of the target inspection scenario, and collecting page operation data; inputting the page operation data into a multimodal analysis model, wherein the multimodal analysis model, in conjunction with the detection rules, detects the page operation data, identifies and outputs inspection anomaly information.

[0005] In the above implementation process, inspection scenarios are automatically obtained from multiple data sources and the terminal type is identified, eliminating the need for manual maintenance of a list of pages to be inspected, reducing the workload of manual collection, and improving the comprehensiveness of the inspection scope. Users only need to select the scenario and configure the detection rules when creating a task, without having to write complex automated scripts for each page, lowering the implementation threshold and maintenance cost of front-end inspection, and greatly improving the efficiency of detection in different scenarios. The corresponding inspector is called for different terminal types, ensuring compatibility with the technical differences between H5, mini-programs, and native client pages, achieving unified inspection execution and reducing the need for redundant tool development for different platforms. The introduction of a multimodal analysis model combines traditional rule-based detection with intelligent detection based on a large model, not only detecting common issues such as interface errors and blank screens, but also identifying text violations and non-compliant images, thereby improving the comprehensiveness and accuracy of problem detection.

[0006] Optionally, in this embodiment, obtaining the inspection scenario of the front-end page from multiple data sources includes: obtaining access entry information of the front-end page from multiple data sources; the data sources include at least two of system menus, database configuration links, external projection links, traffic data, and UI scripts; standardizing the access entry information and identifying the corresponding scenario type based on the standardized access entry information; associating the access entry information with the scenario type to generate an inspection scenario.

[0007] In the above implementation process, access entry information is obtained from multiple data sources, improving the tedious work of manually collecting page data. It also covers various sources such as system menus, database activity pages, external links, high-frequency traffic, and automated scripts, ensuring a more comprehensive inspection scope. Standardized processing reduces URL format inconsistencies. Identifying scene types allows the same inspection platform to distinguish pages from different technology stacks, providing a basis for subsequently calling the corresponding inspector based on scene type. Reduced manual intervention improves the efficiency and accuracy of scene library updates, enabling front-end inspection to cover more pages.

[0008] Optionally, in this embodiment, the detection rules include general detection rules and / or special detection rules; wherein, the general detection rules are applicable to multiple scenario terminal types; the special detection rules are determined according to the scenario terminal type or preset tags of the target inspection scenario; the general detection rules include at least one of white screen detection, interface error detection, code error detection, routing error detection, console error detection, path correctness detection, and login status verification; the special detection rules include at least one of sensitive word detection, non-compliant image detection, benefit point exposure detection, component drop detection, product category consistency detection, and page title duplication detection.

[0009] In the above implementation process, by dividing the detection rules into general detection rules and special detection rules, the inspection needs of different types of pages can be flexibly adapted. General detection rules are applicable to multiple page types, reducing the omission of basic operational status checks; special detection rules are determined according to the scenario page type or tag, enabling inspections to delve into the business level and discover deeper issues. When creating an inspection task, users only need to select the rules, without having to write logic for each page, reducing the configuration difficulty of inspection tasks. The selectivity and combinability of the rules allow inspections to be adjusted for different purposes, improving the targeting and effectiveness of inspections.

[0010] Optionally, in this embodiment of the application, after generating the inspection task, the method further includes: if the number of target inspection scenarios included in the inspection task exceeds a preset threshold, then the inspection task is split into multiple sub-tasks, each sub-task including a set of target inspection scenarios; and multiple sub-tasks are placed into an execution queue for multiple inspector instances to execute in parallel or serially.

[0011] In the above implementation process, when the inspection task involves a large number of target inspection scenarios, splitting it into multiple sub-tasks can prevent a single inspector instance from being overloaded or timed out due to processing too many pages, thus improving the stability and success rate of task execution. Placing sub-tasks in an execution queue and having multiple inspector instances execute them in parallel utilizes system resources and shortens the overall completion time of large-scale inspection tasks. It also supports a serial execution mode, adapting to different resource environments and execution strategy requirements. The clustered deployment of inspector instances and the queue mechanism also bring good scalability; when the number of inspection tasks increases, simply increasing the number of inspector instances can improve processing capacity. Through the splitting and queue scheduling mechanism, the processing efficiency, resource utilization, and scalability of the inspection system are improved, enabling batch inspections of hundreds or thousands of pages to be completed in a short time, meeting the actual needs of large-scale front-end quality assurance.

[0012] Optionally, in this embodiment, page inspection operations are performed using the inspector corresponding to the scene type of the target inspection scenario, including: if the scene type of the target inspection scenario is an H5 page, the H5 inspector is invoked to perform page inspection operations by simulating user operations; if the scene type of the target inspection scenario is a mini-program, the mini-program inspector is invoked, which consists of a native inspection mini-program running on the terminal and a cloud scheduling service, and drives the native inspection mini-program to perform page jumps and simulated operations through cloud instructions; if the scene type of the target inspection scenario is a native client page, the client inspector is invoked, which connects to the terminal device through a system debugging bridge, locates page elements, and then simulates page operations.

[0013] In the above implementation process, inspection coverage can be performed separately for front-end pages of different technology stacks: the H5 inspector simulates user operations and can completely reproduce the user's real behavior in the browser; the mini-program inspector adopts a cloud scheduling and native mini-program collaboration approach, improving the industry's difficulty in automating inspections of closed environments such as Alipay mini-programs; the client-side inspector connects to devices through a system debugging bridge, enabling it to deeply penetrate the native application to perform operations and collect data. There is no need to develop and maintain multiple inspection tools for different client types. During execution, the inspector not only collects log information but also preserves visual evidence of the page through screenshots, providing rich data input for multimodal analysis and improving inspection coverage and execution efficiency.

[0014] Optionally, in this embodiment of the application, collecting page operation data includes: during the page loading process, collecting log information by listening; the log information includes operation error information and network request records; in the event of an anomaly, calling the screenshot interface to capture a screenshot of the page area; associating the collected log information and the screenshot of the page area with the corresponding target inspection scene identifier to generate page operation data.

[0015] In the above implementation process, the monitoring mechanism can capture various errors and request states during page runtime. The screenshot method triggered by exceptions preserves the scene at the time of the problem, reducing the chance of missing key clues. Associating logs and screenshots with the same scene identifier provides well-structured and complete input data for multimodal analysis. This improves the data quality upon which subsequent anomaly identification relies, enabling anomaly analysis to be based on the actual page state and improving the accuracy of anomaly identification.

[0016] Optionally, in this embodiment, the page operation data includes log information and page area screenshots; the page operation data is input into a multimodal analysis model, which combines detection rules to detect the page operation data, including: parsing the detection rules to determine an anomaly type set, which includes log anomaly types and visual anomaly types; the log information in the page operation data is input into a rule engine, which matches the log information according to the anomaly judgment logic corresponding to the log anomaly type to identify the log anomaly type; the page screenshots in the page operation data are input into the multimodal analysis model, which includes an optical character recognition module, a large language model module, and a visual language model module; wherein, the optical character recognition module extracts text information from the page area screenshots; the visual language model module extracts visual features from the page area screenshots; the large language model module combines the extracted text information and visual features, and infers based on the prompts corresponding to the visual anomaly type to identify the visual anomaly type; and inspection anomaly information is generated based on the identified log anomaly type and visual anomaly type.

[0017] In the above implementation process, the parsing detection rules determine the set of anomaly types, allowing subsequent processing to focus only on the anomaly types that users care about. The rule engine quickly and accurately matches common issues from logs, and the multimodal analysis model combines OCR text recognition, VLM visual understanding, and LLM inference to deeply analyze page screenshots and discover complex issues that traditional automation cannot cover. Finally, the data is merged to generate structured inspection anomaly information, reducing the workload of manual inspections and improving inspection efficiency.

[0018] Optionally, in this embodiment of the application, after identifying and outputting the inspection anomaly information, the method further includes: performing correlation analysis on the inspection anomaly information and the historical anomaly database to obtain correlation analysis results; the correlation analysis results include common source anomaly patterns; based on the correlation analysis results, aggregating multiple anomaly information belonging to the same cause to generate an anomaly root cause report, the anomaly root cause report including at least one of anomaly type, occurrence frequency, impact range, and alarm level.

[0019] In the above implementation process, correlation analysis can identify multiple anomaly records with the same root cause, avoiding repeated alarms and handling of the same problem, and reducing the workload of maintenance personnel in dealing with noise. The anomaly root cause report aggregates scattered anomaly records into a structured problem description, including anomaly type, occurrence frequency, impact scope, and alarm level, shortening the time for problem localization and remediation.

[0020] Optionally, in this embodiment of the application, after identifying and outputting inspection anomaly information, the method further includes: performing statistical analysis on the inspection anomaly information identified in historical inspections to generate a quality trend chart; aligning the quality trend chart with the business release event in time to generate event anomaly information; the event anomaly information includes code release anomalies and / or configuration change anomalies; generating optimization suggestions based on the event anomaly information, the optimization suggestions including adjusting the threshold of the detection rules, adding detection rules, or supplementing samples for the multimodal analysis model; the supplementary samples are used to optimize the multimodal analysis model.

[0021] In the above implementation process, the quality trend chart provides a clear visual representation of changes in front-end quality, reducing the need for manual recall or fragmented records. Aligning the trend chart with business release events allows for quick identification of whether newly deployed code or configuration changes have introduced problems, improving the speed of issue localization. Optimization suggestions generated based on event anomaly information improve operational efficiency, and supplementary samples enhance the accuracy of the multimodal analysis model.

[0022] Secondly, this application also provides a multi-scenario front-end inspection device, comprising: a scenario acquisition module, used to acquire inspection scenarios of front-end pages from multiple data sources, wherein the inspection scenarios include access entry information and scenario terminal type; a task generation module, used to respond to a task creation instruction, determine at least one target inspection scenario from multiple inspection scenarios, obtain the detection rules to be executed for the target inspection scenario, and generate an inspection task; an execution module, used to execute page inspection operations using an inspector corresponding to the scenario terminal type according to the scenario terminal type of the target inspection scenario, and collect page operation data; and an analysis module, used to input the page operation data into a multimodal analysis model, wherein the multimodal analysis model combines the detection rules to detect the page operation data, identify and output inspection anomaly information.

[0023] Thirdly, embodiments of this application also provide a computer program product, including computer program instructions, which are executed by a processor to perform the method provided in the first aspect or any implementation thereof.

[0024] Fourthly, embodiments of this application also provide an electronic device, including: a processor and a memory, the memory storing computer program instructions, which are executed by the processor to perform the method provided in the first aspect or any implementation thereof.

[0025] Fifthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, perform the method provided in the first aspect or any implementation thereof.

[0026] This application provides a multi-scenario front-end inspection method, program product, electronic device, and storage medium. It automatically acquires inspection scenarios and identifies terminal types from multiple data sources, eliminating the need for manual maintenance of a list of pages to be inspected, reducing manual collection workload, and improving the comprehensiveness of the inspection scope. Users only need to select scenarios and configure detection rules when creating tasks, without needing to write complex automation scripts for each page, lowering the implementation threshold and maintenance costs of front-end inspection, and greatly improving the efficiency of detection in different scenarios. It calls corresponding inspectors for different terminal types, ensuring compatibility with the technical differences between H5, mini-programs, and native client pages, achieving unified inspection execution and reducing redundant tool development for different platforms. By introducing a multimodal analysis model, it combines traditional rule-based detection with intelligent detection based on a large model, not only detecting common issues such as interface errors and blank screens, but also identifying text violations and non-compliant images, thereby improving the comprehensiveness and accuracy of problem detection. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a multi-scenario front-end inspection method provided in an embodiment of this application; Figure 2 This is a schematic flowchart of an element positioning method provided in an embodiment of this application; Figure 3 This application provides a schematic diagram of a task generation process. Figure 4 A multi-scenario schematic diagram provided for an embodiment of this application; Figure 5 This is a schematic diagram of the structure of the multi-scenario front-end inspection device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application.

[0031] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0032] Please see Figure 1 The illustrated diagram shows a flowchart of a multi-scenario front-end inspection method provided in an embodiment of this application. The multi-scenario front-end inspection method provided in this application can be applied to electronic devices, which may include physical devices such as servers, PCs, tablets, or smartphones, or virtual devices such as virtual machines or containers. The electronic device can be a single device, a combination of multiple devices, or a cluster of a large number of devices. The multi-scenario front-end inspection method may include: Step S110: Obtain the inspection scenario of the front-end page from multiple data sources. The inspection scenario includes access entry information and scenario type.

[0033] Step S120: In response to the task creation instruction, determine at least one target inspection scenario from multiple inspection scenarios, obtain the detection rules to be executed for the target inspection scenario, and generate an inspection task.

[0034] Step S130: Based on the scene type of the target inspection scene, use the inspector corresponding to the scene type to perform page inspection operations and collect page operation data.

[0035] Step S140: Input the page running data into the multimodal analysis model. The multimodal analysis model combines the detection rules to detect the page running data, identify and output inspection anomaly information.

[0036] In step S110: An inspection scenario refers to a front-end page unit that needs to be inspected. It can be an independent page or a page flow containing a series of user operations. An inspection scenario includes access entry information and scenario type. Access entry information is usually a Uniform Resource Locator (URL) used to locate the specific network address of the page; for scenarios defined by automated scripts, access entry information can also be the script's unique identifier, with the script itself describing the initial page and subsequent operation steps. Scenario type identifies the terminal environment in which the page runs, including H5 pages, WeChat mini-programs, Alipay mini-programs, native client pages, etc. Different terminal types of pages differ in their technical implementation and inspection methods.

[0037] As one implementation method, the data source may include: a menu table in the internal permission system, which records the access paths of each page in the PC backend; an activity configuration table stored in the business database, which contains complete links to various marketing pages; external advertising link files uploaded by operations personnel, which record the page addresses to which advertising channels are placed; a list of pages with the highest number of visits counted in the event tracking data warehouse; and an automated script library written by testers.

[0038] In step S120: The task creation command can be initiated by the user through the front-end interface of the inspection platform. The user can select one or more existing inspection scenarios as the target of this inspection on the interface, or can filter them in batches according to the scenario's terminal type, tag, or source.

[0039] After selecting the target inspection scenario, the user needs to configure the detection rules to be executed for this inspection. Detection rules can include general detection rules and / or special detection rules; general detection rules are applicable to pages of all terminal types, such as blank screen detection, interface error detection, JavaScript error detection, routing error detection, console error detection, URL correctness detection, and login status verification; special detection rules are automatically recommended or manually selected by the user based on the terminal type or preset tags of the target inspection scenario. For example, for e-commerce promotion pages, sensitive word detection, non-compliant image detection, benefit disclosure detection, component drop-down detection, product category consistency detection, and page title duplication detection can be enabled.

[0040] The system associates the user-selected list of target inspection scenarios with the configured detection rules to generate an inspection task. The inspection task includes a unique task identifier, an execution time plan (immediate execution or scheduled execution), a list of associated inspection scenarios, and detection rules.

[0041] In step S130: The inspector is the program module that actually executes page opening and operations. Corresponding inspectors can be set for different scenario terminal types to perform detection operations. When an inspection task is scheduled for execution, each target inspection scenario in the task is traversed, and the corresponding inspector is called according to the terminal type of that scenario. For example, for a scenario with an H5 page terminal type, the H5 inspector is called; for a scenario with a WeChat mini-program or Alipay mini-program terminal type, the mini-program inspector is called; and for a scenario with a native client-side page terminal type, the client-side inspector is called.

[0042] During the data collection process, the H5 inspector uses a browser event listening mechanism to capture real-time log information output from the console, JavaScript runtime errors, and success and failure records of network requests. The mini-program inspector employs a combination of cloud scheduling and native mini-program functionality. The cloud scheduling service sends instructions to the connected device pool (such as mobile emulators or real devices), designating an idle device to perform the inspection. A native "Inspection Assistant" mini-program is pre-installed on the device. Upon receiving the instruction, this mini-program uses its API to navigate to the target scenario's mini-program page and leverages the mini-program framework's listening capabilities to collect page lifecycle events, error messages, and network requests. The collected log information, error records, network request records, and screenshots are packaged, associated with a unique identifier for that scenario, and uploaded to the data storage service as the page runtime data for that scenario.

[0043] In step S140: The multimodal analysis model is an analysis module that integrates multiple intelligent detection capabilities, receiving page execution data and detection rules as input. First, the model parses the detection rules to determine the set of anomaly types that need to be identified in this detection. Each detection rule can correspond to one or more anomaly types; for example, for a detection rule targeting logs, the anomaly types may include interface errors, JS errors, etc., while for a detection rule targeting screenshots, the anomaly types may include sensitive words or non-compliant images, etc.

[0044] For example, for log information, the multimodal analysis model sends the log information to the rule engine. The rule engine has pre-defined judgment logic for each common exception type, such as identifying interface errors by matching error codes and identifying JS errors by stack trace features. The rule engine matches the log information one by one according to the enabled items in the detection rules and outputs the log exception type.

[0045] For page screenshots, the multimodal analysis model calls its Optical Character Recognition (OCR) module to extract all visible text content and its location information from the screenshot. Simultaneously, the Visual Language Model (VLM) module extracts visual features from the screenshot to understand objects, scenes, and layouts within the image. Then, the Large Language Model (LLM) module receives the text information extracted by OCR and the visual features extracted by VLM, and combines this with detection prompts corresponding to the intelligent anomaly types enabled in the detection rules (e.g., "Check if the image contains sensitive content" or "Determine if the benefit copy matches the product") to perform multimodal inference. The output of the LLM is the visual anomaly type.

[0046] Finally, the model merges the log anomaly types and visual anomaly types output by the rule engine, removes duplicates, and labels each anomaly record with its corresponding detection rule and confidence level, generating inspection anomaly information. This anomaly information is written to the anomaly database and displayed to users through the platform interface, or pushed to relevant personnel according to preset alarm rules.

[0047] In the implementation of the above embodiments: inspection scenarios are automatically obtained from multiple data sources and terminal types are identified, eliminating the need for manual maintenance of a list of pages to be inspected, reducing the workload of manual collection, and improving the comprehensiveness of the inspection scope. Users only need to select the scenario and configure the detection rules when creating a task, without having to write complex automated scripts for each page, lowering the implementation threshold and maintenance cost of front-end inspection, and improving the efficiency of detection in different scenarios. The corresponding inspector is called for different terminal types, ensuring compatibility with the technical differences between H5, mini-programs, and native client pages, achieving unified inspection execution and reducing the need for redundant tool development for different platforms. The introduction of a multimodal analysis model combines traditional rule-based detection with intelligent detection based on a large model, not only detecting common issues such as interface errors and blank screens, but also identifying text violations and non-compliant images, thereby improving the comprehensiveness and accuracy of problem detection.

[0048] Optionally, in this embodiment of the application, the inspection scenario of the front-end page is obtained from multiple data sources, including: Retrieve entry point information for front-end pages from multiple data sources. Entry point information refers to the starting address that can locate a front-end page or the operation flow of a page. It is usually expressed in the form of a Uniform Resource Locator (URL). For inspection scenarios defined by automated scripts, entry point information can also be the unique identifier of the script or the storage path of the script file, because the script itself describes the initial page address and subsequent operation steps.

[0049] Data sources include at least two of the following: system menus, database configuration links, external projection links, traffic data, and UI scripts. Multiple pre-written data collection modules can be connected to different data sources to obtain access entry information periodically or in real-time.

[0050] For example, system menu data sources can come from internal permission system interfaces or database tables, which record the page URLs corresponding to each function menu in the PC backend. Database configuration link data sources refer to activity configuration tables and page setup records stored in the business database, containing complete links to various activity pages. External campaign link data sources are campaign files uploaded by operations personnel or a collection of landing page links obtained from the advertising system API. Traffic data sources are a list of page URLs with the highest user visits over a past period, extracted from the event tracking data warehouse. UI script data sources refer to automated script libraries written by testers; each script file describes the starting page and subsequent operation steps, and its filename or ID can serve as access entry information. Raw information can be retrieved from these data sources through interface calls, database queries, file parsing, etc., and the results are aggregated in a temporary storage area for further processing.

[0051] The access entry information is standardized, and the corresponding scenario type is identified based on the standardized access entry information.

[0052] Standardization processes are used to improve the uniformity of access entry information formats. For example, missing protocol headers can be added to URLs, redundant spaces and invisible characters can be removed, and obviously invalid test links or duplicates can be eliminated. For URLs containing dynamic parameters, parameter templates can be retained or tracking parameters can be removed for normalization. After cleaning, the system identifies the scenario type corresponding to each access entry information according to preset rules.

[0053] The context type refers to the environment in which the page runs, including H5 pages, WeChat mini-programs, Alipay mini-programs, and native client pages. Identification rules can be based on the characteristics of the access entry information: if the URL's domain name contains common mobile identifiers, or the path conforms to the H5 page pattern, it is determined to be an H5 page; if the URL's protocol header is or contains path parameters specific to mini-programs, it is determined to be a WeChat mini-program; if the URL uses a custom protocol, it is determined to be a native client page. URLs that cannot be clearly identified can be marked as H5 pages by default and await manual confirmation. The output of this step is to append a context type tag to each access entry piece of information.

[0054] The access entry information is associated with the scene terminal type to generate an inspection scene. The standardized access entry information, marked with the terminal type, is stored as an independent inspection scene record in the scene management database. Each inspection scene record contains at least the following fields: access entry information (URL or script ID) and scene terminal type. Of course, the inspection scene may also include a unique scene identifier, data source, creation time, status, etc.

[0055] As one implementation method, tag fields can be added according to business needs. For example, tags such as "major promotion" and "core" can be automatically added based on keywords in the URL to facilitate subsequent filtering. The generation of inspection scenarios is the foundation for the construction of subsequent inspection tasks. Scenarios in the scenario library can be reused by multiple inspection tasks without the need for re-collection each time. The scenario library can also be updated incrementally on a regular basis, for example, by re-fetching traffic data and generating new scenarios every day, while retaining historical scenarios for traceability.

[0056] In the implementation of the above embodiments: access entry information is obtained from multiple data sources, improving the tedious work of manually collecting pages. It also covers various sources such as system menus, database activity pages, external links, high-frequency traffic, and automated scripts, ensuring a more comprehensive inspection scope. Standardized processing reduces URL format inconsistencies. Identifying scene types allows the same inspection platform to distinguish pages from different technology stacks, providing a basis for subsequently calling the corresponding inspector based on scene type. Reduced manual intervention improves the efficiency and accuracy of scene library updates, enabling front-end inspection to cover more pages.

[0057] Optionally, in this embodiment, the detection rules include general detection rules and / or special detection rules; wherein, general detection rules are applicable to multiple scenario terminal types; special detection rules are determined according to the scenario terminal type or preset tags of the target inspection scenario. Detection rules refer to the specific inspection items that need to be performed when inspecting a page. General detection rules are basic inspection items, unrelated to the specific business content of the page, and applicable to all scenario terminal types, such as H5 pages, mini-programs, and native client pages, all of which can perform white screen detection. Special detection rules are related to the business attributes or terminal type of the page and are usually enabled for specific scenarios.

[0058] When a user creates an inspection task, the system provides a rule configuration interface, where the user can select the general detection rules to be executed for this task. For dedicated detection rules, the system will automatically recommend a batch of available dedicated detection rules based on the scenario type or preset tags (such as "big promotion" or "core link") of the target inspection scenario selected by the user. Users can also manually select or deselect these rules.

[0059] The general detection rules include at least one of the following: white screen detection, interface error detection, code error detection, routing error detection, console error detection, path correctness detection, and login status verification.

[0060] General detection rules focus on the basic operational status of the page. For example, blank screen detection analyzes screenshots after the page has finished loading to determine if the main content area is blank or nearly blank. Interface error detection listens to network requests and identifies requests returning 4xx or 5xx status codes, or requests that timed out or failed. Code error detection captures exceptions thrown by JavaScript runtime. Routing error detection monitors for 404 pages or route switching failures during page redirection. Console error detection collects all warnings and error messages. Path correctness detection verifies if the current page's URL matches expectations, preventing redirection errors. Login status verification checks if the page is redirected to the login page or if the interface returns an unauthorized status to determine if the user's login status has expired.

[0061] The dedicated detection rules should include at least one of the following: sensitive word detection, non-compliant image detection, profit-sharing detection, component drop-down detection, product category consistency detection, and page title duplication detection.

[0062] Specialized detection rules focus on the business compliance and content quality of the page. For example, sensitive word detection uses OCR technology to extract text from page screenshots and then matches it with a sensitive word database to identify prohibited words. Non-compliant image detection analyzes the image content in screenshots using a visual language model to determine if any prohibited images exist. Benefit highlighting detection checks whether the information advertised on the page is consistent with the actual products or activities displayed, and whether there are any instances where no corresponding product is mentioned. Component drop detection checks whether any modules on a page composed of multiple independent modules are not rendered due to missing or abnormal data. Product category consistency detection verifies whether the products displayed on the page belong to the correct category. Page title duplication detection identifies whether the titles of multiple modules on the same page are duplicated, avoiding information redundancy. These specialized detection rules typically require the integration of technologies such as OCR, large language models, and visual language models, and the detection results are output as inspection anomaly information.

[0063] In the implementation of the above embodiments: by dividing the detection rules into general detection rules and special detection rules, the inspection needs of different types of pages can be flexibly adapted. General detection rules are applicable to multiple page types, reducing the omission of basic operational status checks; special detection rules are determined according to the scenario page type or tag, enabling inspections to delve into the business level and discover deeper problems. When creating an inspection task, users only need to select the rules, without having to write logic for each page, reducing the configuration difficulty of inspection tasks. The selectivity and combinability of the rules allow inspections to be adjusted for different purposes, improving the targeting and effectiveness of inspections.

[0064] Optionally, in this embodiment of the application, after generating the inspection task, the method further includes: If the number of target inspection scenarios included in an inspection task exceeds a preset threshold, the inspection task will be split into multiple sub-tasks, each of which contains a set of target inspection scenarios.

[0065] The preset threshold is a pre-defined value used to determine whether an inspection task needs to be broken down into smaller execution units. This threshold can be set based on system performance, the processing capacity of the inspector, and actual business needs; for example, it can be set to 50 or 100 scenarios. When the number of target inspection scenarios in a user-created inspection task exceeds this threshold, the task splitting logic is applied. The goal of splitting is to decompose a large-scale task into multiple smaller subtasks. Each subtask contains a set of target inspection scenarios, which are usually consecutively assigned, for example, divided into subtasks of every 50 scenarios according to scenario ID. Each subtask inherits the original inspection task's detection rules, execution plan, and other attributes, and is assigned an independent subtask identifier, recording its contained scenario list. The subtask's status is initialized to "pending execution." In this way, a task that originally required processing a large number of pages at once is transformed into multiple small tasks that can be independently scheduled and executed.

[0066] Multiple subtasks are placed in the execution queue for multiple inspector instances to execute in parallel or serially.

[0067] The pending queue is a data structure used to store and manage subtasks to be processed, typically implemented based on a message queue or database task table. All subtasks generated from the splitting process are placed into this queue in order of creation time or priority. An inspector instance is the program process that actually performs page inspection operations. Multiple inspector instances can run simultaneously; these instances can be deployed on the same server or distributed across multiple servers to form an inspector cluster. Each inspector instance continuously polls the pending queue, actively retrieving one or more subtasks in the "pending execution" state. After successful retrieval, the inspector instance updates the subtask status to "in execution" and, according to the target inspection scenario list contained in the subtask, sequentially executes the inspection operation for each scenario as described in step S130. If configured for parallel execution, multiple inspector instances can simultaneously retrieve different subtasks from the queue, achieving concurrent processing of multiple subtasks; if configured for serial execution, the inspector instance processes only one subtask at a time, retrieving the next one only after completion. After a subtask is completed, the inspector instance will report the execution result and update the subtask status to "completed" or "failed". Once all subtasks are completed, the original inspection task is considered to have been completed.

[0068] In the implementation of the above embodiments: when an inspection task contains a large number of target inspection scenarios, splitting it into multiple sub-tasks can prevent a single inspector instance from being overloaded or timed out due to processing too many pages, thus improving the stability and success rate of task execution. Placing sub-tasks in an execution queue and having multiple inspector instances execute them in parallel utilizes system resources and shortens the overall completion time of large-scale inspection tasks. It also supports a serial execution mode, adapting to different resource environments and execution strategy requirements. The clustered deployment of inspector instances and the queue mechanism also bring good scalability; when the number of inspection tasks increases, simply increasing the number of inspector instances can improve processing capacity. Through the splitting and queue scheduling mechanism, the processing efficiency, resource utilization, and scalability of the inspection system are improved, enabling batch inspections of hundreds or thousands of pages to be completed in a short time, meeting the actual needs of large-scale front-end quality assurance.

[0069] Optionally, in this embodiment of the application, the page inspection operation is performed using the inspector corresponding to the scene type of the target inspection scene, including: When the target inspection scenario is an H5 page, the H5 inspector is invoked to perform page inspection operations by simulating user actions.

[0070] The H5 Inspector is a program module specifically designed for inspecting H5 pages. It leverages the Puppeteer framework to automate browser control. Once the target inspection scenario's endpoint is an H5 page, the H5 Inspector is invoked to execute the inspection tasks for that scenario. The H5 Inspector first starts a browser instance and creates a new browser context for that instance to ensure isolation between different scenarios.

[0071] The H5 inspector opens the access entry point (i.e., URL) of the target inspection scenario and begins listening for page lifecycle events. During page loading, the H5 inspector captures console logs, JavaScript runtime errors, and network request success and failure records through an event listening mechanism. Throughout the inspection process, the H5 inspector calls the screenshot API to capture screenshots of the visible area of ​​the current page after page loading, after each key operation step, and when an anomaly is detected. Finally, the H5 inspector packages the collected log information, error records, network request records, and screenshot files, associates them with a unique identifier for the scenario, forms page runtime data, and uploads it to a data storage service.

[0072] When the target inspection scenario is a mini-program, the mini-program inspector is invoked. The mini-program inspector consists of a native inspection mini-program running on the terminal and a cloud scheduling service. Cloud commands drive the native inspection mini-program to perform page jumps and simulated operations.

[0073] The mini-program inspector adopts an architecture that coordinates cloud scheduling with the native mini-program on the terminal. The inspector consists of two parts: a native inspection mini-program running on a mobile terminal or emulator, and a scheduling service deployed in the cloud. The cloud scheduling service manages the terminal device pool, receives inspection tasks, and issues instructions. When an inspection of a specific mini-program scenario needs to be performed, the cloud scheduling service selects an idle terminal device from the device pool and sends instructions to the native inspection mini-program on that device through a pre-established long-term connection channel. After receiving the instructions, the native inspection mini-program uses the jump API provided by the mini-program framework to open the mini-program page of the target scenario, for example, by jumping through the mini-program's URL Scheme or page path.

[0074] During page loading and operation, the native inspection mini-program collects page lifecycle events, JavaScript error messages, network request status, etc., through the event listening mechanism of the mini-program framework, and sends this data back to the cloud scheduling service. If user operations need to be simulated, the cloud scheduling service will issue specific operation instructions. After receiving the instructions, the native inspection mini-program uses the selector API provided by the mini-program framework to locate page elements and simulate user operations by triggering corresponding events. At preset nodes (such as page loading completion or operation completion), the native inspection mini-program calls the mini-program's screenshot API to capture a screenshot of the current page, uploads the screenshot file to object storage, and reports the screenshot path along with other log data. The cloud scheduling service summarizes and organizes all collected data to form the page operation data for this scenario.

[0075] When the target inspection scenario is a native client page, the client inspector is invoked. The client inspector connects to the terminal device through the system debugging bridge, locates the page elements, and then simulates page operations.

[0076] The client-side inspector is a program module used to inspect native application pages installed on mobile terminals. When the system determines that the target inspection scenario's endpoint type is a native client page, it will invoke the client-side inspector to perform the inspection. The client-side inspector first needs to establish a connection with the terminal device through the system debugging bridge. Before executing the inspection of a specific scenario, the client-side inspector will send a command through the system debugging bridge to launch the target application and redirect to the specified page, based on the scenario's access entry information. After the application launches, the client-side inspector uses UI automation frameworks to identify elements of the current interface. These frameworks allow the location of interactive elements such as buttons and input fields using resource IDs, text content, XPath, etc.

[0077] Based on the automated scripts or preset operations that may be included in the inspection task, the client-side inspector locates the target element and simulates user operations such as clicks, long presses, swipes, and input through the APIs provided by the framework. Throughout the inspection process, the client-side inspector captures device logs in real time through the system debugging bridge, filtering out application crash information, network request logs, etc. The client-side inspector captures images of the current screen through system commands or the framework's built-in screenshot function. All collected logs and screenshots are uploaded to the server and associated with scene identifiers to form page runtime data.

[0078] In the implementation of the above embodiments, inspection coverage can be performed separately for front-end pages of different technology stacks: the H5 inspector simulates user operations and can completely reproduce the user's real behavior in the browser; the mini-program inspector adopts a cloud scheduling and native mini-program collaboration approach, improving the industry's difficulty in automating inspections of closed environments such as Alipay mini-programs; the client-side inspector connects to the device through a system debugging bridge, enabling it to deeply penetrate the native application to perform operations and collect data. There is no need to develop and maintain multiple inspection tools for different client types. During execution, the inspector not only collects log information but also preserves visual evidence of the page through screenshots, providing rich data input for multimodal analysis and improving inspection coverage and execution efficiency.

[0079] Optionally, in this embodiment of the application, collecting page operation data includes: During page loading, log information is collected by listening; the log information includes runtime error messages and network request records.

[0080] Listening refers to the process by which the inspector registers event handlers during page execution to capture various information generated within the page in real time. In the H5 inspector, the `page.on` method of `Puppeteer` is used to listen for console output, error events, and request failure events. In the mini-program inspector, error and lifecycle information is collected through the APIs provided by the mini-program framework. In the client-side inspector, relevant output from device logs is filtered through the system debug bridge. Runtime error information includes uncaught exceptions during JavaScript execution, failed API calls, and resource loading errors. Each error record typically includes the error type, error stack trace, and occurrence time. Network request records include the URL, request method, status code, request duration, and response content for each request initiated by the page. The inspector writes this information to a memory buffer in real time and formats the output uniformly after page operations are completed.

[0081] Upon detecting an anomalies, a screenshot of a specific area of ​​the page is captured via an API call. Anomalies include runtime errors, network request failures, page load timeouts, and critical nodes requiring status recording according to predefined rules. A page area screenshot is a visual representation of the current page content saved as an image, typically in PNG format. Screenshots are not only used for detecting visual issues such as blank screens but also provide raw material for text recognition and image analysis. Each screenshot automatically records a filename containing a scene identifier and timestamp upon saving, facilitating subsequent association.

[0082] The collected log information and page area screenshots are associated with the corresponding target inspection scene identifiers to generate page operation data.

[0083] Association refers to establishing a correspondence between the target inspection scenario and all corresponding log information and screenshot files in the same round of inspection. After executing all operations for a target inspection scenario, the inspector serializes the log information in memory into a pre-formatted log file and organizes all the screenshot files collected in this operation into the same folder. The inspector adds the target inspection scenario's scene identifier, inspection task identifier, and execution timestamp to this data set, forming a complete data package. Page execution data refers to the raw inspection results set that has been initially processed and is available for subsequent analysis. It includes both structured text logs and unstructured image files.

[0084] In the implementation of the above embodiments: the monitoring mechanism can capture various errors and request states during page runtime, and the screenshot method triggered by an anomaly preserves the scene when the problem occurs, reducing the omission of key problem clues. Associating logs and screenshots with the same scene identifier provides well-structured and complete input data for multimodal analysis. This improves the data quality upon which subsequent anomaly identification relies, enabling anomaly analysis to be based on the actual page scene and improving the accuracy of anomaly identification.

[0085] Optionally, in this embodiment, the page execution data includes log information and page area screenshots; the page execution data is input into a multimodal analysis model, and the multimodal analysis model detects the page execution data in conjunction with detection rules, including: The detection rules are analyzed to determine the set of anomaly types, which includes log anomaly types and visual anomaly types.

[0086] Log-related anomaly types refer to anomalies that can be discovered by analyzing text logs, such as interface errors, JavaScript errors, routing errors, console errors, and login status failures. Visual anomaly types refer to anomalies that require analysis of page screenshots for identification, such as blank screens, sensitive words, non-compliant images, anomalies revealing profit points, and components falling off the screen.

[0087] Log information from the page's runtime data is input into the rule engine. The rule engine matches the log information according to the exception judgment logic corresponding to the log exception type and identifies the log exception type.

[0088] The rule engine is a module that performs pattern matching based on predefined logic. It receives log information from page execution data as input, including console output, network request records, error stack traces, and other text content. Internally, the rule engine has pre-defined judgment logic for each log exception type. For example, for JavaScript errors, the judgment logic matches whether the error stack trace contains specific exception type keywords. The rule engine iterates through each log message, matching it one by one against the enabled log exception type judgment logic. When a log message meets the judgment conditions for a certain exception type, the rule engine records the exception, including the exception type, occurrence time, key information (such as error stack trace or request URL), and associated scenario identifier.

[0089] The page screenshots from the page operation data are input into the multimodal analysis model, which includes an optical character recognition module, a large language model module, and a visual language model module. The optical character recognition module extracts text information from the page area screenshot; the visual language model module extracts visual features from the page area screenshot; and the large language model module combines the extracted text information and visual features, and infers based on the clues corresponding to the visual anomaly type to identify the visual anomaly type.

[0090] Upon receiving a screenshot, the Optical Character Recognition (OCR) module first processes the screenshot, identifying all the text characters it contains and outputting the text content and its position coordinates within the image. Simultaneously, the Visual Language Model (VLM) module performs deep visual analysis on the screenshot, extracting semantic features such as scene category, object recognition, and layout structure. These two outputs are then fed into the Large Language Model (LLM) module. The LLM module also receives a prompt, dynamically generated based on the visual anomaly type determined in step S710, such as "Please determine if the screenshot contains sensitive words" or "Check if the benefit description matches the product." The LLM combines the text extracted by OCR, the visual features extracted by VLM, and the prompt to perform comprehensive reasoning and determine whether the screenshot matches the characteristics of a certain visual anomaly type. For each enabled visual anomaly type, the LLM outputs a judgment result; if an anomaly is determined, the anomaly type and relevant evidence, such as the original text of sensitive words and their location information, are recorded.

[0091] Based on the identified log anomaly types and visual anomaly types, inspection anomaly information is generated.

[0092] Log anomaly types and visual anomaly types are merged. During the merging process, duplicate anomaly records are removed. For example, if the same type of anomaly in the same inspection is detected by both the rule engine and the multimodal model, it needs to be merged into one record. Information can also be added to each anomaly record, including the specific time the anomaly occurred, the associated scene identifier, the anomaly level, and the path to the evidence file. All anomaly records are then formatted into a unified inspection anomaly information structure and written to the anomaly database. This inspection anomaly information will be displayed on the platform's anomaly management page and can trigger alarms to notify relevant personnel for handling based on preset rules.

[0093] In the implementation of the above embodiments: the detection rules are parsed to determine the set of anomaly types, so that subsequent processing only focuses on the anomaly types that users care about. The rule engine quickly and accurately matches common problems from logs, and the multimodal analysis model combines OCR text recognition, VLM visual understanding, and LLM inference to deeply analyze page screenshots and discover complex problems that are difficult to cover by traditional automation. Finally, the structured inspection anomaly information is merged to generate, reducing the workload of manual inspection and improving inspection efficiency.

[0094] Optionally, in this embodiment of the application, after identifying and outputting the inspection anomaly information, the method further includes: The abnormal inspection information is correlated with the historical abnormal database to obtain the correlation analysis results; the correlation analysis results include common source abnormal patterns.

[0095] The historical anomaly database is a long-term storage collection of all historical inspection anomaly records. Correlation analysis involves comparing newly added inspection anomaly information with records in the historical database to find anomalies with similar characteristics. A common-origin anomaly pattern refers to a set of anomalies with the same underlying cause, whose characteristics may include identical error stacks, identical interface URLs, identical visual features of anomaly screenshots, or identical scene URL patterns.

[0096] For each newly added anomaly, fingerprint information can be extracted. For example, hashing JavaScript error stack traces, extracting URL and status code combinations for API errors, and extracting feature vectors from screenshots for visual anomalies. Then, the historical database is searched for anomaly records with the same fingerprint or fingerprints with similarity exceeding a threshold; these records are marked as belonging to the same source anomaly pattern. The correlation analysis results include information such as the identifiers of these source anomaly patterns, the number of anomaly records included, the time of first discovery, and the time of most recent discovery.

[0097] Based on the correlation analysis results, multiple abnormal information belonging to the same cause are aggregated to generate an abnormal root cause report. The abnormal root cause report includes at least one of the following: abnormal type, occurrence frequency, impact range, and alarm level.

[0098] Aggregation refers to merging multiple anomaly messages belonging to the same common anomaly pattern identified by correlation analysis into a single root cause record. Statistics are performed on all anomaly records under the same pattern: anomaly type is the type of the majority of records or the highest priority type; occurrence frequency refers to the number of times the anomaly under this pattern occurs per unit of time, such as the number of occurrences in the last 24 hours; impact scope refers to the number of different inspection scenarios or different business lines involved by the anomaly; alarm level is calculated comprehensively based on anomaly type, occurrence frequency, and impact scope. For example, a white screen with high frequency is classified as a high level, while occasional console warnings can be classified as a low level. An anomaly root cause report is generated based on this statistical information.

[0099] In the implementation of the above embodiments: Correlation analysis can identify multiple anomaly records with the same root cause, avoiding repeated alarms and handling of the same problem, and reducing the workload of maintenance personnel in dealing with noise. The anomaly root cause report aggregates scattered anomaly records into a structured problem description, including anomaly type, occurrence frequency, impact scope, and alarm level, shortening the time for problem localization and repair.

[0100] Optionally, in this embodiment of the application, after identifying and outputting the inspection anomaly information, the method further includes: Statistical analysis is performed on the inspection anomalies identified in historical inspections to generate quality trend charts.

[0101] Statistical analysis refers to the summarization and calculation of abnormal information according to a preset time period (such as daily or weekly). Quality trend charts are a visual representation of the results of statistical analysis, usually showing the changes of various indicators over time in the form of line charts, bar charts, etc.

[0102] A scheduled task can query anomaly records within a specified time range from the anomaly database, group and count them by anomaly type, scenario type, business line, etc., and store the results in a statistical database. The front-end display module reads data from the statistical database and calls a chart component to render and generate a quality trend chart. The quality trend chart includes curves showing the quantity changes of various anomalies, the distribution ratio of anomalies of different scenario types, or a ranking list of high-frequency anomaly scenarios.

[0103] Align the quality trend chart with the business release event in time to generate event exception information; the event exception information includes code release exceptions and / or configuration change exceptions.

[0104] Business release events refer to the time points recorded when the development team performs operations such as code deployment, configuration changes, and function switch adjustments. These events can be obtained from continuous integration systems, release platforms, or operation and maintenance systems and include information such as release time, release content, and the applications or modules involved.

[0105] Time alignment refers to matching the time points of abnormal fluctuations in the quality trend chart with the time points of business release events. This can be done by identifying inflection points in the quality trend chart—time periods where the number of anomalies significantly increases or decreases. Then, business release events occurring within that time period are queried, and the time difference between the abnormal fluctuations and the release events is calculated. If a business release event occurs before the start of an abnormal fluctuation, and the time difference between the abnormal fluctuation and the release event is within a preset range (e.g., anomalies begin to increase within 30 minutes of release), then the abnormal fluctuation is marked as related to that release event. The generated event anomaly information includes the associated release event identifier, anomaly type, fluctuation amplitude, and duration of impact.

[0106] Based on the event anomaly information, optimization suggestions are generated, including adjusting the threshold of the detection rules, adding detection rules, or supplementing the multimodal analysis model with additional samples; the supplementary samples are used to optimize the multimodal analysis model.

[0107] When event exception information indicates an increase in a certain type of exception due to event publishing, optimization suggestions include adjusting the threshold of the corresponding detection rules. For example, lowering the pixel ratio threshold for white screen detection to reduce false alarms, or increasing the number of retries for interface errors to filter temporary faults. If a new exception type is found that is not covered by existing detection rules, optimization suggestions include adding new detection rules, such as adding component fall detection for newly emerging component types.

[0108] For multimodal analysis models, when event anomaly information indicates that the model is inaccurate in recognizing a certain type of visual anomaly, representative screenshots and manual verification results can be selected from relevant anomaly records and added to the model's training dataset as supplementary samples. These supplementary samples, after format conversion and annotation, are used for incremental training of the model, updating model parameters, and improving the model's ability to recognize similar anomalies.

[0109] In the implementation of the above embodiments: the quality trend chart can intuitively show the changes in front-end quality, reducing manual recall or scattered records. Aligning the trend chart with business release events allows for quick determination of whether newly deployed code or configuration changes have introduced problems, improving the speed of problem localization. Optimization suggestions generated based on event anomaly information improve operational efficiency, and supplementary samples enhance the accuracy of the multimodal analysis model.

[0110] Please see Figure 2 The illustration shows a flowchart of an element positioning method provided in an embodiment of this application.

[0111] After opening the new scene page, confirm that the element positioning tool is installed correctly. Open the inspection page in your browser, then configure page events, defining interactive behaviors such as page loading and clicks. Next, modify the element configuration path and adjust the element's positioning parameters in the DOM. After completing these steps, save the configuration to store the current settings.

[0112] Next, we'll use CSS styles to locate elements and employ CSS selectors to precisely position them. We'll then configure the elements to their corresponding positions on the inspection platform, ensuring the inspection task can correctly identify them. Clicking "Inspect Element" verifies accurate positioning and confirms that the elements have been correctly captured and configured.

[0113] Please see Figure 3 The illustration shows a task generation process provided by an embodiment of this application.

[0114] First, open the inspection platform and enter the system's main interface. Next, add a new client inspection scenario to lay the foundation for subsequent tasks. Then, directly obtain the Tesla platform UI automation script and utilize existing script resources to inspect native routes or H5 pages, ensuring coverage of critical paths in mobile applications. After obtaining the script, create a client inspection task, selecting the client scenario to be inspected and clarifying the inspection target. Next, select the device to be inspected, ensuring the task executes on the specified model, and enter the account and password to simulate a real user login. Then, select the client inspection type, set the inspection mode according to requirements, and configure general task settings such as timing and alarm parameters. Finally, complete the configuration, finishing the entire client inspection task creation process.

[0115] Please see Figure 4 The illustration shown is a multi-scenario diagram provided by an embodiment of this application.

[0116] The H5 page anomaly monitor is a lightweight UI automation tool capable of detecting URL correctness, page request and response failures, page JavaScript errors, page blank screens, and partial blank screens, and supports simple event triggering. It also features personalized inspection functions for the builder, covering specific checks such as empty venue floors, expired venues, venue instability, venue domain name checks, and venue privacy information. The front-end inspection, mini-program anomaly scanner, and client-side inspection explorer collectively focus on core anomalies such as page request and response failures, page errors, and page blank screens. These inspection methods are crucial for improving user experience, ensuring page quality and interaction stability through WYSIWYG, image comparison, and event triggering mechanisms.

[0117] Please see Figure 5 The diagram shown is a structural schematic of the multi-scenario front-end inspection device provided in this application embodiment; this application embodiment provides a multi-scenario front-end inspection device 200, including: The scene acquisition module 210 is used to acquire the inspection scene of the front-end page from multiple data sources. The inspection scene includes access entry information and scene type. The task generation module 220 is used to respond to the task creation instruction, determine at least one target inspection scenario from multiple inspection scenarios, obtain the detection rules to be executed for the target inspection scenario, and generate an inspection task. The execution module 230 is used to perform page inspection operations and collect page operation data by using the inspector corresponding to the scene type of the target inspection scene. The analysis module 240 is used to input page operation data into the multimodal analysis model. The multimodal analysis model combines detection rules to detect the page operation data, identify and output inspection anomaly information.

[0118] Optionally, in this embodiment, the multi-scenario front-end inspection device 200 and the scenario acquisition module 210 are used to obtain access entry information of the front-end page from multiple data sources; the data sources include at least two of system menus, database configuration links, external projection links, traffic data, and UI scripts; the access entry information is standardized, and the corresponding scenario type is identified based on the standardized access entry information; the access entry information is associated with the scenario type to generate an inspection scenario.

[0119] Optionally, in this embodiment, the multi-scenario front-end inspection device 200 includes detection rules for general detection rules and / or special detection rules; wherein, the general detection rules are applicable to multiple scenario end types; the special detection rules are determined according to the scenario end type or preset tags of the target inspection scenario; the general detection rules include at least one of white screen detection, interface error detection, code error detection, routing error detection, console error detection, path correctness detection, and login status verification; the special detection rules include at least one of sensitive word detection, non-compliant image detection, benefit point exposure detection, component drop detection, product category consistency detection, and page title duplication detection.

[0120] Optionally, in this embodiment of the application, the multi-scenario front-end inspection device 200 further includes a task division module, which is used to divide the inspection task into multiple sub-tasks if the number of target inspection scenarios included in the inspection task exceeds a preset threshold, and each sub-task includes a set of target inspection scenarios; and put the multiple sub-tasks into an execution queue for multiple inspector instances to execute in parallel or serially.

[0121] Optionally, in this embodiment, the multi-scenario front-end inspection device 200 and the execution module 230 are used to: 1) call the H5 inspector when the target inspection scenario is an H5 page, and perform page inspection operations by simulating user operations; 2) call the mini-program inspector when the target inspection scenario is a mini-program, which consists of a native inspection mini-program running on the terminal and a cloud scheduling service, and drives the native inspection mini-program to perform page jumps and simulated operations through cloud instructions; 3) call the client inspector when the target inspection scenario is a native client page, and the client inspector connects to the terminal device through a system debugging bridge, locates page elements, and then simulates page operations.

[0122] Optionally, in this embodiment, the multi-scenario front-end inspection device 200 and the execution module 230 are further configured to collect log information by listening during the page loading process; the log information includes running error information and network request records; in the event of an anomaly, the screenshot interface is called to capture a screenshot of the page area; the collected log information and the screenshot of the page area are associated with the corresponding target inspection scenario identifier to generate page running data.

[0123] Optionally, in this embodiment, the multi-scenario front-end inspection device 200 includes page operation data such as log information and page area screenshots; the analysis module 240 is used to parse detection rules, determine anomaly type set, which includes log anomaly type and visual anomaly type; input the log information in the page operation data into the rule engine, which matches the log information according to the anomaly judgment logic corresponding to the log anomaly type to identify the log anomaly type; input the page screenshot in the page operation data into the multimodal analysis model, which includes an optical character recognition module, a large language model module, and a visual language model module; wherein, the optical character recognition module extracts text information from the page area screenshot; the visual language model module extracts visual features from the page area screenshot; the large language model module combines the extracted text information and visual features, and infers according to the prompts corresponding to the visual anomaly type to identify the visual anomaly type; and generate inspection anomaly information based on the identified log anomaly type and visual anomaly type.

[0124] Optionally, in this embodiment of the application, the multi-scenario front-end inspection device 200 further includes: a correlation analysis module, used to perform correlation analysis on the inspection anomaly information and the historical anomaly database to obtain correlation analysis results; the correlation analysis results include common source anomaly patterns; based on the correlation analysis results, multiple anomaly information belonging to the same cause are aggregated to generate an anomaly root cause report, the anomaly root cause report includes at least one of anomaly type, occurrence frequency, impact range and alarm level.

[0125] Optionally, in this embodiment, the multi-scenario front-end inspection device 200 includes an optimization module, which is used to perform statistical analysis on the inspection anomaly information identified in historical inspections to generate a quality trend chart; align the quality trend chart with the business release event in time to generate event anomaly information; the event anomaly information includes code release anomalies and / or configuration change anomalies; and generate optimization suggestions based on the event anomaly information, including adjusting the threshold of the detection rules, adding detection rules, or supplementing samples for the multimodal analysis model; the supplementary samples are used to optimize the multimodal analysis model.

[0126] It should be understood that this device corresponds to the above-described multi-scenario front-end inspection method embodiment and is capable of performing the various steps involved in the above method embodiment. The specific functions of this device can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0127] Please see Figure 6 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. An electronic device 300 provided in this application includes a processor 310 and a memory 320. The memory 320 stores machine-readable instructions executable by the processor 310. When the machine-readable instructions are executed by the processor 310, the method described above is performed.

[0128] Figure 6 The components shown can be implemented using hardware, software, or a combination thereof. Electronic device 300 may be a physical device, such as a server or PC, or a virtual device, such as a virtual machine or virtualization container. Furthermore, electronic device 300 is not limited to a single device; it can be a combination of multiple devices or a cluster of numerous devices.

[0129] This application also provides a storage medium storing a computer program, which is executed by a processor to perform the above-described method.

[0130] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0131] This application also provides a computer program product, including computer program instructions, which are executed by a processor to perform the method described above.

[0132] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, given the several embodiments provided in this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0133] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0134] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. A multi-scenario front-end inspection method, characterized in that, include: The inspection scenarios of the front-end page are obtained from multiple data sources, and the inspection scenarios include access entry information and scenario type. In response to the task creation instruction, at least one target inspection scenario is determined from the multiple inspection scenarios, and the detection rules to be executed for the target inspection scenario are obtained to generate an inspection task. Based on the scene terminal type of the target inspection scene, the page inspection operation is performed using the inspector corresponding to the scene terminal type, and page operation data is collected. The page operation data is input into a multimodal analysis model, which combines the detection rules to detect the page operation data, identify and output inspection anomaly information.

2. The method according to claim 1, characterized in that, Inspection scenarios for front-end pages are obtained from multiple data sources, including: Access entry information for the front-end page is obtained from multiple data sources; the data sources include at least two of the following: system menu, database configuration link, external projection link, traffic data, and UI script; The access entry information is standardized, and the corresponding scenario terminal type is identified based on the standardized access entry information. The access entry information is associated with the scene type to generate the inspection scene.

3. The method according to claim 1, characterized in that, The detection rules include general detection rules and / or special detection rules; wherein, the general detection rules are applicable to multiple scene terminal types; the special detection rules are determined according to the scene terminal type or preset label of the target inspection scene; The general detection rules include at least one of the following: white screen detection, interface error detection, code error detection, routing error detection, console error detection, path correctness detection, and login status verification; The specific detection rules include at least one of the following: sensitive word detection, non-compliant image detection, profit-sharing detection, component drop-down detection, product category consistency detection, and page title duplication detection.

4. The method according to claim 1, characterized in that, After generating the inspection task, the method further includes: If the number of target inspection scenarios included in the inspection task exceeds a preset threshold, the inspection task will be divided into multiple sub-tasks, each of which contains a set of target inspection scenarios. The multiple subtasks are placed into an execution queue for multiple inspector instances to execute in parallel or serially.

5. The method according to claim 1, characterized in that, Based on the scene type of the target inspection scenario, the page inspection operation is performed using the inspector corresponding to the scene type, including: When the target inspection scenario is an H5 page, the H5 inspector is invoked to perform page inspection operations by simulating user operations. When the target inspection scenario is a mini-program, the mini-program inspector is invoked. The mini-program inspector consists of a native inspection mini-program running on the terminal and a cloud scheduling service. The native inspection mini-program is driven by cloud commands to perform page jumps and simulated operations. When the target inspection scenario is a native client page, the client inspector is invoked. The client inspector connects to the terminal device through the system debugging bridge, locates the page elements, and then simulates page operations.

6. The method according to claim 1, characterized in that, Collect page runtime data, including: During page loading, log information is collected by listening; the log information includes runtime error information and network request records; If an anomaly is detected, the screenshot API is called to capture a screenshot of a portion of the page. The collected log information and the screenshot of the page area are associated with the corresponding target inspection scene identifier to generate the page operation data.

7. The method according to claim 1, characterized in that, The page execution data includes log information and screenshots of page areas; the page execution data is input into a multimodal analysis model, which, in conjunction with the detection rules, performs detection on the page execution data, including: The detection rules are analyzed to determine the set of anomaly types, which includes log anomaly types and visual anomaly types. The log information in the page running data is input into the rule engine, and the rule engine matches the log information according to the exception judgment logic corresponding to the log exception type to identify the log exception type; The page screenshot from the page operation data is input into the multimodal analysis model, which includes an optical character recognition module, a large language model module, and a visual language model module. The optical character recognition module extracts text information from the page area screenshot; the visual language model module extracts visual features from the page area screenshot; and the large language model module combines the extracted text information and visual features, and infers the visual anomaly type based on the clues corresponding to the visual anomaly type. The inspection anomaly information is generated based on the identified log anomaly type and the visual anomaly type.

8. The method according to claim 1, characterized in that, After identifying and outputting inspection anomaly information, the method further includes: The inspection anomaly information is correlated with the historical anomaly database to obtain correlation analysis results; the correlation analysis results include common source anomaly patterns. Based on the correlation analysis results, multiple abnormal information belonging to the same cause are aggregated to generate an abnormal root cause report. The abnormal root cause report includes at least one of the following: abnormal type, occurrence frequency, impact range, and alarm level.

9. The method according to claim 1, characterized in that, After identifying and outputting inspection anomaly information, the method further includes: Statistical analysis is performed on inspection anomalies identified in historical inspections to generate quality trend charts. The quality trend chart is aligned with the business release event in time to generate event anomaly information; the event anomaly information includes code release anomalies and / or configuration change anomalies. Based on the event anomaly information, optimization suggestions are generated, including adjusting the threshold of the detection rules, adding detection rules, or supplementing the multimodal analysis model with additional samples; the supplementary samples are used to optimize the multimodal analysis model.

10. A computer program product, characterized in that, It includes computer program instructions that are executed by a processor to perform the method as described in any one of claims 1 to 9.

11. An electronic device, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, perform the method as described in any one of claims 1 to 9.

Images