XPath generation method based on multi-strategy candidate generation and scoring screening

Abstract

The XPath generation method based on multi-strategy candidate generation and scoring selection includes: S1: Determining the target DOM element to be collected on the browser side and extracting the page feature information of the target DOM element; S2: Based on the extracted page feature information of the target DOM element, generating multiple candidate XPaths for the same target DOM element in parallel using multiple strategies; S3: Performing uniqueness verification on each generated candidate XPath when executed in the current page document object, retaining the candidate XPath that uniquely hits the target DOM element; S4: Constructing a scoring model, scoring and ranking the verified candidate XPaths, and selecting the candidate XPath with the highest score as the optimal XPath output; S5: Sending the optimal XPath and associated information back to the local server, where the local server writes the optimal XPath into the target configuration file according to a preset hierarchical structure. This invention significantly improves the probability of obtaining stable positioning expressions under complex page structures through multi-strategy parallel generation and can adapt to different front-end frameworks and page design patterns.

Description

Technical Field

[0001] This invention relates to the field of page element positioning technology, specifically to an XPath generation method based on multi-strategy candidate generation and scoring filtering. Background Technology

[0002] In scenarios such as web automation testing, page configuration data collection, and element location and maintenance, XPath is a commonly used method for locating page elements and is widely applied in the development of automated scripts and configuration management. Users typically expect to quickly obtain stable location expressions for target elements from a page for subsequent reuse in automated processes or configuration platforms.

[0003] In existing technologies, XPath generation methods mainly include the following: First, it relies on browser developer tools to directly copy the absolute path of the element in the DOM tree, which expands layer by layer from the document root node to the target element; Secondly, a positioning expression can be generated by simply concatenating elements' single attributes (such as id, class, name, etc.) or element text content; Third, a person can subjectively select an expression that they deem usable from among several available positioning schemes, and then manually organize it into a configuration file or script.

[0004] The above implementation method has at least the following problems: 1. Absolute paths expanded from the DOM root node are extremely sensitive to changes in the page hierarchy. After the front-end framework is recompiled, the page is redesigned, or components are dynamically rendered, the original paths are very easy to become invalid, resulting in high maintenance costs. 2. Single attribute locator methods are prone to hitting dynamically generated attribute values, such as hash class names injected by front-end frameworks, runtime random IDs, or frequently changing test attributes, causing the generated XPath to become inapplicable after page refresh; 3. There is a lack of a unified evaluation mechanism and screening criteria for multiple candidate localization expressions. Manual selection often relies on personal experience, and the quality of XPath output by different operators varies, making it difficult to ensure that the generated localization expressions have sufficient stability and maintainability. 4. Existing XPath generation methods lack effective integration with configuration management systems. The collected results still need to be manually copied, organized, and pasted into configuration files, which is not only inefficient but also makes it difficult to ensure the consistency and correctness of configuration formats in batch collection scenarios. Summary of the Invention

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: An automatic XPath generation method based on multi-strategy candidate generation and scoring filtering is characterized by the following steps: S1: On the browser side, determine the target DOM element to be collected, and extract the page feature information of the target DOM element. The page feature information includes at least tag information, attribute information, text information, sibling relationship, ancestor relationship and contextual semantic information. S2: Based on the extracted page feature information of the target DOM element, multiple candidate XPaths for the same target DOM element are generated in parallel using multiple strategies. The candidate XPaths include at least the exact matching candidates based on stable attributes, the matching candidates based on attribute prefixes, the positioning candidates based on element text, the axial positioning candidates based on the relationship between sibling DOM elements, the relative path candidates based on ancestor anchors, the text anchoring candidates based on semantic context, and the compressed path candidates extracted based on intermediate distinguishing DOM elements. S3: When each generated candidate XPath is executed in the current page document object, perform uniqueness verification and retain the candidate XPath that uniquely hits the target DOM element; S4: Construct a scoring model, score and rank the candidate XPaths that have passed the verification, and select the candidate XPath with the highest score as the optimal XPath output; S5: The optimal XPath and associated information are sent back to the local server, and the local server writes the optimal XPath into the target configuration file according to a preset hierarchical structure.

[0006] Furthermore, the specific principle of the scoring model in step S4 is as follows: determine the attribute stability, business semantics, path length, number of indexes, and structural chain complexity of the candidate XPath, and assign scores to them respectively. Use the scores of attribute stability, business semantics, path length, number of indexes, and structural chain complexity of the candidate XPath to calculate the score of the candidate XPath.

[0007] Furthermore, the uniqueness means that after executing the XPath through the page DOM, the result must hit only one DOM element, and that DOM element must be the target DOM element currently being collected. Only the candidate XPath that passes the uniqueness verification will enter the subsequent scoring and sorting process.

[0008] Furthermore, before performing the uniqueness verification, the candidate XPaths also need to undergo stability-oriented feature filtering and multi-stage semantic enhancement: The stability-oriented feature filtering includes: Filter out unstable features in the candidate XPath, which include at least: empty attribute value, pure number attribute value, and attribute value that is suspected to be a random hash string or a dynamic class name generated by the front-end framework. The semantic enhancement is as follows: Based on the stability-oriented feature filtering, the candidate XPaths are solved step by step in the order of "prioritizing the target element itself, assisting with sibling relationships, expanding ancestor anchors, and supplementing with text context", thereby enhancing the semantics of the candidate XPath expressions.

[0009] Furthermore, the feature is that the local server writing of the target configuration file in step S5 specifically includes: After receiving the optimal XPath and the associated information returned by the browser, it is temporarily stored in the local memory cache. The local server performs temporary storage, verification, and batch saving. Before structured writing, a backup file of the target configuration file is generated. The associated information includes at least: configuration name, source page address, tag name, and text summary. Write the cached XPath configuration data to the target file in a hierarchical structure in YAML format, and roll back based on the backup file if the write fails.

[0010] Furthermore, the scoring model records a basic score for the candidate XPath. When the average or median score of the candidate XPath is lower than the basic score, the overall quality of the candidate XPath is considered insufficient. The scoring model then returns a result indicating that no valid XPath has been generated and prompts the user to reselect a more semantically meaningful target region.

[0011] Furthermore, the local server provides the user with the optimal XPath and associated information from its local memory cache in real time via a state update request.

[0012] Furthermore, users can perform operations such as deleting a single record, clearing all records, or adding new collection results based on the optimal XPath and associated information they have viewed.

[0013] Furthermore, steps S1-S4 are all performed on the browser side.

[0014] Furthermore, the target DOM elements to be collected originate from user interactions with the page.

[0015] Compared with the prior art, this invention has the following beneficial effects: 1. Compared with existing technologies that only provide a single absolute path or single attribute for positioning, this invention significantly improves the probability of obtaining a stable positioning expression under complex page structures by generating multiple strategies in parallel, and can adapt to different front-end frameworks and page design patterns.

[0016] 2. Compared to existing technologies that directly output unverified path expressions, this invention ensures that the generated XPath has accurate element pointing at the moment of acquisition, reducing subsequent automated execution errors caused by inaccurate positioning.

[0017] 3. Compared to existing technologies that rely on manual experience to select localization expressions, this invention achieves unified and automated evaluation criteria. The output XPath maintains a higher survival rate even after local changes in the page structure, significantly reducing subsequent localization and maintenance costs. Furthermore, this invention introduces a structural chain penalty mechanism, deducting points or rejecting output for low-quality candidate expressions such as those with excessively long consecutive general structural nodes, too many indexes, or overly verbose paths, thus suppressing the generation of fragile XPaths at the source.

[0018] 4. Compared with the existing technology that requires manual copying, sorting and pasting into the configuration file, the present invention realizes an automated closed loop from page collection to configuration disk storage, which is especially suitable for batch collection scenarios, and can greatly improve the efficiency of configuration generation and ensure the consistency of configuration format. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Figure 1 This is a diagram illustrating the method steps of the present invention; Figure 2 This is a flowchart of the XPath generation algorithm of the present invention; Figure 3 This is a system flowchart of the present invention; Figure 4-7 This is a schematic diagram illustrating the actual use of the present invention. Detailed Implementation

[0021] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0022] Example 1 This invention provides a method for automatically generating and writing structured XPath based on browser-side page capture and local server-side collaborative processing, such as... Figure 1 As shown, the specific method includes the following steps: S1: On the browser side, determine the target DOM element to be collected and extract the page feature information of the target DOM element. The page feature information includes at least tag information, attribute information, text information, sibling relationship, ancestor relationship and contextual semantic information. S2: Based on the extracted page feature information of the target DOM element, multiple candidate XPaths for the same target DOM element are generated in parallel using multiple strategies. The candidate XPaths include at least the following: precise matching candidates based on stable attributes, matching candidates based on attribute prefixes, positioning candidates based on element text, axial positioning candidates based on sibling DOM element relationships, relative path candidates based on ancestor anchors, text anchoring candidates based on semantic context, and compressed path candidates based on intermediate distinguishing DOM elements extracted. S3: Perform a uniqueness verification on each generated candidate XPath in the current page document object at runtime, and retain the candidate XPath that uniquely hits the target DOM element; S4: Construct a scoring model, score and rank the validated candidate XPaths, and select the candidate XPath with the highest score as the optimal XPath output; S5: The optimal XPath and associated information are sent back to the local server, which then writes the optimal XPath into the target configuration file according to the preset hierarchical structure.

[0023] Furthermore, it can be subdivided into the following stages: Phase 1: Start the local service process The user triggers the startup module to run, reads preset or input running parameters, initializes the listening address, listening port and default hierarchical path configuration, and starts the local HTTP service to establish the communication foundation between the browser and the local server.

[0024] Phase Two: Generate Local Management Page The local browser sends a homepage access request to the local HTTP service. The request processing routing module identifies the request and calls the page template processor to combine the page template with the current running parameters to generate a local management page, which is then returned to the browser for display. The local management page is used for management functions such as displaying collection results, providing collection scripts, deleting results, clearing results, and saving results.

[0025] Phase 3, Data Acquisition Result Status Synchronization Mechanism After the local management page is loaded, it periodically sends status update requests to the request processing routing module. The request processing routing module reads the current results to be saved from the collection result cache and returns the results and corresponding status information to the local management page, so that users can view the configuration name, XPath, source page address and other relevant information that have been collected in real time.

[0026] Phase Four: Provide page scraping scripts to the target business webpages. After a user executes the data collection script on the local management page, the page provides the script to the user for execution on the target business webpage. The data collection script runs in the actual page environment of the target business webpage, used to detect the target element selected by the user and trigger the subsequent XPath generation process.

[0027] Phase 5: Determine the target elements to be collected. Users select elements to be collected on the target business webpage. These selection actions include, but are not limited to, mouse movement, clicking buttons, clicking input boxes, clicking table cells, or other page interactions. The page DOM provides the corresponding target elements to the page collection script based on the user's current action. The page collection script then identifies the target object and confirms it through methods such as highlighting, click interception, and named input.

[0028] Stage Six: Generate the XPath corresponding to the target element After identifying the target element, the page scraping script sends an XPath generation request to the XPath generator, passing the current target element and its associated DOM context to the XPath generator. The XPath generator generates multiple candidate XPaths based on tag information, attribute information, text information, sibling relationships, ancestor relationships, and contextual semantic information within the page DOM. Subsequently, each candidate XPath undergoes rule-based filtering, stability-oriented feature filtering, multi-stage semantic enhancement, uniqueness verification, and scoring ranking to obtain the optimal XPath result that best matches the current target element.

[0029] In other words, the ranking of candidate XPaths is not based solely on whether the XPath can hit the target element. Instead, it comprehensively evaluates the maintainability, semantic strength, and structural stability of the candidate XPaths, based on the fact that they have passed uniqueness verification, stability-oriented feature filtering, and multi-stage semantic enhancement.

[0030] Uniqueness verification means that after executing the XPath through the page DOM, the result must hit only one node, and that node must be the target element being sampled. Only XPaths that pass this verification will proceed to the subsequent scoring and ranking process.

[0031] So-called stability-oriented feature filtering includes: Filter out unstable features in candidate XPaths. Unstable features include at least: empty attribute values, pure numeric attribute values, and attribute values ​​that are suspected to be random hash strings or dynamically generated class names by the front-end framework. Semantic enhancement to: Based on stability-oriented feature filtering, the candidate XPath is solved step by step in the order of "prioritizing the target element itself, assisting with sibling relationships, expanding ancestor anchors, and supplementing with text context", thereby enhancing the semantics of the candidate XPath expression.

[0032] In other words, the XPath generation process does not use a single absolute path concatenation method, but rather generates multiple candidate XPaths for the current target element. These candidate XPaths include at least: exact match candidates based on stable attributes, match candidates based on attribute prefixes, location candidates based on element text, axial location candidates based on sibling node relationships, relative path candidates based on ancestor anchors, text anchoring candidates based on semantic context, and compressed path candidates extracted from intermediate distinguishing nodes. For each generated candidate XPath, the system further performs uniqueness verification and ranks them based on factors such as attribute stability, business semantics, path length, number of indexes, and structural chain complexity, selecting the XPath with the highest score as the output.

[0033] Attribute stability is primarily determined by the type of attribute. The system prioritizes attributes such as `id`, `data-testid`, `data-test`, `data-qa`, `data-name`, `aria-label`, `name`, `placeholder`, `title`, `role`, and `type`, which are considered more stable and have stronger business meaning. Attributes themselves also undergo stability checks; for example, pure numbers, excessively long strings, UUIDs, long hashes, dynamic values ​​with many numeric suffixes, and values ​​suspected of being generated by React / Vue / CSS frameworks are considered less stable. In quantification, different attributes are assigned different base scores. For example, `id` has the highest weight, followed by `data-testid`, `data-test`, and `data-qa`, and then `aria-label`, `name`, `placeholder`, and `title`. If `id` is suspected of being dynamically generated, additional points will be deducted. Therefore, the more stable, business-oriented, and unlike runtime-generated values ​​an attribute is, the higher its score.

[0034] As shown in Table 1:

[0035] Business semantics are primarily determined through keywords in attribute values, text, and class names. The system extracts semantic tokens from id, data-testid, data-test, data-qa, aria-label, name, title, and stable class tokens. If a token matches business-related terms such as search, input, form, dialog, modal, panel, menu, table, tree, login, upload, download, filter, or toolbar, the semantic score increases; conversely, if it matches generalized container terms such as wrapper, container, content, item, box, or layout, the score decreases. Essentially, data-testid="search-submit" better expresses the business intent than class="container item," thus giving the former an advantage in ranking.

[0036] The business semantic quantification rules are shown in Table 2:

[0037] Path length is primarily measured by the length of the XPath string. The probability of failure increases with slight adjustments to the page structure. Therefore, the system deducts points for paths exceeding a certain length; for example, after exceeding three path segments, each additional segment lowers the score. Furthermore, excessively long XPath strings also incur additional point deductions.

[0038] The quantization method can be expressed as: P_length = max( 0, L_chars-120) × 0.12 The `max` function represents the maximum value of the numbers in parentheses, `S` represents the score, `P_length` represents the penalty for path length, `L_segments` represents the number of XPath path segments, and `L_chars` represents the length of the XPath string.

[0039] The index count refers to the number of numerical positional predicates appearing in the XPath, such as div[2], span[3], and button[1]. This type of index depends on the DOM order, and adding or deleting sibling nodes on the page can easily lead to positioning offsets. Therefore, the system will deduct points from the index count. The quantification method can be expressed as: P_index = N_index × 8 Here, P_index represents the penalty for the number of indexes, and N_index represents the number of numeric index predicates in the XPath. In remediation scenarios or risk analysis, this type of XPath will also be marked as a location index risk.

[0040] Structural chain complexity is primarily used to measure whether an XPath depends on consecutive nodes without semantic meaning, such as consecutive div / span / section / ul / li nodes. If an XPath contains multiple consecutive structural fragments without business attributes, text anchors, or only tag names or pure numeric indices, it indicates that the XPath is essentially relying on page layout rather than business semantics.

[0041] The specific rules for quantifying the complexity of structural chains are shown in Table 3:

[0042] The system will count the longest consecutive general structure chain and apply a squared penalty: P_structure = C_generic^2×18 Here, P_structure represents the penalty for excessive complexity in the structure chain, and C_generic represents the number of longest consecutive generic structure segments. If this value reaches 4, the system will consider the XPath to be overly dependent on the DOM structure, directly deem it non-compliant, and will not include it in the final output.

[0043] In summary, the score for a candidate XPath can be summarized as follows: S=S_anchor+S_semantic+ -P_index-P_length-P_structure Where S_anchor represents the attribute stability score and S_semantic represents the business semantic score; finally, the system will use the scoring model to prioritize XPath with stable attributes, clear business semantics, short path, few indexes, and low structural chain complexity as the optimal result.

[0044] In addition, the scoring model records a basic score for the candidate XPath. If the average or median score of the candidate XPath is lower than the basic score, the overall quality of the candidate XPath is considered insufficient. The scoring model returns a result that no valid XPath was generated and suggests reselecting a more semantically meaningful target region.

[0045] Phase 7, transmitting the collected results After obtaining the optimal XPath, the page collection script sends the associated information, such as the configuration name, XPath, source page address, tag name, and text summary, to the request processing routing module. After receiving the associated information, the request processing routing module writes it into the collection result cache to form a set of results to be saved.

[0046] Phase 8: Managing the collected results The local management page retrieves the latest content from the cached collection results via a status update request and displays the content to the user. Users can then perform operations such as deleting a single record, clearing all records, or adding more collection results based on the displayed results. The corresponding management requests are forwarded to the collection result cache by the request processing and routing module for execution, thus ensuring consistency between the cached data and the content displayed on the page.

[0047] Phase Nine: Triggering the Structured Saving Operation Once the user confirms that the current collection results meet the requirements, a save request is initiated on the local management page. The request processing routing module passes the current collection results, the target YAML file path, and the target hierarchical path to the YAML file writer to enter the structured writing stage.

[0048] Phase 10: Perform YAML backup and write. Before actually writing to the target YAML file, the YAML file writer first generates a pre-write backup file to preserve the original configuration content, which can be used for rollback based on the backup file in case of write failure. Then, according to the target hierarchical path, it writes the configuration name and corresponding XPath result to the target YAML file according to the preset structure. During the writing process, the YAML file writer is responsible for performing path creation, result overwriting, file writing to disk, and write security control, thereby completing the structured output of the collected results.

[0049] Phase 11: Return the processing result and complete the loop closure. After the YAML file is written, the request processing routing module returns the processing result to the local management page, which then displays information to the user indicating whether the save was successful or failed. This completes the entire processing flow from target element selection, automatic XPath generation, result caching, to the structured storage of the YAML file on disk.

[0050] In this invention, the local server-side processing system comprises the startup entry point, local HTTP service, request processing routing, page template processor, data collection result cache, and YAML file writer; the browser-side execution system comprises the local management page, page data collection script, XPath generator, and page DOM. The browser-side execution system is responsible for target element detection and XPath generation in a real page environment, while the local server-side processing system is responsible for result reception, state management, and structured configuration writing. Together, they achieve a stable, reusable, and easily batch-manageable automatic XPath generation method.

[0051] This embodiment achieves the following effects: 1. A multi-strategy candidate XPath parallel generation mechanism based on real page context.

[0052] This invention does not involve offline path concatenation outside the page environment. Instead, it operates within the browser's real-world page environment, taking the target DOM element as input, reading its tags, attributes, text, sibling relationships, ancestor relationships, and local structural semantics, and generating multiple candidate XPaths in parallel around the same target element. This mechanism forms the basis for subsequent selection of the optimal XPath.

[0053] 2. Stability-oriented feature filtering and multi-stage semantic enhancement localization mechanism.

[0054] This invention does not unconditionally use all attributes and values ​​on elements. Instead, it first filters out unstable features such as null values, pure numeric values, suspected random hashes, suspected UUIDs, dynamic class names, and state classes. Based on this, the system solves the problem step by step in the order of "prioritizing the target element itself, assisting with sibling relationships, expanding ancestor anchors, and supplementing with text context", thereby obtaining XPath expressions with stronger semantics and more stable structure.

[0055] 3. Runtime uniqueness verification and unified scoring and screening mechanism for candidate XPaths.

[0056] For each generated candidate XPath, this invention requires that it be evaluated and verified in the actual current page document object of the target page, and only the candidate that uniquely hits the current target element is retained. For the candidates that pass the verification, they are then uniformly scored and sorted according to dimensions such as anchor stability, business semantics, number of indexes, path depth, structural complexity and total length, and the XPath with the highest score is automatically selected as the final output.

[0057] 4. Structural penalties, compressed paths, and controlled output mechanisms.

[0058] This invention adopts a rejection strategy for low-quality candidate XPaths that are too long in continuous general structure chains, have too many indexes, have redundant paths, or are mechanically expanded from the root node. When the overall quality of the candidates is insufficient, the system does not directly degenerate into a fragile absolute path, but returns the result of not generating a valid XPath and prompts for reselection of a more semantic target region, thus suppressing the generation of low-quality XPaths from the source.

[0059] 5. A closed-loop processing mechanism for browser-side generation and local structured configuration writing.

[0060] After generating and validating the optimal XPath on the browser side, this invention associates the XPath result with information such as configuration name, source page address, tag name, and text summary and sends it back to the local server. The local server then performs temporary storage, validation, batch saving preprocessing, and YAML structure writing, thus forming a complete closed loop from page collection and automatic XPath generation to configuration file storage.

[0061] Example 2 This embodiment describes the actual implementation plan and system architecture construction.

[0062] 1. Technological groundwork: The underlying object of this solution is the DOM (Domain of Documentation) in the browser. The DOM can be understood as a structure tree that the browser creates for web page content. Interactive objects on the page, such as buttons, input boxes, tables, text areas, and icons, correspond to nodes in this structure tree. This solution does not identify screenshots during data collection; instead, it directly analyzes and locates the actual DOM nodes clicked by the user. Therefore, it can read structured information such as element tag names, attributes, text content, parent-child hierarchy, and sibling relationships.

[0063] XPath is a path representation method used to describe the positional relationships of nodes within a page structure. It is widely used in scenarios such as web automation testing page configuration collection and data extraction. Compared to methods that rely solely on screen coordinates or visual position, XPath can directly locate target elements based on the page structure. However, traditional absolute paths or single-attribute XPaths are prone to quickly becoming invalid due to changes in page hierarchy, front-end framework re-rendering, dynamic class names, or runtime ID changes. Therefore, a mechanism is needed to automatically filter and generate more stable positioning expressions.

[0064] To obtain a more stable XPath, this scheme relies on several page feature information that can be used as a basis for localization, including the element's own stable attributes and contextual semantic information. The former mainly includes attributes such as id, data-testid, data-test, data-qa, data-name, aria-label, name, placeholder, title, role, type, alt, and value, as well as the filtered stable attributes; the latter mainly includes the element's own available text, adjacent sibling nodes, upper-level ancestor containers, text context, and local structural relationships. This information together forms the basis for XPath candidate generation and subsequent scoring and selection.

[0065] This solution also relies on the browser's native page evaluation capabilities. After the collection script is injected into the target webpage, the script runs in the real page context, using a scoring system to perform real-time evaluation and verification of candidate XPaths. This allows the system to determine whether a given XPath uniquely matches the current target element at the moment of collection.

[0066] From a system architecture perspective, this solution relies on a collaborative approach of "browser-side data collection + local server-side management." The browser is responsible for detecting the user-selected elements, generating and validating XPaths; the local Python service is responsible for providing the management page, receiving the collected results, maintaining the cache to be saved, and handling operations such as deletion, clearing, and batch saving. XPath generation occurs in the actual DOM environment of the target webpage, while the management, confirmation, and storage of the collected results are completed in the local server environment, forming a complete closed loop.

[0067] Furthermore, this solution relies on a structured configuration medium to save and reuse the collected results. Currently, YAML files are used as the configuration medium. After the user specifies the target YAML file path and hierarchical path in the management page, the system writes the "configuration name + XPath" to the file according to the predetermined hierarchical structure, and automatically generates a backup file before writing. With this structured output method, the collected results are no longer fragmented text, but standardized configuration data that can be directly reused by subsequent automation scripts, configuration platforms, or testing processes.

[0068] 2. Actual performance of the product The following is the practical procedure, referring to the attached diagram: like Figure 4 As shown, start it from the system command line; like Figure 5 As shown, users access the terminal output link using a browser to open the console page, and fill in the target YAML file path and target hierarchy path; Subsequently, the user executes the collection script through the console on the target business webpage and enters the collection mode. At this time, the system displays a highlight box and prompt information on the page. After the user clicks on the element to be collected, the system pops up a name input box to enter the name. like Figure 6 As shown, after the user enters the configuration name, the browser immediately calculates the optimal XPath for the target element based on the current DOM and sends the configuration name, XPath, page address, tag name, and text digest back to the local service. The returned results are displayed on the local management page, where users can delete, clear, or save in batches. like Figure 7 As shown, when you click "Save to YAML", the system writes the results to the target file according to the predetermined YAML structure.

[0069] 3. Detailed steps and process of technical implementation like Figure 3 As shown, the following is the production system architecture for XPath constructed based on this technical solution: Startup entry point: The entry point for the entire program. It is responsible for reading startup parameters, initializing the program, and starting local services.

[0070] Local HTTP service: This is a small server running on the local machine. It is responsible for receiving various requests from the browser, allowing local management pages and data scraping scripts to communicate with the local Python process.

[0071] Request processing routing: This is the "request distribution center". It determines whether the browser's current request is to open the homepage, query the status, add the collected results, delete the record, or save to YAML, and then assigns the request to the corresponding processing logic.

[0072] Page Template Processor: This module generates page content. It assembles a fixed page template with the current running parameters to generate the final management page and data collection script content displayed to the browser.

[0073] Data collection result cache: A temporary storage location for data collection results. Think of it as an "in-memory list to be saved." Elements collected by the user are not immediately written to a file; instead, they are stored here until the user confirms and then saved collectively.

[0074] YAML file writer: A module specifically responsible for writing configuration files. It writes the cached collection results into the target YAML file according to a predetermined format, while handling path creation, name overwriting, and write security.

[0075] Target YAML file: The final configuration file that saves the results. Subsequent programs that need to use these XPaths will generally read from this file.

[0076] YAML pre-write backup file: An automatically generated backup file before the actual write operation. Its purpose is to prevent unrecoverable data loss due to accidental overwriting or write failure, essentially providing a backup copy of the original file.

[0077] Local management page: The tool's user interface, equivalent to a "console". Here, users can view the collected content, copy the collection script, delete unnecessary records, or save the results to a configuration file.

[0078] Page DOM: The "structured page model" of a business webpage within the browser. It can be simply understood as: buttons, input boxes, tables, and text on a webpage each have a corresponding structural node within the browser, which the system uses to determine "which element the user clicked".

[0079] Page scraping script: A small tool that temporarily runs on a business webpage. It is responsible for detecting which page element the user is currently pointing to or clicking, highlighting the element, and passing the target element to the subsequent XPath generation logic for processing.

[0080] XPath generator: A module specifically designed to calculate the location path of page elements. It analyzes the current element's position, attributes, and context on the page to generate a stable and reusable XPath, making it easier for subsequent automated scripts to accurately locate the element.

[0081] User -> Page DOM: When a user moves the mouse, clicks a button, or enters content on a business page, they are actually selecting a specific element on the page. The system will then identify and locate this element.

[0082] User -> Local Management Page: Users can view the collected results, perform deletion or clearing operations through the local management page, and decide whether to save the results.

[0083] User -> Startup Entry Point: The user manually starts this tool program, and the entire local service begins to run.

[0084] Startup entry point -> Local HTTP service: The startup entry point is responsible for starting the local HTTP service, allowing the browser to access the local management page and related interfaces.

[0085] Local HTTP service -> Request processing route: All requests from the browser first enter the local HTTP service, and then the request handling route determines which functional module should handle them.

[0086] Request processing route -> Page template processor: When the browser needs to open the local management page or retrieve the collection script, the request processing route will call the page template processor to generate the corresponding content.

[0087] Request processing route -> Collection result caching: When the browser sends requests such as adding, deleting, clearing, or querying status, the request processing route will read or update the collected results in the cache.

[0088] Request processing route -> YAML file writer: When the user clicks save, the request processing route will hand over the currently cached result to the YAML file writer to complete the actual writing.

[0089] Local management page -> Request processing route: When the local management page is opened, it requests the homepage content from the backend; subsequent operations on the interface are also sent to the request processing route via requests.

[0090] Local Management Page -> Page Collection Script: The local management page is responsible for providing users with the collection script content. After copying it, users can execute it on the business page to enter the element collection mode.

[0091] Page DOM -> Page scraping script; The specific elements on the page are provided to the data collection script as the current target, which tells the script which button, input box, or text area the user is selecting.

[0092] Page scraping script -> XPath generator: The collection script hands over the currently selected page element to the XPath generator, asking it to calculate a usable element location path.

[0093] XPath generator -> Page DOM: The XPath generator reads the entire page structure, analyzes element attributes, hierarchical relationships, and surrounding context, in order to generate more stable and accurate XPaths.

[0094] Page scraping script -> Request processing route: Once the XPath is generated, the collection script will submit the collection results back to the local service, and the request processing route will then receive and store them in the cache.

[0095] YAML file writer -> target YAML file: The YAML file writer will write the final confirmed collection results into the official configuration file for use by subsequent systems or scripts.

[0096] YAML file writer -> Backup file before writing YAML: Before the actual write operation, the system will first generate a backup file. In the event of a write failure, the system will roll back based on the backup file to prevent the loss of the original configuration due to accidental operation or write failure.

[0097] This embodiment achieves the following effects: Instead of simply copying an XPath from the page or writing the collected results to a configuration file, this method generates multiple candidate XPaths in parallel within the real page runtime context for the target DOM element. It then performs unstable feature filtering, multi-stage backtracking, uniqueness verification, scoring and filtering, and controlled output on the candidates, ultimately automatically obtaining a more stable, shorter, and easier-to-maintain optimal XPath.

[0098] This embodiment establishes a complete recording system and describes the practical process of the recording method in more detail. The user's operation process is completely recorded and saved through the recording system.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An XPath automatic generation method based on multi-strategy candidate generation and scoring screening, characterized in that, The method includes the following steps: S1: On the browser side, determine the target DOM element to be collected, and extract the page feature information of the target DOM element. The page feature information includes at least tag information, attribute information, text information, sibling relationship, ancestor relationship and contextual semantic information. S2: Based on the extracted page feature information of the target DOM element, multiple candidate XPaths for the same target DOM element are generated in parallel using multiple strategies. The candidate XPaths include at least the exact matching candidates based on stable attributes, the matching candidates based on attribute prefixes, the positioning candidates based on element text, the axial positioning candidates based on the relationship between sibling DOM elements, the relative path candidates based on ancestor anchors, the text anchoring candidates based on semantic context, and the compressed path candidates extracted based on intermediate distinguishing DOM elements. S3: Perform a uniqueness verification on each generated candidate XPath in the current page document object at runtime, and retain the candidate XPath that uniquely hits the target DOM element; S4: Construct a scoring model to score and rank the verified candidate XPaths, and select the candidate XPath with the highest score as the optimal XPath output; S5: The optimal XPath and associated information are sent back to the local server, and the local server writes the optimal XPath into the target configuration file according to a preset hierarchical structure.

2. The XPath automatic generation method based on multi-strategy candidate generation and scoring screening according to claim 1, characterized in that, The specific principle of the scoring model in step S4 is as follows: determine the attribute stability, business semantics, path length, number of indexes, and structural chain complexity of the candidate XPath, and assign scores to them respectively. Use the scores of attribute stability, business semantics, path length, number of indexes, and structural chain complexity of the candidate XPath to calculate the score of the candidate XPath.

3. The method for XPath auto-generation based on multi-policy candidate generation and scoring screening according to claim 1, characterized in that, The uniqueness refers to the fact that after executing the XPath through the page DOM, the result must hit only one DOM element, and that DOM element must be the target DOM element currently being collected. Only the candidate XPath that passes the uniqueness verification will enter the subsequent scoring and sorting process.

4. The method for XPath auto-generation based on multi-policy candidate generation and scoring screening according to claim 1, characterized in that, Before performing the uniqueness verification, the candidate XPaths also need to undergo stability-oriented feature filtering and multi-stage semantic enhancement: The stability-oriented feature filtering includes: Filter out unstable features in the candidate XPath, which include at least: empty attribute value, pure number attribute value, and attribute value that is suspected to be a random hash string or a dynamic class name generated by the front-end framework. The semantic enhancement is as follows: Based on the stability-oriented feature filtering, the candidate XPaths are solved step by step in the order of "prioritizing the target element itself, assisting with sibling relationships, expanding ancestor anchors, and supplementing with text context", thereby enhancing the semantics of the candidate XPath expressions.

5. The method for XPath auto-generation based on multi-policy candidate generation and scoring filtering according to claim 1, characterized in that, The specific steps in step S5, where the local server writes the target configuration file, include: After receiving the optimal XPath and the associated information returned by the browser, it is temporarily stored in the local memory cache. The local server performs temporary storage, verification, and batch saving. Before structured writing, a backup file of the target configuration file is generated. The associated information includes at least: configuration name, source page address, tag name, and text summary. Write the cached XPath configuration data to the target file in a hierarchical structure in YAML format, and roll back based on the backup file if the write fails.

6. The method for XPath auto-generation based on multi-policy candidate generation and scoring screening according to claim 2, characterized in that, The scoring model records a basic score for the candidate XPath. When the average or median score of the candidate XPath is lower than the basic score, the overall quality of the candidate XPath is considered insufficient. The scoring model returns a result indicating that no valid XPath has been generated and suggests reselecting a more semantically meaningful target region.

7. The method of claim 5, wherein the method further comprises: The local server provides the user with the optimal XPath and associated information from its local memory cache in real time via a status update request.

8. The method of claim 7, wherein the XPath autogeneration based on multi-policy candidate generation and scoring screening is characterized by, Users can perform operations such as deleting a single record, clearing all records, or adding new collection results based on the optimal XPath and related information they see.

9. The method for XPath auto-generation based on multi-policy candidate generation and scoring filtering according to claim 1, characterized in that, Steps S1-S4 are all performed on the browser side.

10. The method for XPath auto-generation based on multi-policy candidate generation and scoring filtering according to claim 2, characterized in that, The target DOM elements to be collected originate from user interactions with the page.

Images