Intelligent element positioning method and system based on AI and dynamic feature library

Abstract

The invention discloses an intelligent element positioning method and system based on AI and a dynamic feature library, and the method comprises the steps: carrying out the anomaly detection and abnormal data capture, executing an element positioning operation through an automatic script, and recording a page DOM tree structured snapshot when the positioning is judged to be failed; multi-modal feature extraction: analyzing the DOM tree, and extracting an XPath / CSS path and semantic attributes of a target element; performing dynamic feature library retrieval and matching, calculating DOM structure similarity, and taking a high-similarity rule as a correction basis if the high-similarity rule exists; an AI intelligent rule is generated, and candidate positioning rules are generated through the classification model; performing script correction and real-time feedback, replacing the failure positioning statement and updating the dynamic feature library; and generating a structured report, and recording abnormal data. According to the method, the AI is combined with the dynamic feature library, so that the intelligent correction of element positioning in the automatic test process is realized, and the stability and efficiency of the automatic test are improved.

Description

Technical Field

[0001] The present invention relates to the field of automated testing, and in particular to an intelligent element positioning method and system based on AI and a dynamic feature library. Background Art

[0002] With the rapid development of internet technology and the increasing complexity of web and mobile applications, automated testing is playing an increasingly important role in the software development lifecycle. Element location is a fundamental and critical step in automated testing, and its stability and accuracy directly impact the reliability and maintenance costs of test scripts.

[0003] Currently, mainstream element location methods mainly include static location methods based on XPath, CSS selectors, ID, Name, and other attributes. However, with the development of front-end technology, especially the widespread use of dynamic rendering, responsive design, and single-page applications (SPAs), traditional static location methods face many challenges.

[0004] Existing element location methods suffer from the following technical issues: 1. Strong reliance on static rules: Existing element location methods mostly rely on static rules. Changes to element attributes or page structure require manual script modification, resulting in high maintenance costs. This is especially true in agile development environments, where frequent interface changes necessitate constant adjustments to automated test scripts, severely impacting test efficiency.

[0005] 2. Poor adaptability to dynamic content: Existing technologies struggle to effectively handle dynamically generated elements or complex UI interaction scenarios. In single-page applications (SPAs), asynchronously loaded content, and dynamically rendered pages, element attributes and positions can change frequently, resulting in a high rate of positioning failures.

[0006] 3. Low efficiency in handling false positives: When element positioning fails, existing technologies often require manual intervention to analyze the cause and repair the script. These technologies lack automated repair and feedback learning mechanisms, making closed-loop optimization impossible.

[0007] 4. Lack of a dynamically updated rule base: Existing technologies have yet to propose an intelligent positioning solution that can dynamically update the rule base and utilize AI self-learning, making it impossible to automatically optimize positioning strategies based on historical positioning successes.

[0008] 5. Insufficient Utilization of Multimodal Features: Existing technologies fail to fully utilize multimodal features such as DOM structure, visual features, and semantic information when locating elements, resulting in insufficient positioning accuracy in complex scenarios.

[0009] Therefore, there is an urgent need for an element positioning method that can automatically detect element positioning anomalies, intelligently generate repair rules, dynamically update the feature library, and have self-learning capabilities to improve the stability and efficiency of automated testing and reduce maintenance costs. Summary of the Invention

[0010] In order to solve technical problems existing in automated testing or web page / application element positioning, such as strong dependence on static rules, poor adaptability to dynamic content, low efficiency in handling false positives, and lack of a dynamically updated rule library and intelligent positioning solution using AI self-learning, the present invention provides an intelligent element positioning method based on AI and a dynamic feature library, which achieves technical effects such as automatic repair and learning capabilities of element positioning, improved adaptability to dynamic content, reduced need for manual intervention, improved positioning accuracy, and provision of structured failure reports. The method comprises the following steps: S1. Anomaly detection and abnormal data capture: Element location operations are performed through automated scripts. If the preset timeout period is exceeded, the location is considered to have failed, abnormal data is captured, and a structured snapshot of the current page DOM tree is recorded. S2. Multimodal feature extraction: Perform structured feature extraction on the captured abnormal data, parse the DOM tree, and extract the XPath / CSS path and semantic attributes of the target element; S3. Dynamic feature library search and matching: First, a rule search is performed in the dynamic feature library for the multimodal feature vector of the current element, and the DOM structure similarity is calculated. If there are rules with a total similarity ≥ 85% in the dynamic feature library, all rules with a total similarity ≥ 85% are sorted from high to low by weight, and the rule with the highest similarity is selected as the basis for correction, and step S5 is performed; if there are no rules with a total similarity ≥ 85% in the dynamic feature library, step S4 is performed; S4, AI intelligent rule generation: Based on the multimodal feature vector and the global structure of the page, a classification model is used to generate multiple candidate positioning rules, which are sorted by weight. The candidate positioning rules are tried in sequence. If positioning is successful, the attempt is stopped and step S5 is executed; S5, script correction and real-time feedback: Replace the invalid positioning statement in the original script with a new rule. If the correction is successful, the new rule is added to the dynamic feature library. If the correction fails, return to step S3 or S4 and select other rules to correct until the correction is successful. If all rules fail, execute step S6; S6. Generate structured report: Record complete abnormal data and generate a structured report, marking possible reasons for failure.

[0011] As a preferred technical solution, the abnormal data in step S1 also includes capturing a screen image and capturing context information; after capturing the screen image, the area where the target element is located is located using the coordinate information of the failed operation and the image is cropped; the captured context information includes the page URL, browser and / or device type, and user operation sequence.

[0012] As a preferred technical solution, the XPath / CSS path of the target element extracted in step S2 is generated based on the DOM hierarchy to generate an absolute path and a relative path.

[0013] As a preferred technical solution, the semantic attributes of the target element extracted in step S2 include element text content, id / class attributes, and neighboring elements.

[0014] As a preferred technical solution, in step S3, "searching for rules in the dynamic feature library for the multimodal feature vector of the current element and calculating the DOM structure similarity" is specifically as follows: traversing the rules of the same platform in the dynamic feature library and calculating the weighted similarity: total similarity = 0.5*DOM structure similarity + 0.5*context similarity; DOM structure similarity is the path difference calculated based on the tree edit distance algorithm.

[0015] As a preferred technical solution, in step S4, "generating multiple candidate positioning rules through a classification model" is specifically: generating Top-3 candidate positioning rules through a LightGBM classification model.

[0016] As a preferred technical solution, the conditions for successful positioning in step S4 are: the elements are interactive and the attributes meet expectations.

[0017] As a preferred technical solution, in step S5, if the correction is successful and the new rule is added to the dynamic feature library, the initial weight = 1, and the weight increases by α each time the same rule in the dynamic feature library is successfully used; in step S5, when the correction fails, the weight decreases by β, and when the weight of a rule in the dynamic feature library is less than the threshold, the rule is set to a frozen state and no longer participates in matching.

[0018] The present invention also provides an intelligent element positioning system based on AI and a dynamic feature library for implementing the above method, comprising an anomaly detection and abnormal data capture module, an intelligent positioning module, a dynamic feature library module, an anomaly handling module, a script correction and feedback module, and a structured report generation module; wherein: The anomaly detection and abnormal data capture module is used to detect positioning anomalies and capture abnormal data; The intelligent positioning module is used for multimodal feature extraction, extracting the XPath / CSS path and semantic attributes of the target element; The dynamic feature library module is used for searching and similarity matching in the dynamic feature library; The exception handling module is used to generate matching rules or AI positioning based on the input multimodal feature vector and the global structure of the page and select the priority according to the failure type; The script correction and feedback module is used to replace the invalid positioning statements in the original script with new rules and make adaptive adjustments to the rules in the dynamic feature library; The structured report generation module is used to record complete abnormal data and generate a structured report.

[0019] As a preferred technical solution, the intelligent positioning module also includes an anchor rule definition module, which is used to perform relative anchor positioning, select adjacent high-stability elements as reference points based on page layout stability analysis, and dynamically generate relative paths.

[0020] This invention achieves a closed loop of "detection → matching → self-repair → learning" through a three-stage architecture of multimodal feature extraction, dynamic feature library weight accumulation, and AI anchor rule generation. Specifically, the invention has the following beneficial effects: 1. Automatic repair and learning capabilities for element positioning have been achieved. When the original positioning rules become invalid, the system can automatically generate new rules and verify them. 2. Improved adaptability to dynamic content. Through multimodal feature extraction and relative anchor point positioning, it effectively handles dynamically generated elements and complex UI interaction scenarios. 3. Reduces the need for manual intervention and reduces maintenance costs through AI intelligent rule generation and feedback learning mechanisms; 4. The weight management mechanism improves positioning accuracy, increasing the weight of successful rules and reducing the weight of failed rules, achieving system self-optimization; 5. Provides structured failure reports that provide valuable information for human intervention even when all rule attempts fail. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flowchart of a specific implementation method of an intelligent element positioning method based on AI and a dynamic feature library of the present invention. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] The terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The singular forms "a," "the," and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. "A plurality" generally includes at least two, but does not exclude the inclusion of at least one.

[0024] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0025] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0026] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.

[0027] like Figure 1 FIG. 1 shows a specific embodiment of an intelligent element positioning method based on AI and a dynamic feature library according to the present invention. The intelligent element positioning method based on AI and a dynamic feature library according to this embodiment includes the following steps: S1. Abnormal detection and abnormal data capture: Element positioning operation is performed through an automated script. If the preset timeout period is exceeded, it is determined that the positioning has failed, abnormal data is captured, and a structured snapshot of the DOM tree of the current page is recorded. The triggering condition for abnormal detection is: when the automated script performs an element positioning operation (such as click() or find_element()), if the target element is not found within the preset timeout period (such as 5 seconds), the system determines that the positioning has failed. After the positioning fails, abnormal data is captured and a structured snapshot of the DOM tree of the current page is recorded. The structured snapshot of the DOM tree of the current page includes the parent-child node hierarchy and element attributes. In this embodiment, the abnormal data also includes screenshots and captured context information; after the screenshot is captured, the area where the target element is located is located and the image is cropped using the coordinate information of the failed operation; the captured context information includes the page URL, browser and / or device type, and user operation sequence (such as "login → search → click on the product").

[0028] In this embodiment, the automated script is a web automation test script written based on the Selenium framework, used to simulate user interactions on a web page. A preset timeout of 5 seconds is set (this timeout can be adjusted as needed in practice). This means that if the script fails to locate an element for more than 5 seconds, the system triggers an exception detection mechanism. This mechanism monitors the script execution status and immediately initiates the exception data capture process when a NoSuchElementException or TimeoutException is detected.

[0029] During the exception data capture process, the system calls the browser's built-in API to retrieve the complete DOM tree of the current page and serializes it into JSON format for storage. This structured snapshot of the DOM tree contains information such as the tag names, attributes, and hierarchical relationships of all elements on the page, forming the foundational data for subsequent analysis.

[0030] In addition to recording a structured snapshot of the DOM tree, the system also captures a screen image of the current page, locates the area where the target element is located based on the coordinate information of the failed operation, and crops the image of that area for subsequent image analysis. The system uses Selenium's screenshot_as_png method to obtain a screenshot of the entire page. To improve the accuracy of subsequent analysis, the system uses the coordinate information of the failed operation to locate the area where the target element is located and crops the image. Then, based on the coordinates of the last mouse operation and the page scrolling position, the approximate position of the target element in the screenshot is calculated and the image is cropped. Image cropping uses an intelligent area recognition algorithm rather than a simple fixed-size cropping. The algorithm steps are as follows: 1. Initialize a 200×200 pixel cropping region centered on the target coordinates. 2. Use edge detection algorithms to identify element boundaries within the region; 3. Expand the cropping region until it completely encompasses the target element and its directly related visual context; 4. If the recognized element area is too large (over 500×500 pixels), the original cropped area remains unchanged.

[0031] After cropping is completed, the system will pre-process the image, including adjusting brightness and contrast, removing noise, marking target areas, etc., for subsequent image analysis.

[0032] At the same time, the system also captures contextual information, including page URL, browser type and version, device type, and user operation sequence. The page URL includes not only the full URL of the current page, but also the parsed results of the URL, such as the domain name, path, query parameters, etc.; browser information includes browser type, version number, rendering engine version, etc.; device type information includes operating system, device model, screen resolution, etc. The user operation sequence is a timestamp-ordered list of operations that records all interactive operations performed by the user in the current session. Each operation record contains the following information: 1. Timestamp: the exact time the operation was performed; 2. Operation type: such as click, input, scroll, hover, etc.; 3. Target element: a brief description of the target element of the operation; 4. Operation parameters: such as the text content entered, scrolling distance, etc.; 5. Operation result: whether the operation was completed successfully.

[0033] The page URL is obtained through driver.current_url; the browser type and version are obtained through the driver.capabilities property; the device type is obtained by parsing the User-Agent string; and the user operation sequence is an ordered list formed by recording all interactive operations (such as clicks, input, scrolling, etc.) during the script execution and their timestamps. This detailed contextual information is crucial for understanding the environmental context of element positioning failures and helps the system perform more accurate analysis and repairs. For example, by analyzing the user operation sequence, the system can identify key operations that may cause page status changes; through browser and device information, the system can identify compatibility issues that may be related to specific platforms.

[0034] S2. Multimodal feature extraction: Perform structured feature extraction on the captured abnormal data, parse the DOM tree, and extract the XPath / CSS path and semantic attributes of the target element. In this embodiment, the XPath / CSS path of the target element is extracted by generating absolute and relative paths based on the DOM hierarchy; the semantic attributes of the target element are extracted, including the element text content, id / class attributes, and neighboring elements (such as the placeholder text of the input box on the left). For example, a button element may extract {"text": "Submit", "class": "btn-primary", "neighbor_text": "Username input box"}.

[0035] In this embodiment, in step S2, the multimodal feature extraction step, the system first performs a deep analysis of the captured DOM tree to construct a hierarchical relationship diagram of the elements. The analysis uses a depth-first search algorithm, traversing the entire DOM tree from the root node, assigning a unique identifier to each node and recording its parent-child relationship.

[0036] For the target element, the system generates absolute and relative paths based on the DOM hierarchy. The absolute path is the complete path from the HTML root node to the target element, providing the element's exact location in the DOM tree. The absolute path types generated by the system include: 1. Full index path: such as " / html / body / div[6] / section / div[7] / button[8]", which includes the index of each node in the same level; 2. Tag attribute mixed path: such as " / html / body / div[@id='main'] / section / div[@class='controls'] / button", which combines the tag name and key attributes; 3. Full CSS selector: such as "html>body>div:nth-child(2)>section>div:first-child>button:nth-child(3)"; A relative path is a partial path from an ancestor node with a unique identifier to the target element, which has better adaptability when part of the page structure changes.

[0037] The following relative paths are generated by the system: 1. ID anchor-relative paths: such as " / / div[@id='main-container'] / / button[@class='submit']", starting from the nearest ancestor element with an ID; 2. Feature node-relative paths: such as " / / form[contains(@action,'login')] / / input[@type='password']", starting from the root node with a significant feature; and 3. CSS selector-relative paths: such as "main-container.form-groupbutton.submit". Path generation uses a bottom-up analysis approach, first determining the characteristics of the target element itself, then analyzing its ancestor nodes step by step, selecting the most recognizable representation for each level.

[0038] The recognition score is based on the following factors: 1. Whether the node has a unique ID (highest identification); 2. Whether the node has a unique name or other key attributes; 3. Whether the node's label name and class name combination is unique among nodes at the same level; 4. The index position of the node among its peers.

[0039] For each level, the system calculates the recognition scores of different representations and selects the one with the highest score. For example, if a div element has a unique ID, the system will prefer "div[@id='unique-id']" over "div[6]".

[0040] The system also analyzes the overall structural characteristics of the page to identify possible dynamically generated patterns. For example, if a page contains a large number of ID naming patterns such as "item-1" and "item-2," the system will avoid directly using these potentially dynamically changing IDs and instead look for more stable features or use pattern matching expressions such as "div[starts-with(@id,'item-')]".

[0041] By generating a variety of absolute and relative paths, the system significantly improves the success rate of element location. In practice, the system dynamically prioritizes different path types based on page characteristics and historical success rates, selecting the most appropriate location strategy for the scenario. The system generates paths in both XPath and CSS selector formats to increase flexibility in subsequent location operations.

[0042] The XPath generation algorithm analyzes the hierarchical relationships of elements and attempts to generate different types of XPath expressions using various strategies: 1. Index-based XPath: such as " / html / body / div[6] / button[7]"; 2. Attribute-based XPath: such as " / / button[@id='submit-btn']"; 3. XPath based on text content: such as " / / button[text()='Submit']"; 4. XPath based on partial attribute matching: such as " / / button[contains(@class,'btn-primary')]".

[0043] CSS selector generation also uses multiple strategies: 1. ID selector: such as "submit-btn"; 2. Class selector: such as ".btn-primary"; 3. Attribute selector: such as "button[type='submit']"; 4. Combined selector: such as "main-form .form-group button".

[0044] In addition to path information, the system also extracts the semantic attributes of the target element, including its text content, id / class attributes, and neighboring elements. Element text content refers to the visible text within an element, accessed through the element.text attribute; id / class attributes are the element's unique identifier and style class name, accessed through element.get_attribute('id') and element.get_attribute('class'); neighboring elements refer to sibling nodes or parent-child nodes adjacent to the target element in the DOM tree. Information about these elements can be used as reference points for auxiliary positioning.

[0045] The following strategies are used to extract neighboring elements: 1. Extract the parent element information of the target element; 2. Extract the previous and next sibling elements of the target element; 3. Extract the target element's child elements (if any).

[0046] For each adjacent element, the system also extracts information such as its tag name, attributes, and text content to construct a local context description. These semantic attributes together form the characteristic fingerprint of the target element, which facilitates fuzzy matching and intelligent positioning when the page structure changes.

[0047] S3. Dynamic feature library retrieval and matching: First, perform rule retrieval in the dynamic feature library for the multimodal feature vector of the current element, and calculate the DOM structure similarity. If there are rules with a total similarity ≥ 85% in the dynamic feature library, all rules with a total similarity ≥ 85% are sorted from high to low by weight, and the rule with the highest similarity is selected as the basis for correction, and step S5 is performed; if there are no rules with a total similarity ≥ 85% in the dynamic feature library, step S4 is performed. In this embodiment, "performing rule retrieval in the dynamic feature library for the multimodal feature vector of the current element and calculating the DOM structure similarity" is specifically: traversing the rules of the same platform (such as Web / iOS) in the dynamic feature library, and calculating the weighted similarity: total similarity = 0.5*DOM structure similarity + 0.5*context similarity; DOM structure similarity is the path difference calculated based on the Tree Edit Distance algorithm. Specifically: In this embodiment, in step S3, the dynamic feature library search and matching step, the system first converts the multimodal features extracted in step S2 into a feature vector. The feature vector contains the following dimensions: 1. DOM structural characteristics: including element tag name, hierarchy depth, number of child elements, etc.; 2. Attribute characteristics: including attribute values ​​such as id, class, name, and type; 3. Text features: the text content of the element, its length, and language features; 4. Positional characteristics: the relative position and size of the element on the page; 5. Contextual features: including key information of parent elements and sibling elements.

[0048] After the feature vector is constructed, the system begins searching for rules in the dynamic feature library. The dynamic feature library is a structured database that stores historically successful positioning rules, their corresponding feature vectors, weights, and usage statistics. The search process first selects a subset of rules that match the current page platform (such as Web, Android, and iOS), then calculates similarity within this subset.

[0049] The system traverses the rules of the same platform in the dynamic feature library and calculates the weighted similarity. The total similarity is calculated as: total similarity = 0.5*DOM structure similarity + 0.5*context similarity.

[0050] DOM structural similarity is calculated based on the tree edit distance algorithm, which calculates path differences. The tree edit distance refers to the minimum number of edit operations (insertion, deletion, and replacement of nodes) required to transform one tree into another. This embodiment uses the improved Zhang-Shasha algorithm to achieve this calculation. This algorithm optimizes the traditional tree edit distance algorithm to improve computational efficiency.

[0051] Specifically, this embodiment parses the XPath path of the target element and the XPath path of the rule in the feature library into tree structures, where each node contains the tag name, attributes, and index information. The edit distance between the two trees is then calculated, and the cost of the edit operation is defined as follows: 1. Inserting a node: cost 1; 2. Deleting a node: cost is 1; 3. Replace node: The cost is dynamically calculated based on node similarity, ranging from 0 to 1; - If the tag names are the same: the base cost is 0; - If the tag name is different: the base cost is 0.6; - Attribute Difference: Each different key attribute increases the cost by 0.1; - Index difference: The cost increases by 0.05 for every 1-digit difference in the index, and the maximum increase is 0.2; After the edit distance is calculated, the DOM structure similarity is calculated using the following formula: DOM structure similarity = 1 - (edit distance / max(number of tree 1 nodes, number of tree 2 nodes)).

[0052] Contextual similarity takes into account factors such as element attributes, text content, and neighboring elements, and is calculated using weighted cosine similarity. The system converts the various features of an element into a feature vector and then calculates the cosine similarity between the feature vectors of the current element and the feature vectors of the rule elements in the feature library. The feature vector contains the following dimensions: 1. Element tag name (one-hot encoding); 2. Element id / class attributes (TF-IDF vector); 3. Element text content (bag-of-words vector); 4. Other attributes of the element (key-value pair vector); 5. Neighboring element features (compressed representation vector); 6. Page URL features (encoded vector of domain name and path); 7. User operation sequence features (encoding vectors of operation types and orders).

[0053] For different types of features, the system assigns different weights: 1. The weight of the id attribute is 0.3; 2. The class attribute weight is 0.2; 3. The text content weight is 0.3; 4. The weight of other attributes is 0.1; 5. The neighboring element feature weight is 0.1.

[0054] The calculation formula for context similarity is: Contextual similarity = Σ(feature i weight * feature i cosine similarity) / Σfeature i weight In order to improve retrieval efficiency, the system adopts a multi-level index structure: 1. First-level index: platform type; 2. Second level index: page URL patterns; 3. Third-level index: element tag name; 4. Fourth-level index: key attribute hash.

[0055] This multi-level index structure significantly reduces the number of candidate rules that require similarity calculations, improving retrieval speed. In practice, for large feature libraries containing hundreds of thousands of rules, the system can complete retrieval and similarity calculations in milliseconds.

[0056] After the calculation is complete, the system checks whether there are any rules with a total similarity ≥ 85%. If so, it sorts all rules with a total similarity ≥ 85% from highest to lowest weight. The weight is a dynamically adjusted value that reflects the rule's historical success rate and applicability. After the sorting is complete, the system selects the rule with the highest similarity as the basis for revision and proceeds to step S5.

[0057] If there is no rule with a total similarity ≥ 85% in the dynamic feature library, it means that the characteristics of the current element are significantly different from the historical rules, and new positioning rules need to be generated through AI intelligence, so the system enters step S4.

[0058] S4, AI intelligent rule generation: Based on the multimodal feature vector and the global structure of the page, a classification model is used to generate multiple candidate positioning rules, which are sorted by weight. The candidate positioning rules are tried in sequence. If the positioning is successful, the attempt is stopped and step S5 is executed. In this embodiment, when generating AI intelligent rules, the model input is the multimodal feature vector and the global structure of the page (such as the page layout skeleton diagram); then the top-3 candidate positioning rules are generated through the LightGBM classification model. For example: Relative 1. XPath: / / div[contains(@class, "header")] / / button[text()="Submit"].

[0059] 2. Image coordinates: (x=320, y=480) combined with OCR text verification.

[0060] 3.CSS selector: .form-container>.btn:last-child.

[0061] Then, rule validation and optimization are performed: candidate rules are tried one by one, and if a successful match is found, the execution stops immediately and the execution results are recorded. Success criteria: the element is interactive (clickable / input-enabled) and its attributes meet expectations (e.g., displayed == true).

[0062] In this embodiment, in step S4, the AI ​​intelligent rule generation step, the system uses a pre-trained machine learning model to generate positioning rules suitable for the current scenario based on multimodal feature vectors and the global structure of the page. The system uses the LightGBM classification model to generate the top-3 candidate positioning rules.

[0063] LightGBM is an efficient machine learning algorithm based on gradient boosting decision trees, with the advantages of fast training speed, low memory usage, and high accuracy. In this system, the LightGBM model receives the following input features: 1. Basic characteristics of the element, such as tag name, attributes, and text content; 2. The element's position and hierarchy in the DOM tree; 3. Global structural characteristics of the page, such as the depth, width, and node density of the DOM tree; 4. Pattern statistics of historical successful positioning rules.

[0064] The output of the model is a success probability score for different positioning strategies, including: 1. The success probability of the ID positioning strategy; 2. The success probability of the CSS selector positioning strategy; 3. The success probability of the XPath location strategy; 4. The success rate of the text content positioning strategy; 5. The success probability of relative positioning strategy.

[0065] Based on these probability scores, the system generates specific positioning rules for each positioning strategy. For example, if the ID positioning strategy has the highest success probability, the system will generate a positioning rule based on the element ID; if the relative positioning strategy has a high success probability, the system will generate a relative positioning rule based on nearby stable elements.

[0066] For each targeting strategy, the system generates multiple candidate rules and calculates the weight of the rules based on the following factors: 1. Success probability score of the positioning strategy; 2. Specificity of the rule (more specific rules have higher weights); 3. Robustness of the rules (resistance to changes in page structure); 4. Statistics on the success rate of historical similarity rules.

[0067] After the weight calculation is completed, the system sorts all candidate positioning rules by weight from high to low and selects the top 3 rules as the final candidates. These candidate rules usually include different types of positioning strategies to increase the possibility of successful positioning.

[0068] The system tries these candidate positioning rules in turn. For each rule, the system performs the positioning operation on the current page and verifies whether the positioning result meets expectations. The judgment conditions for successful positioning are: the element is interactive and the attributes meet expectations. In this embodiment, the system uses a strict multi-verification mechanism to determine successful positioning, ensuring that the positioned element not only exists on the page, but also can interact normally and meets the expected attribute characteristics. Specifically, the system checks the following points: 1. Whether the located element exists and is visible; 2. Whether the located element is interactive (e.g., clickable, input-enabled, etc.); 3. Whether the key attributes of the located element match the target element.

[0069] Element existence verification is the most basic judgment. The system will check the following points: 1. Whether the element exists in the DOM tree; 2. Whether the element is visible in the viewport (not obscured by other elements); 3. Whether the element has visible size (width and height greater than 0); 4. Whether the element is not hidden by CSS (display:none or visibility:hidden).

[0070] Element interactivity verification determines whether an element can respond to user operations. The system performs different verifications based on the element type: 1. For clickable elements such as buttons and links: verify whether the element can receive click events; 2. For form elements such as input boxes and text areas: verify whether the element can receive input; 3. For select elements such as drop-down menus and checkboxes: verify whether the element can change its selected state; 4. For draggable elements: Verify that the element can respond to drag operations.

[0071] Interactivity verification uses JavaScript injection. The system injects test scripts into the page to simulate user interactions and capture element responses. For example, for a button element, the system injects the following script: ```javascript function checkClickable(element) { / / Check if the element is disabled if (element.disabled || element.getAttribute('disabled') !== null){ return false; } / / Check if the element is covered var rect = element.getBoundingClientRect(); var centerX = rect.left + rect.width / 2; var centerY = rect.top + rect.height / 2; var elementAtPoint = document.elementFromPoint(centerX, centerY); / / Check if the element at the mouse pointer position is the target element or its child element return element === elementAtPoint || element.contains(elementAtPoint); } ``` Element attribute compliance verification determines whether the located element matches the expected target. The system compares the following key attributes: 1. Tag name: whether the element's HTML tag meets expectations; 2. Element type: For input elements, whether its type attribute meets expectations; 3. Key attributes: whether the element's key attributes such as id, name, and class match expectations; 4. Text content: Does the element’s visible text match expectations? 5. Element Role: Check whether the ARIA role of the element matches its intended function.

[0072] Attribute conformance verification uses a fuzzy matching method, which allows for changes in some attributes. The system assigns different weights to different attributes and calculates the attribute matching degree: 1. id attribute weight: 0.3; 2. Tag name weight: 0.2; 3. Text content weight: 0.3; 4. Class attribute weight: 0.1; 5. Other attribute weight: 0.1.

[0073] The calculation formula for attribute matching is: Attribute matching degree = Σ(attribute i weight * attribute i matching degree) The matching degree of attribute i is 0 (no match) or 1 (match). For text content, the system uses a string similarity algorithm to calculate the matching degree, allowing for partial text changes.

[0074] The system only successfully locates an element if it passes both existence and interactivity verification, and its attribute matching degree is ≥0.8. This rigorous multi-verification mechanism ensures the accuracy and reliability of the location results, preventing automated script failures caused by incorrect location.

[0075] If a rule successfully locates an element that meets the expectations, the system immediately stops trying other rules and uses the successful rule as a basis for correction, proceeding to step S5. If all candidate rules fail to locate the element successfully, the system readjusts the feature weights and generates new candidate rules to try until a successful rule is found or the number of attempts reaches the upper limit.

[0076] S5, Script Correction and Real-time Feedback: Replace the invalid location statement in the original script with a new rule. If the correction is successful, the new rule is added to the dynamic feature library. If the correction fails, return to step S3 or S4 and select another rule to correct it until the correction is successful. If all rules fail, execute step S6. In this implementation, step S5 first performs a dynamic script replacement, replacing the invalid location statement in the original script with the newly generated rule (such as replacing the old XPath with a relative path). For example: # Original statement (invalid) driver.find_element(By.XPATH, ' / / button[@id="submit_btn"]') # After correction driver.find_element(By.XPATH, ' / / div[@class="modal-footer"] / button[1]') Then a feedback learning mechanism is performed. If the correction is successful and the new rule is added to the dynamic feature library, the initial weight = 1. Each time the same rule in the dynamic feature library is successfully used, the weight increases by α (which can be set as needed, for example, α = 1.0); in step S5, when the correction fails, the weight decreases by β (which can be set as needed, for example, β = 0.5). When the weight of a rule in the dynamic feature library is less than the threshold (which can be set as needed, for example, 0.2), the rule is set to a frozen state and no longer participates in matching.

[0077] In this embodiment, in step S5, the script modification and real-time feedback step, the system first analyzes the failure location statement in the original automation script to determine its location and context within the script. The failure location statement is typically an element selector expression, such as driver.find_element(By.XPATH, " / / button[@id='submit']") .

[0078] The system generates a new targeting statement based on the successful targeting rule obtained in step S3 or S4. The new targeting statement maintains the same API call format as the original statement, but uses a new selector expression. For example, if the new targeting rule uses the CSS selector ".btn-primary[type='submit']", the generated new targeting statement is driver.find_element(By.CSS_SELECTOR, ".btn-primary[type='submit']")

[0079] After generating the new targeting statement, the system will replace the invalid targeting statement in the copy of the original script and execute the modified script fragment in an isolated environment to verify whether the new targeting rule can successfully locate the target element. The verification process includes the following steps: 1. Execute the new positioning statement in the current page state; 2. Check whether the positioning results meet expectations (the element exists, is visible, and is interactive); 3. Perform the intended operation (such as clicking or typing) and verify that the operation was completed successfully.

[0080] If the verification is successful, the correction is effective and the system will add the new rule to the dynamic signature library. During the addition process, the system will assign the new rule an initial weight of 1 and record the detailed information of the rule, including: 1. Rule type (XPath, CSS selector, etc.); 2. Regular expressions; 3. The feature vector of the target element; 4. Page context information; 5. Creation time and last use time.

[0081] Each time a rule in the dynamic feature library is successfully used, its weight increases by α (default α=0.1). This dynamic adjustment mechanism allows frequently successful rules to receive higher weights, giving them higher priority in future matching processes.

[0082] If verification fails, it means the currently selected rule cannot successfully modify the script. The system will return to step S3 or S4 based on the reason for failure. If the failure is related to the rule itself (such as a selector syntax error), the system will return to step S4 to generate a new candidate rule. If the failure is related to the page status (such as a page structure change), the system will return to step S3 and re-match the feature library.

[0083] If the correction fails, the system will reduce the weight of the corresponding rule by β (default β = 0.5). When the weight of a rule in the dynamic signature library is less than the threshold (default threshold = 0.2), the system will freeze the rule and no longer participate in the matching to avoid repeated attempts of invalid rules.

[0084] In this embodiment, the system implements an adaptive rule weight adjustment mechanism, which enables the dynamic feature library to be continuously optimized and evolved, thereby improving the accuracy and adaptability of the positioning rules.

[0085] When the system successfully corrects an element location issue and adds a new rule to the dynamic feature library, it assigns the new rule an initial weight of 1. This initial value represents the basic credibility of the rule, neither too high nor too low, giving the new rule a fair starting point.

[0086] Each time a rule in the dynamic signature library is successfully used, its weight increases by α. By default, α = 1.0, which means that each time a rule is successfully applied, its weight increases by 1. This incremental weight adjustment ensures that frequently successful rules gradually gain higher weights and thus receive higher priority in future matches.

[0087] Weight increases are not unlimited. You can set a weight cap based on actual needs, such as 5.0. When a rule's weight reaches the cap, it will no longer increase, but the rule's last used time will be updated to reflect its continued active status.

[0088] When the correction fails, the weight is reduced by β. By default, β=0.5, which means that each time the rule fails to be applied, its weight is reduced by 0.5.

[0089] When the weight of a rule in the dynamic feature library falls below the threshold, the rule is frozen and no longer participates in matching. The default threshold is 0.3. When the rule weight falls below this value, the system considers the rule no longer applicable to the current page structure and changes its status from "active" to "frozen."

[0090] Frozen rules are not immediately deleted from the signature database. Instead, they are retained for a period of time (adjustable based on actual conditions, with a default of 30 days) during which they are not used for matching. This design takes into account the possibility that a page may be temporarily modified and then restored to its original state, thus preventing the permanent loss of valuable rules.

[0091] This embodiment also implements a rule "unfreezing" mechanism. When a significant change occurs in the page structure, causing the success rate of active rules to drop significantly, the system will attempt to unfreeze some historical rules, giving them a chance to participate in matching again. The conditions for unfreezing are: 1. The current page URL is highly similar to the URL when the rule was created; 2. The DOM structure of the page has changed significantly compared to when the rule was created; 3. The success rate of active rules falls below the preset threshold (30% by default).

[0092] In addition to the basic weight adjustment mechanism, the system also implements the following advanced features: 1. Time decay: The weight of a rule that has not been used for a long time will slowly decay. The decay formula is: new weight = old weight * (0.9 ^ (number of days not used / 30)); 2. Context-sensitive: The success rate of the same rule in different contexts (such as different URL patterns) may vary. The system records the performance of the rule in various contexts and adjusts its effective weight in specific contexts. 3. Batch Adjustment: When a major update to the page structure occurs, the system will detect that multiple rules have become invalid at the same time. At this time, the batch adjustment mechanism will be triggered to temporarily reduce the penalty coefficients of all related rules to prevent valuable rules from being frozen prematurely. 4. Rule clustering: The system clusters rules with similar functions. When a rule fails, it checks the performance of other rules in the same cluster to determine whether it is a local problem or a change in the overall page structure.

[0093] Through this complex and sophisticated weight adjustment mechanism, the dynamic feature library can continuously optimize itself, retain high-quality positioning rules, eliminate invalid rules, and adapt to dynamic changes in page structure to provide continuous and effective element positioning services.

[0094] The system will continue to try different rules to make corrections until a successful correction solution is found. If all possible rules fail, the system will execute step S6 to generate a structured report.

[0095] S6. Generate structured report: Record complete exception data and generate a structured report, marking possible reasons for failure. If all rule attempts fail, the system executes: Logs the full error context (including DOM, screenshots, logs).

[0096] Generate a structured report (JSON / HTML format) with possible failure reasons noted: { "error_code": "E404", "suspected_cause": "Dynamic element ID changed + visual features not obvious", "suggested_action": "Manual intervention to define static anchor points" } In this implementation, in step S6, generating a structured report, the system integrates the information collected in the previous steps to generate a detailed anomaly analysis report. The report uses JSON format and contains the following main parts: 1. Basic information section: - Time when the anomaly occurred; - Page URL; - Browser type and version; - device type; - Script execution environment.

[0097] 2. Exception details section: - Exception type (such as NoSuchElementException, TimeoutException, etc.); - Exception stack information; - Invalid positioning statement; - The type of action expected (click, enter, etc.).

[0098] 3. Page analysis section: - DOM tree structure summary; - target element feature vector; - Page screenshot path; - Element area screenshot path.

[0099] 4. Repair attempt part: - List of attempted targeting rules; - Verification results of each rule; - The final repair solution selected, if any.

[0100] 5. Failure cause analysis: - categories of possible failure causes; - A detailed description of the reasons for the failure; - Confidence score.

[0101] The system automatically identifies possible failure causes by analyzing the collected data. The failure cause analysis is based on the following aspects: 1. The element exists but is invisible (for example, it is blocked by other elements or is collapsed); 2. Whether the element has been removed from the page; 3. Whether the attributes of the element have changed; 4. Whether there are significant changes in the page structure; 5. Are there any issues with dynamic loading or asynchronous updates? 6. Check whether there are special structures such as iframe or shadow DOM.

[0102] For each possible cause of failure, the system assigns a confidence score (0-100%) and provides detailed supporting evidence. For example, if the system determines that a change in an element's properties is the most likely cause, a comparison of the original and current properties is listed, along with the time of the change (if traceable).

[0103] Once a structured report is generated, the system saves it to a designated location and optionally sends notifications to relevant personnel. The information in the report not only helps developers understand and resolve current issues, but also provides valuable data for subsequent optimization of automated test scripts and improvement of AI models.

[0104] This embodiment also provides an intelligent element positioning system based on AI and dynamic feature library to implement the above-mentioned intelligent element positioning method based on AI and dynamic feature library, including an anomaly detection and anomaly data capture module, an intelligent positioning module, a dynamic feature library module, an anomaly handling module, a script correction and feedback module, and a structured report generation module; wherein: the anomaly detection and anomaly data capture module is used to detect positioning anomalies and capture anomaly data; the intelligent positioning module is used to extract multimodal features and extract the XPath / CSS path and semantic attributes of the target element; the dynamic feature library module is used to perform retrieval and similarity matching in the dynamic feature library; the anomaly handling module is used to generate matching rules or AI positioning according to the failure type based on the input multimodal feature vector and the global structure of the page; the script correction and feedback module is used to replace the failed positioning statement in the original script with a new rule, and to make adaptive adjustments to the rules in the dynamic feature library; the structured report generation module is used to record complete anomaly data and generate a structured report.

[0105] In this embodiment, the exception detection and exception data capture module is the system's entry point component, responsible for monitoring the execution status of automated scripts and promptly identifying element positioning anomalies. This module adopts the observer pattern design and registers an exception listener with the script execution engine. When it catches element positioning-related exceptions such as NoSuchElementException and TimeoutException, it immediately triggers the exception handling process.

[0106] The anomaly detection and anomaly data capture module consists of three subcomponents: an anomaly listener, a data collector, and a snapshot generator. The anomaly listener is responsible for intercepting and classifying anomalies; the data collector is responsible for collecting contextual information when an anomaly occurs, including the page URL, browser type, and device information; and the snapshot generator is responsible for creating a structured snapshot of the page DOM tree and saving the page screenshot.

[0107] During the exception data capture process, the module captures screen images and contextual information. After capturing the screenshot, the coordinates of the failed operation are used to locate the target element and crop the image. The cropped area defaults to a 200×200 pixel region centered on the target coordinates, but the system dynamically adjusts the size of the cropped area based on the page zoom ratio and device resolution. The captured contextual information includes the page URL, browser and / or device type, and user operation sequence. The user operation sequence is a timestamped, ordered list of operations that records all user interactions performed in the current session, including information such as the operation type, target element, and operation parameters.

[0108] The intelligent location module is the core component of the system, responsible for multimodal feature extraction from captured anomaly data. This module first parses the DOM tree and then extracts the XPath / CSS path and semantic attributes of the target element. XPath / CSS path extraction generates absolute and relative paths based on the DOM hierarchy. An absolute path begins at the HTML root node and contains the full path to the target element; a relative path is a partial path starting from a root node with a unique identifier.

[0109] The Smart Positioning module also includes an Anchor Rule Definition module, which is used for relative anchor positioning. Based on the page layout stability analysis, it selects nearby highly stable elements as reference points and dynamically generates relative paths. The Anchor Rule Definition module first analyzes the stability of all elements on the page. The stability score is based on the following factors: 1. Does the element have a unique and stable id or name attribute? 2. Frequency of element position changes in historical versions 3. The depth of the element's DOM hierarchy (the shallower it is, the more stable it is) 4. Frequency of content updates of elements After the scoring is completed, the system selects the element closest to the target element and with a stability score ≥ 0.8 as the anchor. After the anchor is determined, the system generates a relative path from the anchor to the target element, such as ". / following-sibling::div[6] / button" or ".. / div[@class='content'] / span". This relative anchor positioning method greatly improves the robustness of element positioning. Even if the page structure changes, as long as the anchor element and the relative structure remain stable, the positioning remains valid.

[0110] When extracting the semantic attributes of a target element, the intelligent positioning module considers the element's text content, id / class attributes, and neighboring elements. Element text content refers to the visible text within an element. The system extracts the complete text and performs natural language processing to identify keywords and semantic features. The id / class attributes are the element's unique identifier and style class name. The system analyzes the naming patterns and semantic meaning of these attributes. Neighboring elements refer to nodes adjacent to the target element in the DOM tree. The system extracts key features from these elements to construct a local context description.

[0111] The dynamic feature library module is the system's knowledge base component, responsible for storing and managing historically successful positioning rules. This module uses a distributed database architecture, supporting high-concurrency read and write and fast retrieval. Each rule record in the feature library contains the following fields: 1. Rule ID: unique identifier 2. Rule type: such as XPath, CSS selector, relative anchor, etc. 3. Regular expressions: specific positioning expressions 4. Feature vector: multimodal features of the target element 5. Applicable platforms: Web, Android, iOS, etc. 6. Weight: reflects the historical success rate of the rule 7. Usage statistics: including number of successes, number of failures, last usage time, etc. 8. Status: Active, Frozen, or Abandoned The dynamic feature library module provides an efficient retrieval interface and supports feature vector-based similarity searches. The retrieval process first selects a subset of rules that match the current platform and then calculates the similarity between the current element's features and the rule features in the library. Similarity is calculated using a weighted approach: total similarity = 0.5 * DOM structure similarity + 0.5 * contextual similarity. DOM structure similarity is calculated based on path differences using the tree edit distance algorithm, while contextual similarity is calculated based on the cosine similarity of feature vectors.

[0112] The exception handling module is the system's decision-making component, responsible for generating matching rules or AI-powered location strategies based on the input multimodal feature vectors and the page's global structure. This module consists of two main subcomponents: a rule matching engine and an AI inference engine.

[0113] The rule matching engine searches the dynamic signature database for rules with a similarity of 85% or higher. If a matching rule is found, the system sorts it by weight from high to low, selecting the one with the highest similarity as the basis for correction.

[0114] The AI ​​inference engine kicks in when a rule match fails, generating the top three candidate targeting rules using the LightGBM classification model. The model inputs include element feature vectors and global page structural features, and outputs success probability scores for different targeting strategies. Based on these scores, the system generates specific targeting rules and ranks them by weight.

[0115] The Script Correction and Feedback Module is the execution component of the system, responsible for applying the new rules to the original script and verifying the effectiveness of the corrections. This module first analyzes the invalid location statements in the original script and then generates replacement statements based on the selected new rules. Once the replacement is complete, the system executes the modified script fragment in an isolated environment to verify that the new rules successfully locate the target element.

[0116] If verification succeeds, the system adds the new rule to the dynamic signature database with an initial weight of 1. Each successful use of the same rule in the dynamic signature database increases its weight by α (default α = 0.1); if the correction fails, the weight decreases by β (default β = 0.2). If the weight of a rule in the dynamic signature database falls below the threshold (default threshold = 0.3), the system freezes the rule and it no longer participates in matching.

[0117] The structured report generation module is the system's output component, responsible for recording complete exception data and generating structured reports. These reports, formatted in JSON, include basic exception information, details, page analysis, remediation attempts, and failure cause analysis. The system comprehensively analyzes the collected data, automatically annotating possible failure causes and providing a confidence score and supporting evidence for each cause.

[0118] The present invention discloses an intelligent element positioning method and system based on AI and dynamic feature library, belonging to the field of automated testing technology. The method includes: anomaly detection and abnormal data capture, executing element positioning operations through automated scripts, and recording a page DOM tree structured snapshot when positioning fails; multimodal feature extraction, parsing the DOM tree, extracting the XPath / CSS path and semantic attributes of the target element; dynamic feature library retrieval and matching, calculating DOM structure similarity, and using high-similarity rules as a basis for correction if they exist; AI intelligent rule generation, generating candidate positioning rules through a classification model; script correction and real-time feedback, replacing failed positioning statements and updating the dynamic feature library; generating a structured report, and recording abnormal data. The system includes an anomaly detection and abnormal data capture module, an intelligent positioning module, a dynamic feature library module, an exception handling module, a script correction and feedback module, and a structured report generation module. The present invention realizes intelligent correction of element positioning during automated testing by combining AI with a dynamic feature library, thereby improving the stability and efficiency of automated testing.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. That is, any equivalent changes and modifications made according to the content of the patent application of the present invention should fall within the technical scope of the present invention.

Claims

1. An intelligent element positioning method based on AI and a dynamic feature library, characterized by: The following steps are involved: S1. Anomaly detection and abnormal data capture: Element location operations are performed through automated scripts. If the preset timeout period is exceeded, the location is considered to have failed, abnormal data is captured, and a structured snapshot of the current page DOM tree is recorded. S2. Multimodal feature extraction: Perform structured feature extraction on the captured abnormal data, parse the DOM tree, and extract the XPath / CSS path and semantic attributes of the target element; S3. Dynamic feature library search and matching: First, a rule search is performed in the dynamic feature library for the multimodal feature vector of the current element, and the DOM structure similarity is calculated. If there are rules with a total similarity ≥ 85% in the dynamic feature library, all rules with a total similarity ≥ 85% are sorted from high to low by weight, and the rule with the highest similarity is selected as the basis for correction, and step S5 is performed; if there are no rules with a total similarity ≥ 85% in the dynamic feature library, step S4 is performed; S4, AI intelligent rule generation: Based on the multimodal feature vector and the global structure of the page, a classification model is used to generate multiple candidate positioning rules, which are sorted by weight. The candidate positioning rules are tried in sequence. If positioning is successful, the attempt is stopped and step S5 is executed; S5, script correction and real-time feedback: Replace the invalid positioning statement in the original script with a new rule. If the correction is successful, the new rule is added to the dynamic feature library. If the correction fails, return to step S3 or S4 and select other rules to correct until the correction is successful. If all rules fail, execute step S6; S6. Generate structured report: Record complete abnormal data and generate a structured report, marking possible reasons for failure.

2. The intelligent element positioning method based on AI and dynamic feature library according to claim 1, characterized in that: The abnormal data in step S1 also includes capturing a screen image and capturing context information; after capturing the screen image, the area where the target element is located is located using the coordinate information of the failed operation and the image is cropped; the captured context information includes the page URL, browser and / or device type, and user operation sequence.

3. The intelligent element positioning method based on AI and dynamic feature library according to claim 1, characterized in that: In step S2, the XPath / CSS path of the target element is extracted to generate an absolute path and a relative path based on the DOM hierarchy.

4. The intelligent element positioning method based on AI and dynamic feature library according to claim 1, characterized in that: The semantic attributes of the target element extracted in step S2 include the element text content, id / class attributes, and neighboring elements.

5. The intelligent element positioning method based on AI and dynamic feature library according to claim 1, characterized in that: In step S3, "searching for rules in the dynamic feature library for the multimodal feature vector of the current element and calculating DOM structure similarity" specifically involves traversing the rules of the same platform in the dynamic feature library and calculating weighted similarity: total similarity = 0.5 * DOM structure similarity + 0.5 * context similarity; DOM structure similarity is calculated based on path differences using the tree edit distance algorithm.

6. The intelligent element positioning method based on AI and dynamic feature library according to claim 1, characterized in that: In step S4, "generating multiple candidate positioning rules through the classification model" is specifically: generating Top-3 candidate positioning rules through the LightGBM classification model.

7. The intelligent element positioning method based on AI and dynamic feature library according to claim 1, characterized in that: The conditions for successful positioning in step S4 are: the elements are interactive and the attributes meet expectations.

8. The intelligent element positioning method based on AI and dynamic feature library according to claim 1, characterized in that: In step S5, if the correction is successful and the new rule is added to the dynamic feature library, the initial weight = 1. Each time the same rule in the dynamic feature library is successfully used, the weight increases by α. In step S5, when the correction fails, the weight decreases by β. When the weight of a rule in the dynamic feature library is less than the threshold, the rule is set to a frozen state and no longer participates in matching.

9. An intelligent element positioning system based on AI and a dynamic feature library for implementing the method according to any one of claims 1 to 8, characterized in that: It includes anomaly detection and abnormal data capture module, intelligent positioning module, dynamic feature library module, exception handling module, script correction and feedback module, and structured report generation module; among them: The anomaly detection and abnormal data capture module is used to detect positioning anomalies and capture abnormal data; The intelligent positioning module is used for multimodal feature extraction, extracting the XPath / CSS path and semantic attributes of the target element; The dynamic feature library module is used for searching and similarity matching in the dynamic feature library; The exception handling module is used to generate matching rules or AI positioning based on the input multimodal feature vector and the global structure of the page and select the priority according to the failure type; The script correction and feedback module is used to replace the invalid positioning statements in the original script with new rules and make adaptive adjustments to the rules in the dynamic feature library; The structured report generation module is used to record complete abnormal data and generate a structured report.

10. The intelligent element positioning system based on AI and dynamic feature library according to claim 9, characterized in that: The intelligent positioning module also includes an anchor point rule definition module, which is used to perform relative anchor point positioning, select adjacent high-stability elements as reference points based on page layout stability analysis, and dynamically generate relative paths.

Images