Reinforcement learning-based intelligent browser adaptive operation method, system, device and medium

Abstract

The application discloses an intelligent browser adaptive operation method and system based on reinforcement learning, equipment and medium, relates to the technical field of intelligent browser operation, and comprises the following steps: acquiring a business process operation sequence, establishing a reverse deduction relationship from an operation result to an operation type through a multi-modal large language model, and generating an operation causal model; the operation causal model is used to autonomously generate a virtual operation sequence for enhancement training during non-task execution; the page position coordinates corresponding to the successful operation are converted into a page element topological relationship and stored in a skill library; when a task is executed, the operation position is reconstructed according to the page element topological relationship, and the execution action is generated in combination with the operation causal model; and the current operation result is fed back to the operation sequence for subsequent iterative training, so that continuous learning and dynamic adaptation of the browser operation process are realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent browser operation technology, and in particular to an intelligent browser adaptive operation method, system, device and medium based on reinforcement learning. Background Technology

[0002] As enterprise business systems evolve towards web-based and cloud-based architectures, browsers have become a crucial operational platform for business processing, data entry, workflow approval, and maintenance management. In practical applications, users frequently need to perform repetitive operations such as page clicks, element positioning, form filling, and status confirmation. Therefore, browser automation technology is widely used in business process execution scenarios.

[0003] Existing browser automation solutions typically rely on fixed scripts, XPath paths, page element IDs, or coordinate positioning to automate operations. These solutions are highly dependent on the page structure. When the webpage layout, element hierarchy, component structure, or interaction logic changes, problems such as element recognition failure, operation position misalignment, and process interruption can easily occur, resulting in poor stability of automated tasks and difficulty in adapting to dynamic webpage environments.

[0004] To improve automation capabilities, some existing technologies have begun to incorporate machine learning or reinforcement learning methods. By recording users' historical operation trajectories, they train operation models, enabling the system to automatically mimic certain operational behaviors. Most existing solutions rely solely on supervised learning based on historical samples, lacking the ability to deeply model the relationship between "operation results" and "operation behaviors." Consequently, they cannot autonomously derive new execution strategies after page changes or task flow adjustments, resulting in limited model generalization capabilities.

[0005] Traditional reinforcement learning methods typically rely on interactive training in real web environments, requiring a large number of manual operation samples and online trial-and-error processes. This not only results in high training costs but also may lead to problems such as misoperation, data anomalies, or business process interruptions. Furthermore, current technologies lack mechanisms for autonomous extrapolation and virtual training using historical success experiences, making it difficult to achieve continuous learning and long-term optimization of browser operation models.

[0006] Furthermore, most existing element positioning technologies use fixed page coordinates or static tag structures for identification. When the page undergoes dynamic rendering, partial refresh, or cross-version upgrade, the original positioning relationship is prone to failure, resulting in insufficient adaptability of the automated system.

[0007] Therefore, how to provide a smart browser operation method has become an urgent problem to be solved in current technology. Summary of the Invention

[0008] The purpose of this invention is to provide a method, system, device, and medium for adaptive operation of an intelligent browser based on reinforcement learning, aiming to solve at least one of the technical problems existing in the prior art.

[0009] The technical solution of this invention is: an intelligent browser adaptive operation method based on reinforcement learning, comprising:

[0010] Obtain the operation sequence of the target business process, wherein the operation sequence includes operation type, page position coordinates, and operation result;

[0011] By training the operation sequence in reverse through a multimodal large language model, a reverse inference relationship from operation result to operation type is established, and an operation causal model is generated.

[0012] During non-task execution, the operational causal model autonomously generates virtual operation sequences and predicts virtual operation results. Virtual operation sequences with successful virtual operation results are used as enhancement samples to iteratively train the operational causal model.

[0013] Convert the page position coordinates of successful operations in the operation sequence into page element topological relationships and store them in the skill library in association with the operation type;

[0014] Receive an operation task containing the type of operation to be performed and the current page;

[0015] Read the topological relationship of page elements corresponding to the type of operation to be performed from the skill library, reconstruct the operation position in the current page according to the topological relationship of page elements, and input the type of operation to be performed and the operation position into the operation causal model after iterative training to generate the execution action;

[0016] The execution action is performed and the current operation result is obtained. The current operation result is added to the operation sequence for subsequent iterative training.

[0017] By training the operation sequence in reverse using a multimodal large language model, a reverse inference relationship from operation result to operation type is established, generating an operation causal model including:

[0018] The operation sequence is divided into success group and failure group according to the operation result. The operation type and page position coordinate in the success group and failure group are time-series encoded to generate success trajectory representation and failure trajectory representation.

[0019] Calculate the causal difference features between the success trajectory representation and the failure trajectory representation, and use the causal difference features as a constraint for the reverse generation training of the multimodal large language model.

[0020] Samples from the operation sequence that have successful operation results are input into a multimodal large language model to extract the page state features corresponding to the operation results;

[0021] Based on the constraints, the page state features are decoded in reverse to generate candidate operation types and candidate page position coordinates. Candidate operation types and candidate page position coordinates that satisfy the causal difference features are selected as the reverse derivation output.

[0022] Calculate the prediction error between the back-derivative output and the actual operation type and page position coordinates in the operation sequence. Minimize the prediction error through gradient backpropagation to update the parameter weights of the multimodal large language model. Establish the back-derivative relationship from operation result to operation type. Use the multimodal large language model after establishing the back-derivative relationship as the operation causal model.

[0023] The operational causal model autonomously generates virtual operation sequences and predicts virtual operation results. Virtual operation sequences with successful results are used as boosting samples for iterative training of the operational causal model, including:

[0024] The operation type and page position coordinates are extracted from the operation sequence to construct a perturbation sample pool. Random perturbation is applied to the operation type and page position coordinates in the perturbation sample pool to generate perturbation operation combinations. The perturbation operation combinations are input into the operation causal model for reverse decoding to generate virtual operation sequences.

[0025] The virtual operation sequence is predicted by forward reasoning using an operational causal model. The virtual operation sequence with the result of success is compared and analyzed with the virtual operation sequence with the result of failure to extract the decision boundary features of successful operation. Based on the decision boundary features, the virtual operation sequence with the result of success is labeled to generate enhanced samples.

[0026] The operational causal model is input with augmented samples, and the inverse derivation relationship of the operational causal model is adjusted based on the decision boundary features in the augmented samples. The operational causal model is then iteratively trained.

[0027] Converting the page position coordinates of successful operations in the operation sequence into page element topological relationships and storing them in the skill library in association with the operation type includes:

[0028] The page position coordinates with successful operation results in the operation sequence are sorted according to the time sequence of operation execution, and cluster analysis is performed on the sorted page position coordinates to identify spatially clustered operation regions.

[0029] Convert the page position coordinates within the operation area into azimuth and distance ratios relative to the center point of the operation area, and generate a topological description of the operation area.

[0030] Encode the spatial adjacency relationships between the topological descriptions of the operation areas, generate a global topological structure describing the relative positional relationships of multiple operation areas, and convert the global topological structure into the topological relationships of page elements;

[0031] Semantically bind the topological relationships of page elements to the corresponding operation types in the operation sequence, establish a two-way mapping relationship from the topological relationships of page elements to the operation types and from the operation types to the topological relationships of page elements, and store the two-way mapping relationship in the skill library.

[0032] The spatial adjacency relationships between the topological descriptions of the operation areas are encoded to generate a global topological structure describing the relative positional relationships of multiple operation areas. This global topological structure is then converted into page element topological relationships, including:

[0033] Extract the center point coordinates of each operation area from the topology description of the operation area, construct a spatial adjacency graph of the operation area based on the center point coordinates, and identify the transit adjacency paths between operation areas in the spatial adjacency graph.

[0034] The path distance and path orientation in the transmitted adjacent paths are vectorized and encoded to generate a global topology containing indirect spatial relationships of the operation area. The vectorized encoding in the global topology is decomposed into direct adjacency relationships and indirect adjacency relationships.

[0035] Spatial anchor points of page elements are located based on direct adjacency relationships, and hierarchical containment relationships between page elements are derived based on indirect adjacency relationships. Spatial anchor points and hierarchical containment relationships are then integrated with azimuth angles and distance ratios in the topological description of the operation area to convert them into topological relationships of page elements.

[0036] The topological relationships of page elements corresponding to the type of operation to be performed are read from the skill library. The operation position is reconstructed on the current page based on these relationships. The operation type and position are then input into the iteratively trained operation causal model to generate the execution action, including:

[0037] Read the topological relationships of page elements corresponding to the type of operation to be performed from the skill library, and extract the global topological structure recorded in the topological relationships of page elements as a topological matching template;

[0038] The spatial adjacency relationship between page elements is calculated based on the position coordinates and size information of page elements in the current page to generate the current page topology. The structural similarity between the current page topology and the topology matching template is calculated, and the matching region with the highest structural similarity is located.

[0039] Extract the azimuth and distance ratio from the topological description of the operation area from the topological relationship of page elements. Use the centroid of the matching area as the anchor point to determine the operation direction based on the azimuth and determine the operation distance based on the product of the distance ratio and the size of the matching area. Offset the operation distance in the operation direction to reconstruct the operation position.

[0040] The operation type and operation location are input into the operation causal model after iterative training. The operation causal model derives the state features of the successful operation result from the operation type based on the reverse inference relationship. The operation location is associated with the state features to generate the operation context and the execution action is generated based on the operation context.

[0041] Executing the execution action and obtaining the current operation result, and adding the current operation result to the operation sequence for subsequent iterative training includes:

[0042] The execution action is performed, the page state before the operation and the page state after the operation are collected, the set of page element positions in the page state before the operation and the page state after the operation are extracted, the position offset between the set of page element positions is calculated, and the position offset is compared with the preset successful operation position offset range to obtain the current operation result.

[0043] The operation type to be executed, operation position, current operation result, and position offset are combined to form the current operation record and added to the operation sequence;

[0044] Filter historical operation records from the operation sequence, extract the position offset difference between the current operation record and the historical operation record, and construct state transition constraint relationships based on the position offset difference;

[0045] The current operation record is input into the operation causal model, which then uses the reverse inference relationship to reverse infer the current operation result and generate a predicted operation type.

[0046] The consistency deviation between the predicted operation type and the operation type to be executed is calculated. The consistency deviation is then fused with the state transition constraint relationship to generate a comprehensive adjustment amount. The reverse derivation relationship is adjusted based on the comprehensive adjustment amount for subsequent iterative training.

[0047] This invention provides an intelligent browser adaptive operating system based on reinforcement learning, the system comprising:

[0048] The sequence acquisition module is used to acquire the operation sequence of the target business process, wherein the operation sequence includes the operation type, page position coordinates and operation result;

[0049] The causal model generation module is used to reverse-generate and train operation sequences through a multimodal large language model, establish a reverse inference relationship from operation result to operation type, and generate an operation causal model.

[0050] The iterative training module is used to autonomously generate virtual operation sequences and predict virtual operation results by the operation causal model during non-task execution. Virtual operation sequences with successful virtual operation results are used as enhancement samples to iteratively train the operation causal model.

[0051] The skill library construction module is used to convert the page position coordinates of successful operations in the operation sequence into page element topology relationships and store them in the skill library in association with the operation type.

[0052] The task receiving module is used to receive operation tasks that include the type of operation to be performed and the current page.

[0053] The action generation module is used to read the topological relationship of page elements corresponding to the type of operation to be performed from the skill library, reconstruct the operation position in the current page according to the topological relationship of page elements, and input the type of operation to be performed and the operation position into the operation causal model after iterative training to generate the execution action.

[0054] The result feedback module is used to execute the execution action and obtain the current operation result, and add the current operation result to the operation sequence for subsequent iterative training.

[0055] One technical solution provided in this embodiment of the invention is an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in any of the aforementioned methods.

[0056] One technical solution provided in this embodiment of the invention is a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps in any of the aforementioned methods.

[0057] This invention, by constructing a reverse inference relationship between operation results and operation behaviors, enables browser automation systems to possess causal learning and autonomous decision-making capabilities. This allows for dynamic adjustment of operation strategies when page structure changes, improving operational accuracy and adaptability in complex webpage environments. Enhanced training through autonomously generated virtual operation sequences reduces training costs in real-world environments and improves model generalization and continuous learning capabilities. Reconstructing operation positions using the topological relationships of page elements reduces reliance on fixed coordinates and static labels, enhancing operational stability and robustness across pages and versions. Attached Figure Description

[0058] Figure 1 A flowchart illustrating the reinforcement learning-based intelligent browser adaptive operation method provided in this embodiment of the invention;

[0059] Figure 2 This is a schematic diagram of the structure of an intelligent browser adaptive operating system based on reinforcement learning, according to an embodiment of the present invention. Detailed Implementation

[0060] like Figure 1 As shown, Figure 1This is a flowchart of a reinforcement learning-based intelligent browser adaptive operation method provided in an embodiment of the present invention. The method includes the following steps:

[0061] Step 101: Obtain the operation sequence of the target business process, wherein the operation sequence includes operation type, page position coordinates and operation result.

[0062] Step 102: The operation sequence is back-generated and trained using a multimodal large language model to establish a reverse inference relationship from operation result to operation type, thereby generating an operation causal model.

[0063] In some embodiments of the present invention, step 102 may specifically include the following sub-steps:

[0064] Sub-step 1021: Divide the operation sequence into success group and failure group according to the operation result, and perform time-series encoding on the operation type and page position coordinates in the success group and failure group respectively to generate success trajectory representation and failure trajectory representation.

[0065] Sub-step 1022: Calculate the causal difference features between the successful trajectory representation and the failed trajectory representation, and use the causal difference features as a constraint for the reverse generation training of the multimodal large language model.

[0066] Sub-step 1023: Input the samples in the operation sequence that have a successful operation result into the multimodal large language model to extract the page state features corresponding to the operation result;

[0067] Sub-step 1024: Based on the constraints, the page state features are reverse decoded to generate candidate operation types and candidate page position coordinates, and the candidate operation types and candidate page position coordinates that satisfy the causal difference features are selected as the reverse derivation output.

[0068] Sub-step 1025: Calculate the prediction error between the back-derivative output and the actual operation type and page position coordinates in the operation sequence. Minimize the prediction error through gradient backpropagation to update the parameter weights of the multimodal large language model. Establish the back-derivative relationship from the operation result to the operation type. Use the multimodal large language model after establishing the back-derivative relationship as the operation causal model.

[0069] In the specific implementation of adaptive operation in the intelligent browser, the operation sequence is obtained by monitoring the user's actual interactive behavior in the browser. Each user operation is recorded as an operation unit containing complete information. The operation types cover basic interactive actions such as clicking, swiping, inputting, and selecting, and each operation type corresponds to a specific behavior pattern and execution logic. The page position coordinates are standardized using a relative coordinate system, converting absolute pixel coordinates into proportional coordinates relative to the page container to ensure operational accuracy on devices with different resolutions. The operation results are determined through a page state change detection mechanism, including state transition information such as the appearance and disappearance of page elements, content changes, and navigation jumps.

[0070] The reverse generation training of the multimodal large language model adopts an end-to-end learning architecture, inputting operation sequences as training samples into the model for causal relationship learning. The model receives operation results as input conditions and learns to predict the operation type and page position coordinates that lead to the result. The training process adopts a supervised learning approach, using the correspondence between real operation results in historical operation sequences to construct the training objective. The model architecture includes multiple encoding layers to process input information from different modalities, where the text encoder processes page content and operation descriptions, the visual encoder processes page screenshots and element position information, and the temporal encoder processes the temporal dependencies of the operation sequence.

[0071] The grouping of operation sequences is based on a binary classification of the actual effects after operation execution. The success group contains operation records that achieved the expected operation goal, while the failure group contains operation records that failed to achieve the expected effect or produced incorrect results. For the operation data in both the success and failure groups, temporal coding techniques are used to convert the operation type into a vector representation and normalize the page position coordinates. The temporal coding process considers the dependencies between operations, capturing the temporal patterns in the operation sequence through positional coding and attention mechanisms. The success trajectory representation is obtained by aggregating all operation vectors in the success group, and the failure trajectory representation is obtained by aggregating all operation vectors in the failure group.

[0072] The calculation of causal difference features employs a contrastive learning approach, identifying key causal factors in operations by comparing the differences between successful and failed trajectory representations. The calculation process involves vector operations on the two trajectory representations to extract the key feature differences that lead to successful and failed operations. The degree of difference is measured by the distance between the two vector representations; a larger distance indicates a more significant difference between the two trajectories. The calculated causal difference features serve as constraints in the subsequent reverse generation process, guiding the model to learn the key features that distinguish between successful and failed operations.

[0073] Page state feature extraction is performed on samples where operations were successful. The encoder component of a multimodal large language model processes multi-dimensional information about the page. Page state features include page layout structure, element attribute information, content semantic features, and visual appearance features. The extraction process converts the page DOM structure into a tree representation, performs semantic analysis and entity recognition on the page content, and extracts features and segments regions from page screenshots. Multiple feature dimensions are merged into a unified page state vector representation through a fusion mechanism. Feature fusion uses a weighted average method, assigning appropriate weights based on the importance of different feature types.

[0074] The reverse decoding generation process, based on page state features and causal difference constraints, generates candidate operation types and page position coordinates through a decoder network. The decoding process uses a beam search algorithm to generate multiple candidate results, each containing a probability distribution of the operation type and predicted values ​​for the position coordinates. The selection process calculates the degree of matching between the candidate results and the causal difference features, evaluating whether the candidate operation conforms to the feature pattern of a successful operation. The accuracy of the position coordinates is determined by evaluating the offset distance between the predicted coordinates and the actual position of the page element; predictions with an offset distance within 5 pixels are considered accurate. Candidate operations that meet the matching threshold are selected as the reverse derivation output.

[0075] The prediction error is calculated by comparing the differences between the back-derived output and the actual operation type and page position coordinates in the operation sequence. The prediction error for the operation type is calculated using a classification error method, statistically analyzing the proportion of samples where the predicted type does not match the actual type. The prediction error for the page position coordinates is calculated by the Euclidean distance between the predicted and actual coordinates; this distance reflects the prediction accuracy. The overall prediction error is obtained by weighted combining the two errors, with the operation type error and position coordinate error accounting for 0.6 and 0.4 weights, respectively.

[0076] The gradient backpropagation process calculates the gradient values ​​of the parameters of each network layer based on the prediction error. The gradient values ​​indicate the direction and magnitude of parameter adjustments. Parameter updates employ an adaptive learning rate optimization algorithm, with the learning rate dynamically adjusted based on gradient changes. The initial learning rate is set to 0.001. Training is conducted using batch training, with each batch containing 32 operation samples. Multiple iterations are performed until the model converges. Convergence is determined based on the magnitude of the prediction error change over 10 consecutive training epochs; the model is considered converged when the error change is less than 0.01.

[0077] The trained multimodal large language model possesses the ability to infer the operation type from the operation result, forming an operation causal model. This model can automatically recommend appropriate operation types and execution positions based on the target page state, providing adaptive operation decision support for intelligent browsers. The model's inference process receives the current page state as input and outputs the recommended operation type and corresponding page position coordinates. The inference time is controlled within 100ms to ensure real-time requirements.

[0078] This invention establishes a reverse derivation relationship from operation result to operation type, thereby enabling the intelligent browser to make autonomous operation decisions, improving the intelligence level and operational efficiency of user interaction, reducing user learning costs, and enhancing the browser's adaptability and usability in complex webpage environments.

[0079] Step 103: During non-task execution, the operation causal model autonomously generates virtual operation sequences and predicts virtual operation results. Virtual operation sequences with successful virtual operation results are used as enhancement samples to iteratively train the operation causal model.

[0080] In some embodiments of the present invention, step 103 may specifically include the following sub-steps:

[0081] Sub-step 1031: Extract operation type and page position coordinates from operation sequence to construct perturbation sample pool, apply random perturbation to operation type and page position coordinates in perturbation sample pool to generate perturbation operation combination, input perturbation operation combination into operation causal model for reverse decoding to generate virtual operation sequence;

[0082] Sub-step 1032: The virtual operation sequence is forward-inferred by the operation causal model to predict the virtual operation result. The virtual operation sequence with the virtual operation result of success is compared and analyzed with the virtual operation sequence with the operation result of failure to extract the decision boundary features of successful operation. Based on the decision boundary features, the virtual operation sequence with the virtual operation result of success is labeled to generate enhanced samples.

[0083] Sub-step 1033 involves inputting the augmented sample into the operational causal model, adjusting the reverse derivation relationship of the operational causal model based on the decision boundary features in the augmented sample, and iteratively training the operational causal model.

[0084] During non-task execution, the operational causal model self-trains and improves its capabilities by autonomously generating virtual operational sequences. These virtual sequences are generated based on operational patterns in existing sequences, creating new combinations of operations through perturbation and recombination, thus expanding the diversity and coverage of the training samples. This self-learning mechanism enables the model to continuously optimize its operational reasoning capabilities even without new real operational data input.

[0085] The perturbation sample pool is constructed by extracting all occurrences of operation types and their corresponding page position coordinates from historical operation sequences. Operation types include basic interactive actions such as click, double-click, long-press, swipe, input, and selection. Each operation type is associated with its position information in different page scenarios. Page position coordinates are standardized by converting raw pixel coordinates into proportional values ​​relative to the page viewport, ensuring device independence of the coordinate information. The perturbation sample pool is categorized and stored according to operation type, while also recording the execution frequency and success rate statistics for each operation, providing a weighted reference for subsequent perturbation generation.

[0086] Random perturbations are applied by injecting Gaussian noise to modify the original operational data. For perturbations of operation types, semantically similar operation types are randomly selected for replacement using a probability distribution, with the replacement probability determined based on the similarity between operation types. For perturbations of page position coordinates, a random offset is added to the original coordinates, with the offset ranging from 0.1 to 0.3 times the page element size to ensure that the perturbed position remains operationally feasible. The perturbation intensity is controlled through a dynamic adjustment mechanism, adaptively adjusting the perturbation amplitude based on the model's current training state and prediction accuracy.

[0087] The perturbed operation combinations generate diverse operation sequences through random sampling and rearrangement. The sampling process considers the temporal dependencies between operations to avoid generating logically illogical operation sequences. The rearrangement is constrained based on the execution conditions and pre-dependencies of the operations to ensure that the generated virtual operation sequences are actually executable. Each generated operation combination contains complete operation type, location coordinates, and temporal information, forming a structured virtual operation sequence.

[0088] The reverse decoding process of the operational causal model receives the perturbed combination of operations as input and generates a corresponding virtual operation sequence through its internal decoding network. The decoding process utilizes the operational patterns and causal relationships learned by the model to reorganize the input operational elements into a sequence structure that conforms to operational logic. The generated virtual operation sequence maintains the same data format and structural characteristics as the real operation sequence, containing complete information such as the execution order, time intervals, and operation parameters.

[0089] Forward inference prediction utilizes the forward reasoning capability of an operational causal model, taking a generated sequence of virtual operations as input to predict the possible results after execution. The inference process simulates a real-world operation execution environment, considering various outcome types such as page state changes, element responses, and navigation transitions. Prediction results are categorized into success and failure; success indicates the operation achieves its intended goal, while failure indicates the operation fails to execute correctly or produces an incorrect result. Prediction confidence is calculated using the probability distribution output by the model; predictions with a confidence level higher than 0.8 are considered reliable.

[0090] Comparative analysis is conducted by comparing the feature differences between successful and failed virtual operation sequences. The analysis process extracts patterns of difference between the two types of sequences in dimensions such as operation type selection, location coordinate distribution, and timing. The decision boundary characteristics of successful operations are identified through statistical analysis, including the frequency of occurrence of key operation types, clustering areas of location coordinates, and the reasonable range of operation intervals. The decision boundary is quantitatively represented as a feature vector, with vector dimensions corresponding to different operation feature types, and vector values ​​reflecting the degree of influence of that feature on operation success.

[0091] Where n represents the total number of features, w i f represents the weight coefficient of the i-th feature. i This represents the standardized value of the i-th feature. The weight coefficients are determined based on the contribution of each feature to the success rate of the operation, and the feature values ​​are normalized to ensure consistency within the numerical range.

[0092] Augmentation samples are generated by evaluating and labeling successful virtual operation sequences based on decision boundary features. The labeling process assigns a quality score to each virtual operation sequence, taking into account the rationality of the operation, its execution difficulty, and the expected effect. High-quality virtual operation sequences are labeled as positive samples and added to the training dataset as augmentation samples. The sample labeling information includes detailed execution steps, expected results, and key feature identifiers of the operation sequence, providing complete supervision signals for subsequent model training.

[0093] The sample quality score is calculated as follows: α × Reasonableness Score + β × Complexity Score + γ × Diversity Score. The Reasonableness Score assesses the logical correctness of the operational sequence, the Complexity Score measures the difficulty of execution, and the Diversity Score reflects the novelty of the sample. The weighting parameters α, β, and γ are set to 0.5, 0.3, and 0.2, respectively, to ensure the comprehensiveness and accuracy of the quality assessment.

[0094] The iterative training process mixes augmented samples with the original training data to form an expanded training set. The training process employs an online learning approach, with new augmented samples continuously added to the training flow. Model parameters are progressively optimized through a gradient update mechanism. Decision boundary features are included as part of the training objective, guiding the model to learn more accurate operational causal relationships. The loss function during training combines prediction accuracy and decision boundary consistency as two objectives, improving the overall performance of the model through multi-objective optimization.

[0095] The adjustment of the reverse inference relationship is based on the new operational patterns and causal laws reflected in the augmented samples. The adjustment process identifies deviations and deficiencies in the inference logic by comparing the current inference results of the model with the standard answers in the augmented samples. The model parameters are updated using an adaptive learning rate strategy, dynamically adjusting the learning intensity according to training progress and performance improvement. The convergence judgment of iterative training is based on the model's performance on the validation set. Training stops when the performance improvement is less than a set threshold over several consecutive training cycles.

[0096] This invention significantly improves the generalization ability and inference accuracy of the operational causal model by autonomously generating virtual training samples and using an iterative optimization mechanism. It also enhances the model's adaptability to complex operational scenarios and improves the autonomous operation effect and user experience quality of the intelligent browser in diverse web page environments.

[0097] Step 104: Convert the page position coordinates of the operation sequence that result in a successful operation into page element topology relationships and store them in the skill library in association with the operation type.

[0098] In some embodiments of the present invention, step 104 may specifically include the following sub-steps:

[0099] Sub-step 1041: Sort the page position coordinates with successful operation results in the operation sequence according to the time sequence of operation execution, and perform cluster analysis on the sorted page position coordinates to identify spatially clustered operation regions.

[0100] Sub-step 1042 converts the page position coordinates within the operation area into the azimuth angle and distance ratio relative to the center point of the operation area, and generates a topological description of the operation area.

[0101] Sub-step 1043: Encode the spatial adjacency relationship between the topological descriptions of the operation areas, generate a global topological structure describing the relative positional relationship of multiple operation areas, and convert the global topological structure into the topological relationship of page elements.

[0102] Sub-step 1044: Semantically bind the topological relationship of page elements to the corresponding operation type in the operation sequence, establish a bidirectional mapping relationship from the topological relationship of page elements to the operation type and from the operation type to the topological relationship of page elements, and store the bidirectional mapping relationship in the skill library.

[0103] The page position coordinates of successful operations in the operation sequence are arranged in ascending order according to the timestamp of the operation execution, constructing a temporally ordered coordinate sequence. This temporal sorting preserves the logical order and dependencies of user operations, ensuring that subsequent spatial analysis can correctly identify key nodes and turning points in the operation flow. The sorting process further prioritizes multiple operations with the same timestamp based on their operation type. Priority is determined by the importance of the operation's impact on the page state: click operations have higher priority than swipe operations, and input operations have higher priority than selection operations.

[0104] Cluster analysis employs a density-based spatial clustering method to divide the sorted page position coordinates into regions. The clustering process groups closely spaced coordinate points into the same operational region. The degree of proximity is measured by calculating the Euclidean distance between coordinate points, with a distance threshold set to 5% of the page width. This ensures that the clustering results reflect spatial concentration while avoiding over-segmentation. The clustering algorithm scans all coordinate points, assigning adjacent points with a distance less than the threshold to the same cluster, forming spatially contiguous operational regions. Each operational region contains at least three coordinate points; isolated points are either merged into the nearest operational region or form independent single-point regions.

[0105] The center point of the operating area is obtained by calculating the centroid of all coordinate points within the area. The centroid coordinates are the arithmetic mean of the x and y coordinates within the area. The azimuth angle of each coordinate point within the area relative to the center point is calculated using the arctangent function. The azimuth angle ranges from 0 to 360 degrees, with due east as the 0-degree reference and increasing counterclockwise. The distance ratio represents the ratio of the straight-line distance from the coordinate point to the center point to the radius of the operating area. The area radius is defined as the distance from the center point to the farthest point on the area boundary. The combination of azimuth angle and distance ratio forms a relative position description in polar coordinates, eliminating the device dependence of absolute coordinates.

[0106] The topological description of the operating region is generated by structurally organizing the polar coordinate information of all coordinate points within the region. The topological description includes spatial attribute information such as the region's geometric shape features, coordinate point distribution patterns, and density characteristics. Geometric shape features are described by calculating parameters such as the region's aspect ratio, roundness, and convex hull area. The aspect ratio reflects the region's direction of extension, roundness measures the regularity of the region's shape, and the convex hull area represents the region's spatial extent. Distribution patterns are identified by analyzing the frequency of coordinate point distribution in different azimuth intervals, and density characteristics are quantified by calculating the number of coordinate points per unit area.

[0107] The encoding of spatial adjacency relationships is based on the relative positions and distances between operational regions. Adjacency determination employs a combination of minimum boundary rectangle overlap detection and center point distance calculation. Two regions are considered adjacent if their boundary rectangles overlap or the center point distance is less than 1.5 times the sum of their radii. The adjacency relationship is represented by an adjacency matrix, where each element represents the adjacency strength between two regions. This strength is calculated as the ratio of the overlapping area to the smaller of the two region areas.

[0108] The global topology describes the overall spatial layout of multiple regions by constructing a connected graph of the operation areas. The connected graph uses operation areas as nodes and adjacency relationships as edges. Node attributes include the geometric features and coordinate information of the region, while edge attributes include adjacency strength and relative orientation. The topology is generated using a graph traversal algorithm to identify connected components and critical paths. Connected components represent spatially related sets of regions, and critical paths represent the main spatial trajectories in the operation process.

[0109] The transformation of page element topology maps the spatial layout of the operation area to the logical relationships between page elements. This transformation process combines page DOM structure information and visual layout features, associating spatially similar operation areas with the same page element or element container. Page element identification is achieved by reverse-searching the corresponding element node in the page DOM tree using coordinates. The search process considers the element's hierarchy and display attributes, prioritizing the matching of interactive leaf nodes. Topology descriptions include various types of relationships such as containment, sibling, and proximity between elements, with the relationship type determined comprehensively based on DOM structure and spatial location.

[0110] The semantic binding process establishes an association mapping between the topological relationships of page elements and the corresponding operation types in the operation sequence. The binding process analyzes the statistical distribution of operation types executed within each operation area to identify the primary and secondary operation modes for that area. The primary operation mode is the operation type that occurs most frequently, while the secondary operation modes are other operation types that occur more than 20% of the total number of operations. The binding of operation types and topological relationships uses a multi-value mapping method; the same topological relationship can correspond to multiple operation types, and the same operation type can also be applied to multiple topological relationships.

[0111] The establishment of bidirectional mapping relationships includes a forward mapping from page element topology relationships to operation types and a reverse mapping from operation types to page element topology relationships. Forward mapping supports recommending suitable operation types based on page layout features, while reverse mapping supports finding applicable page areas based on the target operation type. The mapping relationships are stored using a weighted graph structure, where nodes represent topology relationships or operation types, and edge weights represent the strength of the association, supporting fast bidirectional queries and reasoning.

[0112] The skill library employs a hierarchical index structure, building multi-level indexes based on page type, element type, and operation complexity. The storage structure supports fast retrieval based on topological similarity, matching and ranking the query topology against the topologies in the library. Similarity calculation comprehensively considers multiple feature dimensions such as geometric shape, spatial distribution, and adjacency, using weighted Euclidean distance to measure topological differences. The skill library supports incremental updates and version management; new skills can be dynamically added to the library, and historical versions of skills can be phased out and optimized based on performance evaluation results.

[0113] The associated stored procedure categorizes and organizes bidirectional mapping relationships according to topology type and operation type, establishing a multi-dimensional indexing system. The storage format uses a key-value pair structure, where the key is the feature vector of the topology relationship, and the value is the corresponding set of operation types and their weight information. The stored procedure also establishes an inverted index, using the operation type as the key and the topology relationship as the value, enabling fast bidirectional queries. The skill library regularly performs data compression and deduplication, removing redundant entries with excessive similarity to maintain storage efficiency and query performance.

[0114] This invention abstracts specific page coordinates into device-independent topological descriptions, enabling cross-platform reuse and intelligent matching of operational skills. This improves the accuracy and efficiency of browser adaptive operations, reduces dependence on specific page layouts, and enhances the generalization and adaptability of operational skills.

[0115] Sub-step 1043, encoding the spatial adjacency relationships between the topological descriptions of the operation areas to generate a global topological structure describing the relative positional relationships of multiple operation areas, and converting the global topological structure into page element topological relationships, further includes:

[0116] Extract the center point coordinates of each operation area from the topology description of the operation area, construct a spatial adjacency graph of the operation area based on the center point coordinates, and identify the transit adjacency paths between operation areas in the spatial adjacency graph.

[0117] The path distance and path orientation in the transmitted adjacent paths are vectorized and encoded to generate a global topology containing indirect spatial relationships of the operation area. The vectorized encoding in the global topology is decomposed into direct adjacency relationships and indirect adjacency relationships.

[0118] Spatial anchor points of page elements are located based on direct adjacency relationships, and hierarchical containment relationships between page elements are derived based on indirect adjacency relationships. Spatial anchor points and hierarchical containment relationships are then integrated with azimuth angles and distance ratios in the topological description of the operation area to convert them into topological relationships of page elements.

[0119] In the topology description of the operation area, the center point coordinates of each operation area are obtained through centroid calculation. The centroid calculation process involves summing the x-coordinates of all points within the area and dividing by the total number of points to obtain the x-coordinate of the center point. Similarly, summing the y-coordinates of all points within the area and dividing by the total number of points yields the y-coordinate of the center point. These center point coordinates serve as the representative location of the operation area on a two-dimensional plane and are used for subsequent spatial relationship analysis and adjacency graph construction. Each operation area corresponds to a unique center point coordinate, and the precision of the center point coordinates remains at the same pixel level as the original page coordinates.

[0120] The spatial adjacency graph is constructed based on the Euclidean distance between the center points of the operating regions. The distance is calculated using the standard two-dimensional planar distance formula, taking the square root of the sum of the squares of the differences in the coordinates of the two center points to obtain the straight-line distance. Nodes in the adjacency graph represent operating regions, and edges represent the spatial adjacency relationships between regions. An edge exists only if the distance between the center points of two regions is less than a preset adjacency threshold. This threshold is dynamically adjusted based on the page size and is set to 15% of the page diagonal length, ensuring that the adjacency relationships reflect actual spatial proximity while avoiding over-connectivity. The adjacency graph uses an undirected graph structure, with each edge bidirectionally connected, and the edge weight is the normalized distance between the two center points.

[0121] The identification of transitive adjacency paths utilizes a graph traversal algorithm to perform a depth-first search in the spatial adjacency graph. The search process starts from each node and extends along adjacent edges to neighboring nodes, recording all possible paths from the starting node to the target node. The path length is limited to three steps to avoid introducing noise through excessively long transitive paths. Transitive adjacency paths include two types: direct adjacency paths and indirect adjacency paths. Direct adjacency paths connect two directly adjacent operation regions, while indirect adjacency paths connect two non-directly adjacent operation regions through one or more intermediate nodes. The path search process records the access order and cumulative distance of each node on the path, providing foundational data for subsequent vectorization encoding.

[0122] Vectorized encoding of path distances converts the cumulative distance on adjacent paths into normalized vector components. The cumulative distance is the sum of the weights of all edges on the path. Normalization divides the cumulative distance by the length of the page diagonal, resulting in distance component values ​​ranging from 0 to 1. These distance components reflect the spatial separation between operating regions; smaller values ​​indicate closer regions, while larger values ​​indicate farther regions. The distance vectors are stored using a fixed-dimensional array structure, with the array length equal to the total number of operating regions. Each array element corresponds to a specific target region.

[0123] The vectorized encoding of path orientation is based on the relative azimuth angle between the path's starting and ending points. The relative azimuth angle is calculated using the arctangent function, representing the angle difference between the coordinates of the two center points. The angle range is from 0 to 360 degrees, with the horizontal rightward direction as the 0-degree reference. Azimuth angle encoding uses coordinates on a unit circle, converting the angle values ​​into a two-dimensional vector composed of cosine and sine values. The cosine component of the azimuth vector represents the horizontal projection, and the sine component represents the vertical projection. The combination of these two components fully describes the directional characteristics of the path.

[0124] The global topology is generated by integrating the vectorized encoding information of all transitive adjacency paths. The structure construction process treats each operational region as a central node, collecting path vectors from that node to all other nodes to form a local topology view centered on that region. The global topology is composed of all local topology views and contains complete spatial relationship information between operational regions. The structure data is organized in matrix form, with rows representing starting regions and columns representing target regions. Matrix elements are the vectorized encodings of the corresponding paths.

[0125] The vectorized encoding decomposition process separates the composite vectors in the global topology into two subsets: direct adjacency relationships and indirect adjacency relationships. The decomposition process is based on path length: paths with a length of 1 correspond to direct adjacency relationships, while paths with a length greater than 1 correspond to indirect adjacency relationships. Direct adjacency relationships describe the direct spatial connections between operational regions, while indirect adjacency relationships describe spatial connections transmitted through intermediate regions. The decomposition results are stored in direct adjacency matrices and indirect adjacency matrices, respectively, for subsequent differential processing and application.

[0126] Spatial anchor points are identified based on the connection strength and direction features of direct adjacency relationships. Connection strength is calculated by the inverse of the weights of directly adjacent paths; a smaller weight indicates a stronger connection. Anchor candidate regions are operational areas with connection strength exceeding the average. The anchor selection process further considers the region's degree and centrality. Degree represents the number of other regions directly connected to the current region, while centrality represents the region's importance in the adjacency graph. Spatial anchor points serve as key reference points for page layout, used to determine the relative positions of other page elements.

[0127] The derivation of hierarchical containment relationships is based on path patterns and propagation distances in indirect adjacency relationships. The derivation process identifies path patterns with containment characteristics. A containment relationship is determined when all indirect paths from one operational region to another pass through the same intermediate region. The intermediate region is identified as the parent container, and the starting and ending regions are identified as child elements. The depth of the hierarchical relationship is determined by the shortest length of the indirect paths, where path length represents the nesting depth of the hierarchy. The containment relationship is organized using a tree structure, with the root node being the top-level container and the leaf nodes being the bottom-level page elements.

[0128] The fusion process combines spatial anchor points and hierarchical inclusion relationships with the azimuth and distance ratios in the topological description of the operating area. The fusion process uses the anchor point as the origin and recalculates the azimuth and distance ratios of other operating areas relative to the anchor point. Hierarchical inclusion relationships provide logical attribution information between elements, which, combined with spatial location information, forms a complete topological relationship between page elements. The conversion result includes multi-dimensional relationship information such as the spatial location, hierarchical attribution, and adjacency connections of each page element.

[0129] The final generation of page element topological relationships involves structuring and standardizing the fused multi-dimensional relationship information. The relationship representation uses a graph structure, where nodes represent page elements and edges represent various relationship types between elements. Relationship types include spatial adjacency, hierarchical containment, directional association, and other semantic relationships. The topological relationship data is encoded using a standardized description language, supporting cross-platform parsing and application. The generated page element topological relationships provide intelligent browsers with an abstract representation of page structure, supporting structure-based operational reasoning and adaptive adjustments.

[0130] This invention achieves precise quantification of spatial relationships in the operation area by constructing transitive adjacency paths and vectorized encoding. It establishes a structured topological representation of page elements through anchor point positioning and hierarchical derivation, thereby improving the intelligent browser's ability to understand page layout and its operational adaptability, and providing a reliable structural foundation for cross-device adaptive operation.

[0131] Step 105: Receive the operation task containing the type of operation to be performed and the current page.

[0132] Step 106: Read the topological relationship of page elements corresponding to the type of operation to be performed from the skill library, reconstruct the operation position in the current page according to the topological relationship of page elements, and input the type of operation to be performed and the operation position into the operation causal model after iterative training to generate the execution action.

[0133] In some embodiments of the present invention, step 106 may specifically include the following sub-steps:

[0134] Sub-step 1061: Read the topological relationship of page elements corresponding to the type of operation to be performed from the skill library, and extract the global topological structure recorded in the topological relationship of page elements as a topological matching template;

[0135] Sub-step 1062: Calculate the spatial adjacency relationship between page elements based on the position coordinates and size information of page elements in the current page to generate the current page topology structure, calculate the structural similarity between the current page topology structure and the topology matching template, and locate the matching region with the highest structural similarity.

[0136] Sub-step 1063: Extract the azimuth and distance ratio from the topological description of the operation area from the topological relationship of page elements; determine the operation direction based on the azimuth using the centroid of the matching area as the anchor point; determine the operation distance based on the product of the distance ratio and the size of the matching area; and reconstruct the operation position by offsetting the operation distance in the operation direction.

[0137] Sub-step 1064: Input the operation type and operation position to be executed into the operation causal model after iterative training. The operation causal model derives the state features of the successful operation result from the operation type to be executed based on the reverse inference relationship. The operation position is associated with the state features to generate the operation context and the execution action is generated based on the operation context.

[0138] The topological relationships of page elements corresponding to the type of operation to be performed in the skill library are indexed and looked up using the operation type identifier. The skill library is organized using a hash table structure, with the operation type identifier as the key and the corresponding page element topological relationship data as the storage value. The reading process calculates the hash value based on the type of operation to be performed, locates the corresponding storage bucket, and traverses the topological relationship records in the bucket until a matching entry is found. The topological relationship data contains complete content such as the global topological structure, the topological description of the operation area, and semantic binding information.

[0139] The global topology structure is extracted from the topological relationships of the read page elements as a topology matching template. The extraction process parses the structured fields in the topology relationship data to obtain the spatial adjacency matrix and hierarchical containment tree between operation regions. The spatial adjacency matrix records the direct connections and connection strengths between operation regions, while the hierarchical containment tree describes the nested structure of the regions. The topology matching template standardizes this structural information into a unified graph representation format, where nodes represent operation regions, edges represent various relationship types between regions, and edge attributes include relationship strength and direction information.

[0140] The position coordinates and size information of page elements in the current page are obtained through the page document object model interface. The process involves traversing all visible element nodes on the page and calling the element's bounding rectangle calculation method to obtain the top-left corner coordinates, width, and height of each element. Position coordinates are represented using the page coordinate system, with the origin at the top-left corner of the page, the horizontal axis pointing right as positive, and the vertical axis pointing down as positive. Size information includes the element's actual display width and height, in pixels. The coordinate and size data constitute the spatial description information of the element, providing the basic data for subsequent spatial relationship calculations.

[0141] The spatial adjacency relationships between page elements are generated based on the overlap detection and distance calculation of the element boundary rectangles. Adjacency determination uses a boundary rectangle expansion method, extending the boundary rectangle of each element outwards by a fixed pixel distance, which can be set to 20% of the smaller side length of the element. Elements are considered adjacent when their expanded boundary rectangles overlap. The strength of the adjacency relationship is calculated as the ratio of the overlapping area to the smaller area of ​​the two elements; a larger ratio indicates a stronger adjacency relationship.

[0142] The current page topology is generated by constructing an undirected graph structure using page elements as nodes and adjacency relationships as edges. The graph construction process creates a corresponding node for each page element, with node attributes including the element's coordinates, size, and type. Edge creation is based on adjacency relationship detection results; adjacent nodes are connected by edges, with edge weights equal to the adjacency strength value. The topology is represented using an adjacency list, where each node maintains a list of its directly connected neighbor nodes.

[0143] Structural similarity is calculated using a graph matching algorithm to compare the similarity between the current page's topology and a topology matching template. The calculation process includes evaluation of two dimensions: node similarity and edge similarity. Node similarity is measured based on element type, size ratio, and positional distribution. Type matching is the ratio of the number of nodes of the same type to the total number of nodes. Size similarity is obtained by calculating the normalized value of the size difference between corresponding nodes. Edge similarity is evaluated by comparing the differences in weight and connection patterns of corresponding edges.

[0144] S=A1·S n +A2·S e Where S represents the overall structural similarity, S n S represents node similarity. e Let A1 represent the edge similarity, A2 represent the node similarity weight coefficient, and the sum of the weight coefficients equals 1.

[0145] The matching region is located by finding the subgraph with the highest similarity to the topology matching template within the current page topology. The localization process uses a sliding window method, employing different node combinations as candidate matching regions and calculating the structural similarity between each candidate region and the template. The similarity calculation results are sorted in descending order, and the candidate region with the highest similarity is determined as the best matching region. The matching region contains one or more consecutive page elements whose spatial distribution most closely resembles the distribution of the operational regions in the template.

[0146] The azimuth and distance ratio values ​​in the topology description of the operation area are extracted from the topology relationship data of page elements. The extraction process involves parsing the polar coordinate encoding field of the topology description to obtain the azimuth angle and normalized distance value of the operation target point relative to the center of the area. The azimuth angle ranges from 0 to 360 degrees, with 0 degrees representing due east and the angle increasing counterclockwise. The distance ratio is the ratio of the actual distance from the operation point to the center point to the radius of the operation area, ranging from 0 to 1, where 0 indicates the location is at the center of the area and 1 indicates the location is at the boundary of the area.

[0147] The centroid of the matched region is calculated by taking a weighted average of the center coordinates of all page elements within the region. The weighted averaging process multiplies each element's center coordinate by its area weight, and the sum of all weighted coordinates is divided by the total weight to obtain the centroid coordinates. The area weight is the ratio of the element's area to the total area of ​​the region; elements with larger areas have a greater impact on the centroid position. The centroid coordinates serve as the anchor point for reconstructing the operation position, and subsequent direction and distance calculations are all based on the centroid as the reference origin.

[0148] The operation direction is determined based on an angle transformation performed between the extracted azimuth and the current page's coordinate system. This transformation maps the azimuth from the topology description to the page's coordinate system, taking into account potential page rotation and scaling. The operation direction is represented by a unit vector, where the horizontal component is the cosine of the azimuth and the vertical component is the sine. This unit direction vector provides a precise directional indication from the anchor point to the target operation point.

[0149] Where D represents the operation distance, R represents the distance ratio extracted from the topology description, W represents the width of the matching region, H represents the height of the matching region, and 0.5 is a scaling factor used to adjust the sensitivity of the distance calculation.

[0150] The reconstruction of the operation position is achieved by offsetting the anchor point coordinates along the operation direction by an operation distance. The reconstruction process multiplies the unit vector of the operation direction by the calculated operation distance to obtain the offset vector from the anchor point to the target position. The offset vector is then added to the anchor point coordinates to obtain the reconstructed operation position coordinates. These coordinates are represented using the page coordinate system to ensure consistency with the coordinate system of the elements on the current page.

[0151] The operational causal model receives the type of operation to be executed and the reconstructed operation location as input data for inference computation. The input data is converted into a vector representation format within the model through a feature encoding layer. The operation type is encoded as a one-hot vector, and the operation location is encoded as a two-dimensional coordinate vector. The encoded feature vectors are then input into the model's inference network for forward computation.

[0152] The establishment of reverse inference relationships is based on the reverse reasoning mechanism of the operation causal model. The inference process starts from the type of operation to be executed and infers the page state characteristics that should be achieved after successfully executing the operation through the causal relationship mapping learned by the model. The state characteristics include multi-dimensional information such as expected changes in page elements, interactive responses, and visual feedback. The reverse inference mechanism utilizes the operational result experience accumulated during model training to provide goal guidance for the successful execution of the current operation.

[0153] The generation of operational context is achieved by associating and encoding operational position information with derived state features. This associative encoding process concatenates the coordinate information of the operational position with the multidimensional vector of the state features to form a comprehensive context representation containing both spatial and target information. The context encoding uses a fixed-length vector format, facilitating unified model processing and efficient computation.

[0154] The decoding of execution actions is based on the operation context vector, which is used to generate action sequences through the model's output layer. The decoding process inputs the context vector into the action decoding network, and the network outputs an action sequence containing specific operation instructions. These instructions include complete execution information such as operation type, execution location, operation parameters, and timing. The generated execution actions are then converted into a format and output to the browser's operation execution module for actual execution.

[0155] This invention enables cross-page reuse of operational skills through topology matching and position reconstruction, and improves the accuracy and adaptability of operation execution through causal inference and context encoding, providing intelligent browsers with reliable adaptive operation capabilities and effectively solving the problem of operation position adaptation under different page layouts.

[0156] Step 107: Execute the execution action and obtain the current operation result, and add the current operation result to the operation sequence for subsequent iterative training.

[0157] In some embodiments of the present invention, step 107 may specifically include the following sub-steps:

[0158] Sub-step 1071: Execute the execution action, collect the page state before operation and the page state after operation, extract the set of page element positions in the page state before operation and the page state after operation, calculate the position offset between the set of page element positions, and compare the position offset with the preset successful operation position offset range to obtain the current operation result.

[0159] Sub-step 1072: Combine the operation type to be executed, operation position, current operation result and position offset to form the current operation record and add it to the operation sequence;

[0160] Sub-step 1073: Filter historical operation records from the operation sequence, extract the position offset difference between the current operation record and the historical operation record, and construct state transition constraint relationship based on the position offset difference;

[0161] Sub-step 1074: Input the current operation record into the operation causal model. The operation causal model uses the reverse inference relationship to reverse infer the current operation result and generate a predicted operation type.

[0162] Sub-step 1075: Calculate the consistency deviation between the predicted operation type and the operation type to be executed, integrate the consistency deviation with the state transition constraint relationship to generate a comprehensive adjustment amount, and adjust the reverse derivation relationship based on the comprehensive adjustment amount for subsequent iterative training.

[0163] The specific execution process of the actions is achieved by translating operation instructions into actual user interactions through the browser's automation interface. The execution process calls the appropriate operation method based on the action type; click operations are implemented through coordinate positioning and mouse event simulation, while input operations are implemented through element selection and keyboard event sending. The atomicity of the operations is maintained throughout the execution process, ensuring that each action is fully executed before state data is collected.

[0164] The page state is captured immediately before the action command is executed, and the captured content includes a complete snapshot of the page's document object model. The capture process traverses all visible element nodes on the page, recording the identifier, position coordinates, size information, and attribute values ​​of each element. The page state data is stored in a structured format, and the element information is organized according to the hierarchical relationship of the document tree.

[0165] The post-operation page state is collected after the action is completed, and the collection method is consistent with the pre-operation state collection. The collection timing is determined by page stability detection. The detection process monitors the dynamic changes of the page, and state collection is triggered when the position and content of page elements remain stable for 200ms.

[0166] The extraction of page element position sets involves filtering the coordinate information of all visible elements from the collected page state data. The extraction process filters out hidden elements and zero-size elements, retaining only elements that are visible to the user and have an actual display area. The position information includes the coordinates of the element's top-left and bottom-right corners, forming a bounding rectangle description of the element.

[0167] The position offset is calculated by comparing the coordinate differences of the page element position set before and after the operation. The calculation process calculates the coordinate change of each coexisting element, including displacement values ​​in both horizontal and vertical directions. The offset calculation uses the Euclidean distance formula, taking the square root of the sum of the squares of the horizontal and vertical offsets to obtain the total displacement distance.

[0168] , where Δ represents the position offset of the element, x1 and y1 represent the coordinates of the element before the operation, and x2 and y2 represent the coordinates of the element after the operation.

[0169] The preset successful operation position offset range is defined based on the expected effect of different operation types. The successful offset range for click operations is typically small, focusing primarily on the state change of the clicked element. The successful offset range for scroll operations is larger, needing to consider the overall movement of the page content. The offset range is represented by a minimum and maximum value range, with different threshold settings corresponding to different operation types.

[0170] The current operation result is obtained by comparing the calculated position offset with a preset successful operation offset range. The comparison process determines whether the offset falls within the successful range; operations within the range are marked as successful, and operations outside the range are marked as failed. For complex operations involving changes to multiple elements, a weighted scoring method is used to comprehensively evaluate the operation result.

[0171] The current operation record encapsulates the operation type, operation location, current operation result, and location offset according to a predefined data structure. The record structure includes fields such as operation identifier, timestamp, operation parameters, execution result, and status change. The unique identifier of the operation record is generated using a hash value combining the operation type, location coordinates, and timestamp.

[0172] The addition operation in an operation sequence inserts the newly generated current operation record at the end of the existing operation sequence. The operation sequence is organized using a linked list structure, supporting efficient insertion and traversal operations. Records in the sequence are arranged in chronological order of execution time. The sequence capacity is managed using a sliding window method; when the sequence length exceeds a preset limit, the oldest record is deleted.

[0173] The historical operation record filtering process selects historical records from the operation sequence that share similar characteristics with the current operation record. Filtering criteria include operation type matching, location proximity, and temporal relevance. The filtering results are sorted by similarity score, and the highest-scoring historical records are selected.

[0174] The position offset difference is extracted by comparing the offset values ​​of the current operation record with those of the filtered historical operation records. The difference is calculated using the absolute value method, subtracting the offset of the current record from the offset of each historical record and taking the absolute value. The difference data constitutes a time series of offset changes.

[0175] The state transition constraints are constructed based on statistical analysis and pattern recognition of position offset differences. The construction process calculates the mean, variance, and distribution characteristics of the difference sequence to identify regular patterns in offset changes. The constraints are represented in two forms: interval constraints and trend constraints, providing prior knowledge for subsequent model adjustments.

[0176] The current process of inputting the operation record into the causal model converts the structured data in the record into a vector format that the model can process. The input process performs one-hot encoding on the operation type, coordinate normalization on the operation position, and binarization encoding on the operation result.

[0177] The application of reverse inference relationships utilizes the backward reasoning mechanism of the causal model to deduce the corresponding operation type from the current operation result. The derivation process leverages the causal mapping relationship between operations and results learned by the model, searching backward from the result state for possible operational causes. The derivation result generates a probability distribution predicting the operation type.

[0178] Where E represents the consistency deviation value, m represents the total number of operation types, and P v T represents the probability value of the predicted operation type in the v-th category. v This represents the actual label value of the operation type to be performed in the v-th category.

[0179] The overall adjustment is generated by weighted fusion of consistency deviation and state transition constraints. The fusion process uses consistency deviation as an error signal and state transition constraints as guidance for the adjustment direction. The calculation of the adjustment takes into account both the magnitude of the deviation and the strictness of the constraints.

[0180] The adjustment of the inverse derivation relationship is based on fine-tuning the internal parameters of the operational causal model using a comprehensive adjustment amount. The adjustment process employs gradient descent, using the comprehensive adjustment amount as the gradient signal of the loss function to update the model weights in the direction that reduces prediction error. The learning rate for parameter updates is dynamically adjusted according to the magnitude of the adjustment amount.

[0181] This invention achieves continuous optimization of the operational causal model through real-time feedback of operational results and accumulation of historical experience. It improves the accuracy and reliability of model predictions through state transition constraints and consistency checks, providing a continuously improving learning mechanism for the adaptive operation of intelligent browsers.

[0182] like Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of an intelligent browser adaptive operating system based on reinforcement learning provided in an embodiment of the present invention. The system includes:

[0183] The sequence acquisition module 201 is used to acquire the operation sequence of the target business process, wherein the operation sequence includes the operation type, page position coordinates and operation result;

[0184] The causal model generation module 202 is used to perform reverse generation training on the operation sequence through a multimodal large language model, establish a reverse inference relationship from the operation result to the operation type, and generate an operation causal model.

[0185] The iterative training module 203 is used to autonomously generate virtual operation sequences and predict virtual operation results by the operation causal model during non-task execution, and use virtual operation sequences with successful virtual operation results as enhancement samples to iteratively train the operation causal model.

[0186] The skill library construction module 204 is used to convert the page position coordinates of the operation sequence in which the operation result is successful into the topological relationship of page elements and store them in the skill library in association with the operation type.

[0187] Task receiving module 205 is used to receive operation tasks that include the type of operation to be performed and the current page;

[0188] The action generation module 206 is used to read the topological relationship of page elements corresponding to the type of operation to be performed from the skill library, reconstruct the operation position in the current page according to the topological relationship of page elements, and input the type of operation to be performed and the operation position into the operation causal model after iterative training to generate the execution action.

[0189] The result feedback module 207 is used to execute the execution action and obtain the current operation result, and add the current operation result to the operation sequence for subsequent iterative training.

[0190] One technical solution provided in this embodiment of the invention is an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in any of the aforementioned methods.

[0191] One technical solution provided in this embodiment of the invention is a computer-readable storage medium storing a computer program, wherein the processor executes the computer program to implement the steps in any of the aforementioned methods.

[0192] The specific embodiments described above are preferred embodiments of the present invention and are not intended to limit the specific scope of the present invention. The scope of the present invention includes, but is not limited to, these specific embodiments. All equivalent changes made in accordance with the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. A method for adaptive operation of an intelligent browser based on reinforcement learning, characterized in that, include: Obtain the operation sequence of the target business process, wherein the operation sequence includes operation type, page position coordinates, and operation result; By training the operation sequence in reverse through a multimodal large language model, a reverse inference relationship from operation result to operation type is established, and an operation causal model is generated. During non-task execution, the operational causal model autonomously generates virtual operation sequences and predicts virtual operation results. Virtual operation sequences with successful virtual operation results are used as enhancement samples to iteratively train the operational causal model. Convert the page position coordinates of successful operations in the operation sequence into page element topological relationships and store them in the skill library in association with the operation type; Receive an operation task containing the type of operation to be performed and the current page; Read the topological relationship of page elements corresponding to the type of operation to be performed from the skill library, reconstruct the operation position in the current page according to the topological relationship of page elements, and input the type of operation to be performed and the operation position into the operation causal model after iterative training to generate the execution action; The execution action is performed and the current operation result is obtained. The current operation result is added to the operation sequence for subsequent iterative training.

2. The method according to claim 1, characterized in that, By training the operation sequence in reverse using a multimodal large language model, a reverse inference relationship from operation result to operation type is established, generating an operation causal model including: The operation sequence is divided into success group and failure group according to the operation result. The operation type and page position coordinate in the success group and failure group are time-series encoded to generate success trajectory representation and failure trajectory representation. Calculate the causal difference features between the success trajectory representation and the failure trajectory representation, and use the causal difference features as a constraint for the reverse generation training of the multimodal large language model. Samples from the operation sequence that have successful operation results are input into a multimodal large language model to extract the page state features corresponding to the operation results; Based on the constraints, the page state features are decoded in reverse to generate candidate operation types and candidate page position coordinates. Candidate operation types and candidate page position coordinates that satisfy the causal difference features are selected as the reverse derivation output. Calculate the prediction error between the back-derivative output and the actual operation type and page position coordinates in the operation sequence. Minimize the prediction error through gradient backpropagation to update the parameter weights of the multimodal large language model. Establish the back-derivative relationship from operation result to operation type. Use the multimodal large language model after establishing the back-derivative relationship as the operation causal model.

3. The method according to claim 1, characterized in that, The operational causal model autonomously generates virtual operation sequences and predicts virtual operation results. Virtual operation sequences with successful results are used as boosting samples for iterative training of the operational causal model, including: The operation type and page position coordinates are extracted from the operation sequence to construct a perturbation sample pool. Random perturbation is applied to the operation type and page position coordinates in the perturbation sample pool to generate perturbation operation combinations. The perturbation operation combinations are input into the operation causal model for reverse decoding to generate virtual operation sequences. The virtual operation sequence is predicted by forward reasoning using an operational causal model. The virtual operation sequence with the result of success is compared and analyzed with the virtual operation sequence with the result of failure to extract the decision boundary features of successful operation. Based on the decision boundary features, the virtual operation sequence with the result of success is labeled to generate enhanced samples. The operational causal model is input with augmented samples, and the inverse derivation relationship of the operational causal model is adjusted based on the decision boundary features in the augmented samples. The operational causal model is then iteratively trained.

4. The method according to claim 1, characterized in that, Converting the page position coordinates of successful operations in the operation sequence into page element topological relationships and storing them in the skill library in association with the operation type includes: The page position coordinates with successful operation results in the operation sequence are sorted according to the time sequence of operation execution, and cluster analysis is performed on the sorted page position coordinates to identify spatially clustered operation regions. Convert the page position coordinates within the operation area into azimuth and distance ratios relative to the center point of the operation area, and generate a topological description of the operation area. Encode the spatial adjacency relationships between the topological descriptions of the operation areas, generate a global topological structure describing the relative positional relationships of multiple operation areas, and convert the global topological structure into the topological relationships of page elements; Semantically bind the topological relationships of page elements to the corresponding operation types in the operation sequence, establish a two-way mapping relationship from the topological relationships of page elements to the operation types and from the operation types to the topological relationships of page elements, and store the two-way mapping relationship in the skill library.

5. The method according to claim 4, characterized in that, The spatial adjacency relationships between the topological descriptions of the operation areas are encoded to generate a global topological structure describing the relative positional relationships of multiple operation areas. This global topological structure is then converted into page element topological relationships, including: Extract the center point coordinates of each operation area from the topology description of the operation area, construct a spatial adjacency graph of the operation area based on the center point coordinates, and identify the transitive adjacency paths between operation areas in the spatial adjacency graph. The path distance and path orientation in the transmitted adjacent paths are vectorized and encoded to generate a global topology containing indirect spatial relationships of the operation area. The vectorized encoding in the global topology is decomposed into direct adjacency relationships and indirect adjacency relationships. Spatial anchor points of page elements are located based on direct adjacency relationships, and hierarchical containment relationships between page elements are derived based on indirect adjacency relationships. Spatial anchor points and hierarchical containment relationships are then integrated with azimuth angles and distance ratios in the topological description of the operation area to convert them into topological relationships of page elements.

6. The method according to claim 1, characterized in that, The topological relationships of page elements corresponding to the type of operation to be performed are read from the skill library. The operation position is reconstructed on the current page based on these relationships. The operation type and position are then input into the iteratively trained operation causal model to generate the execution action, including: Read the topological relationships of page elements corresponding to the type of operation to be performed from the skill library, and extract the global topological structure recorded in the topological relationships of page elements as a topological matching template; The spatial adjacency relationship between page elements is calculated based on the position coordinates and size information of page elements in the current page to generate the current page topology. The structural similarity between the current page topology and the topology matching template is calculated, and the matching region with the highest structural similarity is located. Extract the azimuth and distance ratio from the topological description of the operation area from the topological relationship of page elements. Use the centroid of the matching area as the anchor point to determine the operation direction based on the azimuth and determine the operation distance based on the product of the distance ratio and the size of the matching area. Offset the operation distance in the operation direction to reconstruct the operation position. The operation type and operation location are input into the operation causal model after iterative training. The operation causal model derives the state features of the successful operation result from the operation type based on the reverse inference relationship. The operation location is associated with the state features to generate the operation context and the execution action is generated based on the operation context.

7. The method according to claim 1, characterized in that, Executing the execution action and obtaining the current operation result, and adding the current operation result to the operation sequence for subsequent iterative training includes: The execution action is performed, the page state before the operation and the page state after the operation are collected, the set of page element positions in the page state before the operation and the page state after the operation are extracted, the position offset between the set of page element positions is calculated, and the position offset is compared with the preset successful operation position offset range to obtain the current operation result. The operation type to be executed, operation position, current operation result, and position offset are combined to form the current operation record and added to the operation sequence; Filter historical operation records from the operation sequence, extract the position offset difference between the current operation record and the historical operation record, and construct state transition constraint relationships based on the position offset difference; The current operation record is input into the operation causal model, which then uses the reverse inference relationship to reverse infer the current operation result and generate a predicted operation type. The consistency deviation between the predicted operation type and the operation type to be executed is calculated. The consistency deviation is then fused with the state transition constraint relationship to generate a comprehensive adjustment amount. The reverse derivation relationship is adjusted based on the comprehensive adjustment amount for subsequent iterative training.

8. A reinforcement learning-based intelligent browser adaptive operating system, used to implement the method described in any one of claims 1-7, characterized in that, The system includes: The sequence acquisition module is used to acquire the operation sequence of the target business process, wherein the operation sequence includes the operation type, page position coordinates and operation result; The causal model generation module is used to reverse-generate and train operation sequences through a multimodal large language model, establish a reverse inference relationship from operation result to operation type, and generate an operation causal model. The iterative training module is used to autonomously generate virtual operation sequences and predict virtual operation results by the operation causal model during non-task execution. Virtual operation sequences with successful virtual operation results are used as enhancement samples to iteratively train the operation causal model. The skill library construction module is used to convert the page position coordinates of successful operations in the operation sequence into page element topology relationships and store them in the skill library in association with the operation type. The task receiving module is used to receive operation tasks that include the type of operation to be performed and the current page. The action generation module is used to read the topological relationship of page elements corresponding to the type of operation to be performed from the skill library, reconstruct the operation position in the current page according to the topological relationship of page elements, and input the type of operation to be performed and the operation position into the operation causal model after iterative training to generate the execution action. The result feedback module is used to execute the execution action and obtain the current operation result, and add the current operation result to the operation sequence for subsequent iterative training.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 7.

Images