A Web page intelligent perception method

Abstract

The present invention relates to the technical field of intelligent Web page layout, and particularly to an intelligent Web page perception method, which includes the following steps: obtaining the types, contents, and attributes of Dom elements of the current Web page, and performing intelligent analysis on the collected data by using an AI adaptation algorithm; cleaning the collected data; generating target task pairs; generating text embeddings for the target text content by using an Embedding vector model; performing Lora task fine-tuning on a large language model; embedding the fine-tuned large language model into an AI workflow; realizing dynamic routing recognition and logical control of multiple Web pages through an automated workflow; calculating the similarity of Web elements by using a multimodal vision model; outputting the calculation results and the coordinate positioning of the elements; the present invention enables the system to more intelligently process complex Web page elements and user interactions; by predicting and adapting to user behaviors, the system can operate more stably and reliably, reducing faults caused by human errors or system inadaptation.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent Web page layout preparation, and particularly to a method for intelligent Web page perception. Background Art

[0002] With the rapid development of Internet technology, Web technology has become the main platform for people to obtain information and interact. Web intelligent perception technology, as an emerging technology, aims to locate website elements and automate execution by analyzing Web page structure data. Currently, the development of big data and machine learning technologies provides strong technical support for Web intelligent perception.

[0003] Currently, with the rapid development of Internet technology, the complexity and dynamics of Web pages continue to increase; in traditional Web development, testing, and operation, in the face of page changes or function iterations, it is often necessary to manually adjust the operation process or re-develop adaptation scripts, which is inefficient and costly.

[0004] Therefore, in view of the problem that in traditional Web development, testing, and operation, in the face of page changes or function iterations, it is often necessary to manually adjust the operation process or re-develop adaptation scripts, which is inefficient and costly, a method for intelligent Web page perception can be designed. Summary of the Invention

[0005] In order to overcome the problem that in traditional Web development, testing, and operation, in the face of page changes or function iterations, it is often necessary to manually adjust the operation process or re-develop adaptation scripts, which is inefficient and costly.

[0006] The technical solution of the present invention is: a method for intelligent Web page perception, including the following steps:

[0007] Step 1: AI Adaptive Intelligent Analysis

[0008] 1.1 Obtain the type, content, and attributes of the Dom elements of the current Web page;

[0009] 1.2 Obtain the top-level node of the DOM tree of the page and perform a depth traversal;

[0010] 1.3 Extract page elements intelligently in multiple dimensions;

[0011] 1.4 Preprocess the collected page data;

[0012] 1.5 Use the AI adaptation algorithm to perform intelligent analysis on the collected data;

[0013] Step 2: Large Model Dynamic Routing Strategy

[0014] 2.1 Clean the collected data;

[0015] 2.2 Generate target task pairs and perform data augmentation;

[0016] 2.3 Use the Embedding vector model to generate text embeddings for the target text content;

[0017] 2.4 Perform Lora task fine-tuning on the large language model to improve the routing ability of the model on target tasks;

[0018] 2.5 Embed the fine-tuned large language model into the AI workflow;

[0019] 2.6 Connect to user input and perform vector transformation and semantic analysis;

[0020] 2.7 Implement dynamic routing recognition and logic control of multiple Web pages through an automated workflow;

[0021] Step 3: Multimodal Visual Perception

[0022] 3.1 Input the visual features, text embeddings, and structured features extracted in Step 1 into the multimodal visual model;

[0023] 3.2 Through the multimodal model, combine visual and text features and perform deep fusion;

[0024] 3.3 Collect visual data of target tasks during the automated perception process;

[0025] 3.4 Use the multimodal visual model to calculate the similarity of Web elements;

[0026] 3.5 Output the calculation results and the coordinate positioning of the elements.

[0027] Preferably, use the AI adaptation algorithm to perform intelligent analysis on the collected data, including full strategy matching, Dom content matching, Dom selector matching, Path path matching based on the Dom tree, and analyzing the path of the target element in the DOM tree for the target element.

[0028] Preferably, full strategy matching can intelligently select the most suitable matching strategy for the current page structure by deeply analyzing the overall layout of the web page and the mutual relationship between elements.

[0029] Preferably, Dom content matching is based on the characteristics of element content, and the algorithm can perform intelligent recognition and matching.

[0030] Preferably, Dom selector matching uses the powerful function of CSS selectors, and the algorithm can accurately locate specific elements on the page.

[0031] Preferably, the Path path matching based on the Dom tree analyzes the path of an element in the DOM tree.

[0032] Preferably, by analyzing the path algorithm of an element in the DOM tree, the layout and display mode of the element can be intelligently judged.

[0033] Preferably, multimodal visual perception includes page screenshot acquisition and multimodal image algorithms. Page screenshot acquisition is to collect the images of the page in real time, and the multimodal image algorithm combines image recognition technology and machine learning algorithms.

[0034] Preferably, the large model dynamic routing strategy is the large model data fine-tuning technology, the AI large model dynamic routing algorithm, and the AI model monitoring and behavior prediction. The large model data fine-tuning technology trains the AI model to identify and process Web dynamic page elements. When the AI large model dynamic routing algorithm is running, the algorithm dynamically adjusts the element matching strategy according to the output of the AI model. The AI model monitoring and behavior prediction monitors the usage of page elements and user behaviors, and the algorithm can dynamically adjust the routing to optimize the user experience.

[0035] Advantages of the present invention: Through the AI adaptive intelligent analysis algorithm, the system can quickly adapt to system upgrades and function changes without manually reconstructing the process, thus significantly improving the response speed to business changes; The combination of multimodal visual perception technology and AI algorithms enhances the ability to identify changes in Web page elements, enabling the system to more accurately capture and respond to changes in interface elements; The AI large model dynamic routing strategy can dynamically adjust the routing according to user behaviors and the usage of page elements, optimize the user experience, and improve business efficiency; It reduces the repeated configuration required due to system changes, thereby reducing the waste of time and resources and reducing the operating costs of enterprises; Through self-learning and adaptive capabilities, the system can automatically adjust to adapt to different web designs and structures, reducing the need for manual intervention; The technical solution supports fine-tuning of business processes, making the system more flexible and able to quickly adapt to changes in market and business requirements; Through precise element matching and path recognition, the system can more accurately locate and operate Web page elements, reducing the possibility of incorrect operations; The integrated AI technology enhances the overall intelligence level of the system, enabling the system to more intelligently process complex Web page elements and user interactions; By predicting and adapting to user behaviors, the system can operate more stably and reliably, reducing failures caused by human errors or system inadaptability. Description of the Drawings

[0036] Figure 1 Shown is a schematic diagram of the Web page intelligent perception process of a Web page intelligent perception method of the present invention;

[0037] Figure 2The figure shows a schematic diagram of the Web page intelligent perception principle in a Web page intelligent perception method of the present invention. Detailed implementation manners

[0038] The present invention will be further described below in conjunction with embodiments.

[0039] Please refer to Figure 1-2 , obtain the types, contents and attributes of the Dom elements of the current Web page; obtain the top-level node of the DOM tree of the page and perform a depth traversal; extract page elements intelligently in multiple dimensions; preprocess the collected page data; use an AI adaptation algorithm to perform intelligent analysis on the collected data; a method of dynamically selecting the best matching strategy by globally analyzing the page layout and the relationship between elements significantly improves the matching efficiency and accuracy; based on the characteristics of the element content, the algorithm can perform intelligent recognition and matching, and even in the absence of a clear selector, the algorithm can also identify and match elements through the semantic information of the content; by leveraging the powerful function of CSS selectors, the algorithm can accurately locate specific elements on the page, and this method is particularly effective when dealing with elements with clear CSS classes or IDs; by combining DOM path parsing and adjacent element relationship analysis, precise positioning and layout optimization of deeply nested elements are achieved, and this multi-dimensional analysis strategy is unique; comprehensively applying content, selector, path and adjacent element matching strategies, a multi-level and multi-dimensional intelligent matching framework is constructed to adapt to complex page structures; clean the collected data; generate target task pairs and perform data augmentation; use an Embedding vector model to generate text embeddings for the target text content; perform Lora task fine-tuning on the large language model to improve the routing ability of the model on the target task; embed the fine-tuned large language model in the AI workflow, use the AI large model to monitor the page state in real time, and dynamically adjust the page element matching strategy to ensure that the system can adapt to changes in complex scenarios; access user input and perform vector transformation and semantic analysis; model user behavior through machine learning and dynamically optimize the layout and routing strategies of page elements. The dynamic adjustment mechanism centered on user experience can significantly improve the interaction efficiency; input the extracted visual features, text embeddings and structured features into a multi-modal visual model; through real-time screenshot collection and multi-modal analysis, deeply integrate traditional text analysis and image recognition technologies to provide more comprehensive information support for page element matching; collect target task visual data during the automated perception process; based on the combination of image recognition and machine learning, elements can be extracted and analyzed from the page screenshot, and this way of directly extracting the page structure from visual data is an important breakthrough in technology, especially suitable for dynamic or non-fixed structure pages; output the calculation results and the coordinate positions of the elements.

[0040] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those skilled in the art.

Claims

1. A Web page intelligent perception method, comprising the following steps: Step 1: AI Adaptive Intelligent Analysis 1.1 Obtain the types, contents, and attributes of the Dom elements of the current Web page; 1.2 Obtain the top-level node of the DOM tree of the page and perform a depth traversal; 1.3 Extract page elements intelligently in multiple dimensions; 1.4 Preprocess the collected page data; 1.5 Use the AI adaptation algorithm to perform intelligent analysis on the collected data; Step 2: Large Model Dynamic Routing Strategy 2.1 Clean the collected data; 2.2 Generate target task pairs and perform data augmentation; 2.3 Use the Embedding vector model to generate text embeddings for the target text content; 2.4 Perform Lora task fine-tuning on the large language model to improve the routing ability of the model on the target task; 2.5 Embed the fine-tuned large language model into the AI workflow; 2.6 Connect to user input and perform vector conversion and semantic analysis; 2.7 Implement dynamic routing recognition and logic control of multiple Web pages through an automated workflow; Step 3: Multimodal Visual Perception 3.1 Input the visual features, text embeddings, and structured features extracted in Step 1 into the multimodal visual model; 3.2 Through the multimodal model, combine visual and text features and perform deep fusion; 3.3 Collect target task visual data during the automated perception process; 3.4 Use the multimodal visual model to calculate the similarity of Web elements; 3.5 Output the calculation results and the coordinate positioning of the elements.

2. The Web page intelligent perception method according to claim 1, wherein: Using the AI adaptation algorithm to perform intelligent analysis on the collected data includes full strategy matching, Dom content matching, Dom selector matching, Path path matching based on the Dom tree, and for target elements by analyzing the path of the element in the DOM tree.

3. The Web page intelligent perception method according to claim 1, characterized in that: Full strategy matching is to intelligently select the most suitable matching strategy for the current page structure by deeply analyzing the overall layout of the web page and the mutual relationship between elements.

4. The Web page intelligent perception method according to claim 1, wherein: Dom content matching is based on the characteristics of element content, and the algorithm can perform intelligent recognition and matching.

5. The Web page intelligent perception method according to claim 1, wherein: Dom selector matching uses the powerful function of CSS selectors, and the algorithm can accurately locate specific elements on the page.

6. The Web page intelligent perception method according to claim 1, wherein: Path path matching based on the Dom tree is by analyzing the path of the element in the DOM tree.

7. The Web page intelligent perception method according to claim 1, characterized in that: Through analyzing the path of the element in the DOM tree, the algorithm can intelligently judge the layout and display method of the element.

8. The Web page intelligent perception method according to claim 1, wherein: Multimodal visual perception includes page screenshot collection and multimodal image algorithms. Page screenshot collection is to collect the image of the page in real time, and the multimodal image algorithm is to combine image recognition technology and machine learning algorithms.

9. The Web page intelligent perception method according to claim 1, wherein: The large model dynamic routing strategy is the large model data fine-tuning technology, the AI large model dynamic routing algorithm, and the AI model monitoring and behavior prediction. The large model data fine-tuning technology trains the AI model to identify and process Web dynamic page elements. When the AI large model dynamic routing algorithm is running, the algorithm dynamically adjusts the element matching strategy according to the output of the AI model. The AI model monitoring and behavior prediction can dynamically adjust the routing to optimize the user experience by monitoring the usage of page elements and user behavior.

Images