Large-screen webpage control method and device, electronic equipment and medium

Abstract

The invention provides a large-screen webpage control method and device, electronic equipment and a medium, and the method comprises the steps: obtaining control intention information in a webpage control instruction and an original document object set of an original large-screen webpage in response to the webpage control instruction of a user, identifying webpage operation information from the control intention information by adopting a pre-trained language model, determining a non-display object in an original document object set of the original large-screen webpage, and removing the non-display object from the original document object set to obtain a target document object set, generating a webpage operation instruction according to the webpage operation information and the target document object set, generating a target large-screen webpage in response to the webpage operation instruction, and displaying the target large-screen webpage to the user. According to the method, the document object set of the webpage is effectively simplified, the webpage structure can be analyzed more accurately and quickly, the webpage operation instruction set can be generated without manual intervention, and the efficiency of obtaining the large-screen webpage operation instruction set is improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method, device, electronic device and medium for controlling a large-screen web page. Background Art

[0002] In today's era of information explosion, the Internet has become the main platform for people to obtain information, communicate online and conduct business. Large-screen web pages are rich in content and complex in hierarchy. Manual operation of large-screen web pages is difficult and slow. Through intelligent web interaction systems such as smart workbenches, intelligent page management can be achieved, and the displayed page content can be quickly updated and switched to ensure the timeliness and accuracy of information, while improving user participation and experience.

[0003] However, when processing actual web pages, most existing web intelligent interaction systems need to manually parse the large-screen web page production operation instruction set, which is time-consuming and labor-intensive. Once the page changes, the operation instruction set needs to be updated, which has high maintenance costs. In addition, the length of the token (string) of a web page with rich content is usually over 10,000. The diversity and complexity of web pages and their tendency to be lengthy in wording pose great challenges to the LLM (Large Language Model) in understanding the content and performing correct operations. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application propose a method, device, electronic device and medium for controlling a large-screen web page.

[0005] In a first aspect of the implementation of the present application, a method for controlling a large-screen webpage is first provided, wherein the method comprises:

[0006] In response to a webpage control instruction of a user, obtaining control intent information in the webpage control instruction and an original document object set of an original large-screen webpage;

[0007] Using a pre-trained language model to identify web page operation information from the control intention information;

[0008] Determine non-display objects in the original document object set of the original large-screen webpage, and remove the non-display objects from the original document object set to obtain a target document object set;

[0009] Generate a web page operation instruction according to the web page operation information and the target document object set;

[0010] A target large-screen webpage is generated in response to the webpage operation instruction, and the target large-screen webpage is displayed to the user.

[0011] Optionally, the non-display object includes at least one of a functional object, a blank object, and a redundant object, and the determining the non-display object in the original document object set of the original large-screen webpage and removing the non-display object from the original document object set to obtain a target document object set includes:

[0012] Determine the functional object in the original document object set, and remove the functional object from the original document object set to obtain the target document object set; the functional object is an object that is irrelevant to the content display of the large-screen webpage;

[0013] and / or, determining the blank objects in the original document object set, and removing the blank objects from the original document object set to obtain the target document object set; the blank objects are objects corresponding to blank characters in the large-screen web page;

[0014] And / or, determining redundant objects in the original document object set, and removing the redundant objects from the original document object set to obtain the target document object set; the redundant objects are objects that are not used by the large-screen web page.

[0015] Optionally, the language model is trained in the following manner:

[0016] Acquire training data and test data; the training data includes training operation information and training intention information; the test data includes test operation information and test intention information;

[0017] Using the training intention information as the input of the language model, using the training operation information as the output of the language model, and using the training data to train the language model to obtain a language model to be verified;

[0018] Inputting the test intention information into the language model to be verified, and obtaining the operation information to be verified output by the language model to be verified;

[0019] Determine a loss function corresponding to the language model according to the test operation information and the operation information to be verified;

[0020] The loss function corresponding to the language model is used to adjust the model parameters of the language model to be verified, and the steps of taking the training intention information as the input of the language model, taking the training operation information as the output of the language model, and using the training data to train the language model are re-executed until a preset stopping condition is reached to obtain a trained language model.

[0021] Optionally, the acquiring of training data and test data includes:

[0022] Get the sample large screen web page;

[0023] The sample large-screen web page is parsed using a preset web page parsing tool to obtain operable objects, operable object types and operable object codes in the sample large-screen web page;

[0024] Obtain at least one sample operation instruction in the sample large-screen webpage and a sample operation instruction definition corresponding to the sample operation instruction;

[0025] Generate at least one sample task using the sample operation instruction definition, the operable object and the operable object type, and obtain intent information corresponding to the sample task;

[0026] Dividing the intention information corresponding to the sample task into the training intention information and the test intention information;

[0027] Determine the operation information corresponding to the sample task according to the sample operation instruction corresponding to the sample operation instruction definition and the operable object code corresponding to the operable object;

[0028] Dividing the operation information corresponding to the sample task into the training operation information and the test operation information;

[0029] Using the training intention information and the training operation information as the training data;

[0030] The test intention information and the test operation information are used as the test data.

[0031] Optionally, the acquiring of training data and test data further includes:

[0032] Get several sample large-screen web pages;

[0033] Using a preset task generation tool to generate at least one sample task for any of the sample large-screen web pages;

[0034] In the process of executing the sample task, a preset operation recording tool is used to record the operation information sequence generated by executing the sample task;

[0035] Dividing the operation information sequence into the training operation information and the test operation information;

[0036] For any operation information in the operation information sequence, determine the intention information corresponding to the operation information, and obtain the intention information sequence corresponding to the operation information sequence;

[0037] Dividing the intention information sequence corresponding to the operation information sequence into the training intention information and the test intention information;

[0038] Using the training intention information and the training operation information as the training data;

[0039] The test intention information and the test operation information are used as the test data.

[0040] Optionally, after obtaining the trained language model, the method further includes:

[0041] Acquire a number of the operation information sequences and a number of intention information sequences corresponding to the operation information sequences;

[0042] Determine positive sample data and negative sample data according to the operation information sequence and the intention information sequence;

[0043] Using the operation information sequence in the positive sample data as positive sample training operation information, and using the intention information sequence in the positive sample data as positive sample training intention information;

[0044] Using the operation information sequence in the negative sample data as negative sample training operation information, and using the intention information sequence in the negative sample data as negative sample training intention information;

[0045] Using the positive sample training operation information, the positive sample training intention information, the negative sample training operation information, and the negative sample training intention information as the training data;

[0046] Re-execute the steps of taking the positive sample training intention information or the negative sample training intention information as the input of the language model, taking the positive sample training operation information or the negative sample training operation information as the output of the language model, and using the training data to train the language model until a preset stopping condition is reached, thereby obtaining a trained language model.

[0047] Optionally, the method further comprises:

[0048] Determine the webpage operation instruction to be verified according to the operation information to be verified and the target document object set, and operate the large-screen webpage according to the webpage operation instruction to be verified to obtain the operation result to be verified;

[0049] Determine a reward signal for the operation information to be verified according to a preset reward function or the operation result to be verified;

[0050] The model parameters of the language model to be verified are adjusted according to the reward signal, and the steps of taking the training intention information as the input of the language model, taking the training operation information as the output of the language model, and training the language model using the training data are re-executed until a preset stopping condition is reached, thereby obtaining a trained language model.

[0051] In a second aspect of the implementation of the present application, a control device for a large-screen webpage is also provided, characterized in that the device comprises:

[0052] An information acquisition module, configured to respond to a webpage control instruction of a user and acquire control intent information in the webpage control instruction and an original document object set of an original large-screen webpage;

[0053] An intention recognition module, used to recognize web page operation information from the control intention information using a pre-trained language model;

[0054] A webpage simplification module, configured to determine non-display objects in the original document object set of the original large-screen webpage, and remove the non-display objects from the original document object set to obtain a target document object set;

[0055] An operation instruction acquisition module, used to generate a webpage operation instruction according to the webpage operation information and the target document object set;

[0056] The webpage control module is used to generate a target large-screen webpage in response to the webpage operation instruction, and display the target large-screen webpage to the user.

[0057] In the third aspect of the implementation of the present application, an electronic device is also provided, characterized in that it includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, it implements the control method of the large-screen web page as described above.

[0058] In a fourth aspect of the implementation of the present application, a computer-readable storage medium is further provided, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the control method of the large-screen web page as described above is implemented.

[0059] The embodiments of the present application have the following advantages:

[0060] In an embodiment of the present application, in response to a user's web page control instruction, by obtaining the control intent information in the web page control instruction and the original document object set of the original large-screen web page, a pre-trained language model is used to identify the web page operation information from the control intent information, determine the non-display objects in the original document object set of the original large-screen web page, and remove the non-display objects in the original document object set to obtain a target document object set, generate a web page operation instruction according to the web page operation information and the target document object set, generate a target large-screen web page in response to the web page operation instruction, and display the target large-screen web page to the user. The present application effectively simplifies the document object set of the web page, allowing the large model to analyze the web page structure more accurately and quickly, and the present application can generate a web page operation instruction set without human intervention, thereby improving the efficiency of obtaining the large-screen web page operation instruction set and ensuring the accuracy of the large-screen web page operation instruction set. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0062] Figure 1 This is a flowchart of a method for controlling a large-screen web page provided by an embodiment of the present application;

[0063] Figure 2 This is a simplified flow chart of an original web page document collection provided by an embodiment of the present application;

[0064] Figure 3 is a schematic diagram of a process for obtaining training data provided by an embodiment of the present application;

[0065] Figure 4 It is a schematic diagram of a process of data interaction performed by a webpage interactive agent provided in an embodiment of the present application;

[0066] Figure 5 It is a structural schematic diagram of a large-screen web page control device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0067] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in each embodiment of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not constitute any limitation, and the various embodiments can be combined with each other and referenced to each other without contradiction.

[0068] Based on the problems in the related art, this application develops a web page interactive agent based on a large language model, using an HTML (HyperText Markup Language) simplification algorithm inspired by human browsing patterns to represent web pages and retain key information. In addition, this application constructs a web browsing dataset for model training based on a method of collaboration between humans and artificial intelligence. In addition, this application uses curriculum learning + reinforcement learning + rejection sampling fine-tuning methods to guide the model, promote the effective decomposition of web page understanding, browser operations and related tasks, and achieve high-precision page interaction.

[0069] First, this application takes large-screen intelligent interaction as the business scenario and proposes an HTML simplification algorithm inspired by human browsing patterns to extract key information while removing redundant or interfering elements that may hinder model understanding, and efficiently converts the original element tree into a concise representation.

[0070] It should be noted that HTML (HyperText Markup Language) is a hypertext markup language, which is a standard markup language for creating web pages. In the embodiment of the present application, the analysis and simplification of the web page can be achieved by analyzing the HTML of the large-screen web page.

[0071] LLM (Large Language Model) is a large language model, which is an artificial intelligence model that can understand and generate human language. Its typical feature is that it is large in scale and contains billions of parameters. In the embodiment of the present application, the analysis of the document object model can be realized by LLM.

[0072] DOM (Document Object Model): The Document Object Model is a cross-platform and language-independent interface that allows programs and scripts to dynamically access and update the content, structure, and style of a document. In web page development, DOM usually refers to the object model of an HTML or XML document. In the embodiments of the present application, the difficulty of analyzing a web page can be reduced by simplifying the DOM tree, that is, simplifying the document object set.

[0073] Reference Figure 1 , showing a method for controlling a large-screen web page provided by an embodiment of the present application.

[0074] It should be noted that the control method of the large-screen web page in the present application can be applied to a web page interactive agent (WebAgent), and the web page interactive agent can be combined with different objects to obtain different web page interaction methods.

[0075] The web interactive intelligent agent can be combined with the large-screen control robot to form a large-screen intelligent interactive robot, which can reshape the existing large-screen interaction mode, assist or replace the tour guide in page switching, data retrieval and other tasks, and give voice commands to achieve automatic control of the large screen and realize high-level intelligent interaction.

[0076] Web interactive intelligent entities can be combined with digital employees to form standardized intelligent interactive tools. On the one hand, they can enable cloud network self-intelligence, realize automatic control of web pages / systems, and improve the level of digitalization in the work process. On the other hand, standardized intelligent interactive tools can also directly provide relevant services to external customers.

[0077] The method specifically comprises the following steps:

[0078] S101: In response to a webpage control instruction of a user, control intention information in the webpage control instruction and an original document object set of an original large-screen webpage are acquired.

[0079] It should be noted that a large-screen web page generally refers to a web page displayed on a larger screen device. In an embodiment of the present application, a web page control instruction can be obtained to control the large-screen web page.

[0080] In an embodiment of the present application, in response to a web page control instruction from a user, control intent information in the web page control instruction and an original document object set of the original large-screen web page may be acquired.

[0081] The webpage control instruction may be in the form of voice or text. The control intention information may be the operation goal that the user wants to achieve based on the target webpage. The document object set may be the document object tree of the large-screen webpage.

[0082] In the specific implementation, the user can input the operation target to be achieved based on the target web page through text or voice. You can also use HTML parser to parse the large-screen web page, such as BeautifulSoup, lxml, HTML5Parser, etc., to obtain the original DOM element tree.

[0083] It should be noted that BeautifulSoup, lxml and HTML5Parser are all Python libraries for parsing HTML and XML documents. They can help developers extract data from web pages, modify web page content or perform web page analysis. Among them, LXML (Libraries for XML Processing): is a library for processing XML and HTML data, which provides the functions of parsing, modifying and creating these documents in the Python programming language. In the embodiment of the present application, large-screen web pages can be parsed by these HTML parsers.

[0084] S102: using a pre-trained language model to identify web page operation information from the control intention information.

[0085] In the embodiment of the present application, a pre-trained language model may be used to identify web page operation information from control intention information, wherein the web page operation information may be one or more operation instructions.

[0086] In an embodiment of the present application, the web page interactive agent can determine the user's intention based on the input information and generate a corresponding set of operation instructions in combination with the simplified web page information.

[0087] S103: Determine non-display objects in the original document object set of the original large-screen webpage, and remove the non-display objects from the original document object set to obtain a target document object set.

[0088] In an embodiment of the present application, non-display objects in the original document object set of the original large-screen webpage can be determined, and the non-display objects can be removed from the original document object set to obtain a target document object set. The non-display objects can be objects that are not directly displayed in the large-screen webpage.

[0089] S104: Generate a web page operation instruction according to the web page operation information and the target document object set.

[0090] In the embodiment of the present application, a webpage operation instruction can be generated according to the webpage operation information and the target document object set, wherein the webpage operation information can be one or more operation instructions for the current webpage large screen.

[0091] In a specific implementation, web page operation instructions for the current web page large screen can be generated based on the content of the target document object set and combined with the web page operation information.

[0092] S105: generating a target large-screen webpage in response to the webpage operation instruction, and displaying the target large-screen webpage to the user.

[0093] In the embodiment of the present application, a target large-screen webpage can be generated in response to a webpage operation instruction, and the target large-screen webpage can be displayed to the user. In which, generating a target large-screen webpage in response to a webpage operation instruction can mean that after obtaining the webpage operation instruction, the webpage interactive agent controls the original large-screen webpage to gradually execute the webpage operation instruction, and after the execution is completed, the target large-screen webpage is formed.

[0094] In a specific implementation, the operation instruction set can be gradually executed on the target web page based on Selenium to achieve the user's interaction goal.

[0095] It should be noted that Selenium is an open source tool for automating browser operations, and can also be understood as an automated testing tool set and library for Web applications. Selenium supports multiple programming languages, allowing developers to write scripts to simulate user operations on Web applications to test the functions, performance, and user interface of the application. In the embodiment of the present application, Selenium can be used to achieve step-by-step control of the original large-screen web page.

[0096] In an optional embodiment of the present application, the non-display object includes at least one of a functional object, a blank object, and a redundant object, and step S103 further includes the following sub-steps:

[0097] S11: Determine the functional object in the original document object set, and remove the functional object from the original document object set to obtain the target document object set; the functional object is an object that is irrelevant to the content display of the large-screen webpage.

[0098] S12: and / or, determining the blank objects in the original document object set, and removing the blank objects in the original document object set to obtain the target document object set; the blank objects are objects corresponding to blank characters in the large-screen web page.

[0099] S13: and / or, determining redundant objects in the original document object set, and removing the redundant objects from the original document object set to obtain the target document object set; the redundant objects are objects that are not used by the large-screen web page.

[0100] In the embodiment of the present application, the functional objects in the original document object set can be determined, and the functional objects can be removed from the original document object set to obtain the target document object set. The functional objects are objects that are irrelevant to the content display of the large-screen webpage.

[0101] And / or, blank objects in the original document object set may be determined, and the blank objects may be removed from the original document object set to obtain the target document object set, wherein the blank objects are objects corresponding to blank characters in the large-screen web page.

[0102] And / or, redundant objects in the original document object set may be determined, and the redundant objects may be removed from the original document object set to obtain the target document object set, wherein the redundant objects are objects that are not used by the large-screen web page.

[0103] In the specific implementation, you can refer to Figure 2 Simplify the original document object collection, refer to Figure 2 , showing a simplified flow chart of an original web page document collection provided by an embodiment of the present application.

[0104] You can first use an HTML parser to parse the large-screen web page, such as BeautifulSoup, lxml, HTML5Parser, etc., to obtain the original DOM tree, that is, the original web page document collection.

[0105] Then, you can remove elements from the DOM tree that are not directly involved in displaying the page content. These elements are often used to add styles, scripts, or link to external resources to the page, but they themselves do not contain content that the user sees directly in the browser, such as <script>、<style>、<link>等。在本申请实施例中，不直接参与页面内容展示的元素可以是上述功能对象。

[0106] 可以移除DOM树中的空白文本节点以及属性为空的节点，移除多余的空白字符，如制表符、换行符等。在本申请实施例中，空白文本节点以及属性为空的节点可以是上述空白对象。

[0107] 最后，可以清理不必要的属性。有些属性可能是在页面生成过程中自动添加的，或者是在编辑过程中留下的，但实际上并没有被页面上的脚本或样式表所使用。移除这些不必要的属性对页面功能和样式没有影响，同时可以减少DOM元素树的横向宽度。在本申请实施例中，实际上并没有被页面上的脚本或样式表所使用的属性可以是上述冗余对象。

[0108] 对DOM元素树进行多步骤的过滤后，获得简化后的DOM元素树，大大降低大模型的理解和规划难度。

[0109] 面对内容丰富且层级复杂的大屏网页，其token长度通常过万，网页的多样性和复杂性给LLM理解内容和执行正确操作带来巨大挑战，现有基于LLM的方案往往准确度较差，无法实际应用。

[0110] 本申请通过模拟人类用户在浏览网页时的行为和偏好，优化网页内容和结构，去除不必要的标签和属性(如脚本、样式和注释等)，在保持网页重要内容完整性的前提下，实现页面DOM树的有效简化，从而使大模型能够更准确更迅速地分析网页结构。该算法以大屏智能交互为业务场景，受人类浏览模式启发，提取关键信息并去除冗余或干扰性元素，高效地将原始元素树转换为简洁的表示形式，显著提高大屏交互的准确性。

[0111] 在本申请的一种可选实施例中，所述语言模型采用如下方式训练：

[0112] S21：获取训练数据和测试数据；所述训练数据包括训练操作信息和训练意图信息；所述测试数据包括测试操作信息和测试意图信息；

[0113] S22：将所述训练意图信息作为所述语言模型的输入，将所述训练操作信息作为所述语言模型的输出，采用所述训练数据对所述语言模型进行训练，得到待验证的语言模型；

[0114] S23：将所述测试意图信息输入所述待验证的语言模型，得到所述待验证的语言模型输出的待验证操作信息；

[0115] S24：根据所述测试操作信息和所述待验证操作信息确定所述语言模型对应的损失函数；

[0116] S25：采用所述语言模型对应的损失函数对所述待验证的语言模型进行模型参数调整，并重新执行将所述训练意图信息作为所述语言模型的输入，将所述训练操作信息作为所述语言模型的输出，采用所述训练数据对所述语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。其中，停止条件可以根据实际需求自行设置。

[0117] 在本申请实施例中，可以获取训练数据和测试数据；训练数据包括训练操作信息和训练意图信息；测试数据包括测试操作信息和测试意图信息。然后，可以将训练意图信息作为语言模型的输入，将训练操作信息作为语言模型的输出，采用训练数据对语言模型进行训练，得到待验证的语言模型。其次，可以将测试意图信息输入待验证的语言模型，得到待验证的语言模型输出的待验证操作信息。再次，可以根据测试操作信息和待验证操作信息确定语言模型对应的损失函数。最后，可以采用语言模型对应的损失函数对待验证的语言模型进行模型参数调整，并重新执行将训练意图信息作为语言模型的输入，将训练操作信息作为语言模型的输出，采用训练数据对语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。

[0118] 在具体实现中，可以通过课程学习算法(CL)增强模型对网页的理解，以及作为agent在环境中执行操作的能力。可以使用Supervised Fine-Tuning(SFT)监督微调的方法训练基础大模型通义千问Qwen-7B，借鉴课程学习的思想，模仿人类的学习过程，使模型从简单的样本开始学习，逐渐进步到复杂的样本。其中，CL(Curriculum Learning，课程学习算法)，可以是一种机器学习策略，它模仿人类教育中的教学原则，即先教授简单概念再逐渐过渡到更复杂的概念。其中，SFT(Supervised Fine-Tuning，监督微调)，是一种机器学习中的技术，可以用于改进预训练模型以适应特定的下游任务。

[0119] 本申请可以基于已构建数据集的微调，让大模型能够先学会理解页面和基本操作，然后学会简单推理和设计页面控制指令集合。

[0120] 在本申请实施例中，可以采用以下公式计算损失函数：

[0121]

[0122] 是SFT微调的损失函数，π表示策略，θ表示模型参数，πθ可以理解为给定参数θ的策略或模型，logπθ(y∣x)是模型πθ对给定输入x生成输出y的对数概率。是期望值符号，表示对数据集中的所有(x,y)对进行平均。本申请通过迭代训练最小化损失函数，可以找到一组最优的模型参数。

[0123] 本申请结合课程学习算法(CL)增强模型对网页的理解和执行操作的能力，可以显著提高模型在复杂网页环境中的准确性和效率。

[0124] 在本申请的一种可选实施例中，步骤S21还包括以下子步骤：

[0125] S31：获取样本大屏网页；

[0126] S32：采用预设的网页解析工具对所述样本大屏网页进行解析，得到所述样本大屏网页中的可操作对象、可操作对象类型和可操作对象编码。

[0127] 在本申请实施例中，可以获取样本大屏网页，采用预设的网页解析工具对样本大屏网页进行解析，得到样本大屏网页中的可操作对象、可操作对象类型和可操作对象编码。

[0128] 在具体实现中，可以参照图3，图3示出了本申请一实施例提供的获取训练数据的流程示意图，在此步骤中，可以基于HTML解析器对大屏网页进行解析得到DOM元素树，获取每个网页中的可操作元素、元素类型及元素ID等信息。

[0129] S33：获取所述样本大屏网页中的至少一条样本操作指令和所述样本操作指令对应的样本操作指令定义。

[0130] 在本申请实施例中，可以获取样本大屏网页中的至少一条样本操作指令和样本操作指令对应的样本操作指令定义。

[0131] 在具体实现中，可以参照图3，图3示出了本申请一实施例提供的获取训练数据的流程示意图，在此步骤中，可以构建操作指令列表，操作指令列表包含常规大屏页面所涉及的所有操作指令及每条指令对应的定义。操作指令及每条指令对应的定义如下表1。

[0132] 表1操作指令定义对照表

[0133] 操作指令定义点击模拟鼠标点击元素键入模拟键盘输入文本滚动模拟页面滚动选择模拟下拉框选择元素悬浮模拟鼠标悬浮在元素上旋转模拟跳转到某页面

[0134] S34：采用所述样本操作指令定义、所述可操作对象和所述可操作对象类型生成至少一个样本任务，并获取所述样本任务对应的意图信息。

[0135] S35：将所述样本任务对应的意图信息划分为所述训练意图信息和所述测试意图信息；

[0136] S36：根据所述样本操作指令定义对应的所述样本操作指令和所述可操作对象对应的所述可操作对象编码确定所述样本任务对应的操作信息；

[0137] S37：将所述样本任务对应的操作信息划分为所述训练操作信息和所述测试操作信息；

[0138] S38：将所述训练意图信息和所述训练操作信息作为所述训练数据；

[0139] S39：将所述测试意图信息和所述测试操作信息作为所述测试数据。

[0140] 在本申请实施例中，可以采用样本操作指令定义、可操作对象和可操作对象类型生成至少一个样本任务，并获取样本任务对应的意图信息。将样本任务对应的意图信息划分为训练意图信息和测试意图信息。根据样本操作指令定义对应的样本操作指令和可操作对象对应的可操作对象编码确定样本任务对应的操作信息，将样本任务对应的操作信息划分为训练操作信息和测试操作信息。将训练意图信息和训练操作信息作为训练数据，将测试意图信息和测试操作信息作为测试数据。

[0141] 在具体实现中，可以参照图3，图3示出了本申请一实施例提供的获取训练数据的流程示意图，在此步骤中，对于结构固定的操作，如点击和滚动，定义好模板后可以使用Chatito(生成自然语言处理(NLP)训练数据的工具)直接构建相应任务及操作意图，高效快捷，基于设定的变量进行排列组合，同一种意图创建多种问法，以提高模型的鲁棒性。对于结构相对灵活的操作，如下拉框选择，则可以使用大模型GPT-4o生成相应任务及操作意图。对每个操作指令定义一个操作函数，输入不同的元素ID即可操作各个元素，最终得到了相关任务、操作意图和操作指令集的对应关系，这种方法确保了指令的可执行性，并提供了操作任务的丰富性。

[0142] 在具体实现中，可以参照如下方式使用Chatito直接构建相应任务及操作意图：

[0143] ％[clickElement]('training':*100','testing':*100')

[0144] *[60％]～[hi？]～[please？]～[click？]@[element]

[0145] *[40％]@[element]～[located at？]

[0146] @[element]

[0147] ～[元素1]

[0148] ～[元素2]

[0149] ～[元素3]

[0150] ～[click]

[0151] 点击

[0152] 打开

[0153] 展示

[0154] ～[located at]

[0155] 在哪

[0156] 有什么内容

[0157] 包含什么数据。

[0158] 在本申请的一种可选实施例中，步骤S21还包括以下子步骤：

[0159] S41：获取若干样本大屏网页；

[0160] S42：采用预设的任务生成工具针对任一所述样本大屏网页生成至少一个样本任务；

[0161] S43：在执行所述样本任务的过程中，采用预设的操作记录工具记录执行所述样本任务产生的操作信息序列；

[0162] 在本申请实施例中，可以获取若干样本大屏网页，采用预设的任务生成工具针对任一样本大屏网页生成至少一个样本任务。在执行样本任务的过程中，采用预设的操作记录工具记录执行样本任务产生的操作信息序列。

[0163] 在具体实现中，可以参照图3，图3示出了本申请一实施例提供的获取训练数据的流程示意图，在此步骤中，可以使用WizardLM的提示技术为每个大屏网站设计40个复杂任务，并选择一半任务进行手工标注。对于操作序列，由于任务的高度复杂性，相关技术的大模型无法以令人满意的准确度完成任务。因此，通过一个记录网站任务期间操作的Chrome插件Wildfire结合手工注释来捕获页面执行的操作指令集。最终得到了相关任务、操作意图和操作指令集的对应关系。然后使用大模型生成每步操作对应的意图，以便智能体更好地理解这些操作本身含义及相关性，学习其中的思维链。

[0164] 需要说明的是，WizardLM(Wizard Language Model)：是利用大型语言模型(LLMs)来自动创建大量各种难度级别的指令数据的方法。在本申请实施例中，可以通过WizardLM获取大量训练数据以提升LLMs的性能。

[0165] Wildfire是一个Chrome浏览器插件，可以帮助用户在网页上快速创建和分享注释。可以提供一种简单的方式来标记网页内容、添加评论、高亮文本，并将这些注释分享给其他人。在本申请实施例中，可以通过Wildfire结合手工注释来捕获页面执行的操作指令集。

[0166] 在具体实现中，可以首先使用WizardLM生成任务："请在指标墙内查询当前4G登网用户数”，人工在大屏网页进行操作，通过Wildfire插件记录后可转为操作集：{步骤1:{点击，"指标墙进入按钮”}，步骤2:{点击，"弹出搜索框”}，步骤3:{键入，"4G登网用户数”}}，基于GPT-4o对每步操作生成意图列表：{步骤1:"进入指标墙页面以进行后续搜索”，步骤2:"打开搜索框以便输入文本”，步骤3:"输入文本查询指标”}。

[0167] S44：将所述操作信息序列划分为所述训练操作信息和所述测试操作信息；

[0168] S45：针对所述操作信息序列中的任一所述操作信息，确定所述操作信息对应的意图信息，得到所述操作信息序列对应的意图信息序列。

[0169] S46：将所述操作信息序列对应的意图信息序列划分为所述训练意图信息和所述测试意图信息；

[0170] S47：将所述训练意图信息和所述训练操作信息作为所述训练数据；

[0171] S48：将所述测试意图信息和所述测试操作信息作为所述测试数据。

[0172] 在本申请实施例中，可以将操作信息序列划分为训练操作信息和测试操作信息，针对操作信息序列中的任一操作信息，确定操作信息对应的意图信息，得到操作信息序列对应的意图信息序列。然后，将操作信息序列对应的意图信息序列划分为训练意图信息和测试意图信息，将训练意图信息和训练操作信息作为训练数据，将测试意图信息和测试操作信息作为测试数据。

[0173] 本申请借助AI工具生成训练指令集，快速生成，省时省力。对于复杂操作任务的标注工作，通过一个记录网站任务期间操作的Chrome插件Wildfire结合手工注释来捕获页面执行的操作指令集，可以有效提高标注效率。

[0174] 在本申请的一种可选实施例中，在得到所述经过训练的语言模型之后，所述方法还包括以下步骤：

[0175] S51：获取若干所述操作信息序列和若干所述操作信息序列对应的若干意图信息序列；

[0176] S52：根据所述操作信息序列和所述意图信息序列确定正样本数据和负样本数据；

[0177] S53：将所述正样本数据中的所述操作信息序列作为正样本训练操作信息，将所述正样本数据中的所述意图信息序列作为正样本训练意图信息；

[0178] S54：将所述负样本数据中的所述操作信息序列作为负样本训练操作信息，将所述负样本数据中的所述意图信息序列作为负样本训练意图信息；

[0179] S55：将所述正样本训练操作信息、所述正样本训练意图信息、所述负样本训练操作信息、所述负样本训练意图信息作为所述训练数据；

[0180] S56：重新执行将所述正样本训练意图信息或所述负样本训练意图信息所述作为所述语言模型的输入，将所述正样本训练操作信息或所述负样本训练操作信息作为所述语言模型的输出，采用所述训练数据对所述语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。

[0181] 在本申请实施例中，可以获取若干操作信息序列和若干操作信息序列对应的若干意图信息序列，根据操作信息序列和意图信息序列确定正样本数据和负样本数据，将正样本数据中的操作信息序列作为正样本训练操作信息，将正样本数据中的意图信息序列作为正样本训练意图信息，将负样本数据中的操作信息序列作为负样本训练操作信息，将负样本数据中的意图信息序列作为负样本训练意图信息，将正样本训练操作信息、正样本训练意图信息、负样本训练操作信息、负样本训练意图信息作为训练数据，重新执行将正样本训练意图信息或负样本训练意图信息作为语言模型的输入，将正样本训练操作信息或负样本训练操作信息作为语言模型的输出，采用训练数据对语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。其中，停止条件可以根据实际需求自行设置。

[0182] 在具体实现中，可以使用强化学习(RL)来缓解模型产生的幻觉，从一次次错误中完善模型能力。本申请前述基于课程学习训练后的大模型具备网页理解和推理的基本能力，在前述基于课程学习训练后的大模型的基础上，可以在训练集中对复杂任务操作样本进行抽样，结合标注结果构建正负对立的对比数据，基于对比数据进行强化学习，使得大模型在正负对立的样本中学习，从而进一步提高模型对网页操作的推理能力。其中，RL(Reinforcement Learning，强化学习)，是一种机器学习方法，通过让智能体(agent)在与环境的交互中学习如何做出最优决策来实现目标。

[0183] 在本申请的一种可选实施例中，所述方法还包括以下步骤：

[0184] S61：根据所述待验证操作信息和所述目标文档对象集合确定待验证网页操作指令，根据所述待验证网页操作指令对所述大屏网页进行操作，得到待验证操作结果；

[0185] S62：根据预设的奖励函数或所述待验证操作结果确定所述待验证操作信息的奖励信号；

[0186] S63：根据所述奖励信号对所述待验证的语言模型进行模型参数调整，并重新执行将所述训练意图信息作为所述语言模型的输入，将所述训练操作信息作为所述语言模型的输出，采用所述训练数据对所述语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。

[0187] 在本申请实施例中，可以根据待验证操作信息和目标文档对象集合确定待验证网页操作指令，根据待验证网页操作指令对大屏网页进行操作，得到待验证操作结果，根据预设的奖励函数或待验证操作结果确定待验证操作信息的奖励信号，根据奖励信号对待验证的语言模型进行模型参数调整，并重新执行将训练意图信息作为语言模型的输入，将训练操作信息作为语言模型的输出，采用训练数据对语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。其中，停止条件可以根据实际需求自行设置。

[0188] 在具体实现中，可以使用拒绝采样微调(RSFT)针对目标领域的网页环境进行优化，进一步提高模型能力。RSFT可以优化大模型在特定领域网页的推理能力。RSFT可以从现有模型进行大量抽样来进行有针对性的训练，通过设置奖励信号，提高大模型从缺乏正确路径的实例中选择正确操作轨迹的能力。奖励信号可以通过预先设计的奖励模型或基于网页环境提供。其中，RSFT(Rejection Sampling Fine-Tuning，拒绝采样微调)是一种改进的拒绝采样方法，可以在保持拒绝采样原理简单的优点的同时，提高采样效率。可以通过对建议分布进行微调，使其更接近目标分布，从而提高采样效率。

[0189] 本申请通过课程学习算法(CL)增强了模型对网页的理解，以及作为agent在环境中执行操作的能力。然后使用强化学习(RL)来缓解模型产生的幻觉，从一次次错误中完善模型能力。最后使用拒绝采样微调(RSFT)针对目标领域的网页环境进行优化，进一步提高模型能力。通过从简单到复杂的任务序列来训练模型，可以帮助模型更好地理解网页操作的基本概念和逐步深入的高级概念。可以提高模型在理解网页操作任务上的学习效率、决策质量和泛化能力，同时也能够更有效地利用训练样本，提高模型的最终性能。

[0190] 参照图4，示出了本申请一实施例提供的网页交互智能体进行数据交互的流程示意图。

[0191] 首先，可以将文字或识别语音形成的文字输入网页交互智能体，网页交互智能体可以形成操作指令集，其中可以包括若干操作指令，例如：

[0192] 操作指令1：{"action”；"click”；"id”；"example1”}

[0193] 操作指令2：{"action”；"scroll”；"id”；"example2”}

[0194] 控制器selenium通过操作指令集来控制目标网页。

[0195] 其次，可以在目标网页中提取元素，如采用DOM或OCR(Optical CharacterRecognition，光学字符识别)等技术进行元素提取，再进行元素筛选，移除其中非显示对象，将目标网页的元素作为使用网页交互智能体控制目标网页的依据之一。

[0196] 然后，在训练网页交互智能体时，可以获取简单单步指令和复杂多步指令构建的训练指令集，采用训练指令集对网页交互智能体进行训练。训练网页交互智能体的过程可以是先进行课程学习，学习和理解网页结构，并构建操作指令集，然后进行强化学习，从自己的错误中学习正确的知识，然后是拒绝采样微调，提高模型在复杂网页中的熟练程度。

[0197] 最后，网页交互智能体也可以通过执行操作指令集中的操作指令，形成对应的历史操作，也可以参照此历史操作对网页交互智能体进行训练。

[0198] 参照图5，示出了本申请一实施例提供的一种车辆电池的降温装置的结构示意图，其特征在于，所述装置包括：

[0199] 信息获取模块501，用于响应于用户的网页控制指令，获取所述网页控制指令中的控制意图信息和原始大屏网页的原始文档对象集合；

[0200] 意图识别模块502，用于采用预先训练的语言模型从所述控制意图信息中识别出网页操作信息；

[0201] 网页简化模块503，用于确定所述原始大屏网页的所述原始文档对象集合中的非显示对象，并在所述原始文档对象集合中移除所述非显示对象，得到目标文档对象集合；

[0202] 操作指令获取模块504，用于根据所述网页操作信息和所述目标文档对象集合生成网页操作指令；

[0203] 网页控制模块505，用于响应于所述网页操作指令生成目标大屏网页，并将所述目标大屏网页展示给所述用户。

[0204] 可选地，所述非显示对象包括功能对象、空白对象、冗余对象中的至少一个，所述网页简化模块503，包括：

[0205] 功能对象移除模块，用于确定所述原始文档对象集合中的所述功能对象，并在所述原始文档对象集合中移除所述功能对象，得到所述目标文档对象集合；所述功能对象是与所述大屏网页的内容展示无关的对象；

[0206] 和 / 或，空白对象移除模块，用于确定所述原始文档对象集合中的所述空白对象，并在所述原始文档对象集合中移除所述空白对象，得到所述目标文档对象集合；所述空白对象是所述大屏网页中对应空白字符的对象；

[0207] 和 / 或，冗余对象移除模块，用于确定所述原始文档对象集合中的冗余对象，并在所述原始文档对象集合中移除所述冗余对象，得到所述目标文档对象集合；所述冗余对象是未被所述大屏网页使用的对象。

[0208] 可选地，其特征在于，所述语言模型采用如下方式训练：

[0209] 数据获取模块，用于获取训练数据和测试数据；所述训练数据包括训练操作信息和训练意图信息；所述测试数据包括测试操作信息和测试意图信息；

[0210] 模型训练模块，用于将所述训练意图信息作为所述语言模型的输入，将所述训练操作信息作为所述语言模型的输出，采用所述训练数据对所述语言模型进行训练，得到待验证的语言模型；

[0211] 模型验证模块，用于将所述测试意图信息输入所述待验证的语言模型，得到所述待验证的语言模型输出的待验证操作信息；

[0212] 损失函数获取模块，用于根据所述测试操作信息和所述待验证操作信息确定所述语言模型对应的损失函数；

[0213] 参数调整模块，用于采用所述语言模型对应的损失函数对所述待验证的语言模型进行模型参数调整，并重新执行将所述训练意图信息作为所述语言模型的输入，将所述训练操作信息作为所述语言模型的输出，采用所述训练数据对所述语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。

[0214] 可选地，所述数据获取模块，包括：

[0215] 第一网页获取模块，用于获取样本大屏网页；

[0216] 网页解析模块，用于采用预设的网页解析工具对所述样本大屏网页进行解析，得到所述样本大屏网页中的可操作对象、可操作对象类型和可操作对象编码；

[0217] 操作指令对照获取模块，用于获取所述样本大屏网页中的至少一条样本操作指令和所述样本操作指令对应的样本操作指令定义；

[0218] 任务意图获取模块，用于采用所述样本操作指令定义、所述可操作对象和所述可操作对象类型生成至少一个样本任务，并获取所述样本任务对应的意图信息；

[0219] 第一数据划分模块，用于将所述样本任务对应的意图信息划分为所述训练意图信息和所述测试意图信息；

[0220] 操作信息确定模块，用于根据所述样本操作指令定义对应的所述样本操作指令和所述可操作对象对应的所述可操作对象编码确定所述样本任务对应的操作信息；

[0221] 第二数据划分模块，用于将所述样本任务对应的操作信息划分为所述训练操作信息和所述测试操作信息；

[0222] 第一训练数据获取模块，用于将所述训练意图信息和所述训练操作信息作为所述训练数据；

[0223] 第一测试数据获取模块，用于将所述测试意图信息和所述测试操作信息作为所述测试数据。

[0224] 可选地，所述数据获取模块，还包括：

[0225] 第二网页获取模块，用于获取若干样本大屏网页；

[0226] 样本任务获取模块，用于采用预设的任务生成工具针对任一所述样本大屏网页生成至少一个样本任务；

[0227] 操作记录模块，用于在执行所述样本任务的过程中，采用预设的操作记录工具记录执行所述样本任务产生的操作信息序列；

[0228] 第三数据划分模块，用于将所述操作信息序列划分为所述训练操作信息和所述测试操作信息；

[0229] 意图序列获取模块，用于针对所述操作信息序列中的任一所述操作信息，确定所述操作信息对应的意图信息，得到所述操作信息序列对应的意图信息序列；

[0230] 第四数据划分模块，用于将所述操作信息序列对应的意图信息序列划分为所述训练意图信息和所述测试意图信息；

[0231] 第二训练数据获取模块，用于将所述训练意图信息和所述训练操作信息作为所述训练数据；

[0232] 第二测试训练数据获取模块，用于将所述测试意图信息和所述测试操作信息作为所述测试数据。

[0233] 可选地，在得到所述经过训练的语言模型之后，所述装置还包括：

[0234] 复杂信息获取模块，用于获取若干所述操作信息序列和若干所述操作信息序列对应的若干意图信息序列；

[0235] 正负样本获取模块，用于根据所述操作信息序列和所述意图信息序列确定正样本数据和负样本数据；

[0236] 正样本训练意图获取模块，用于将所述正样本数据中的所述操作信息序列作为正样本训练操作信息，将所述正样本数据中的所述意图信息序列作为正样本训练意图信息；

[0237] 负样本训练意图获取模块，用于将所述负样本数据中的所述操作信息序列作为负样本训练操作信息，将所述负样本数据中的所述意图信息序列作为负样本训练意图信息；

[0238] 第三训练数据获取模块，用于将所述正样本训练操作信息、所述正样本训练意图信息、所述负样本训练操作信息、所述负样本训练意图信息作为所述训练数据；

[0239] 二次模型训练模块，用于重新执行将所述正样本训练意图信息或所述负样本训练意图信息所述作为所述语言模型的输入，将所述正样本训练操作信息或所述负样本训练操作信息作为所述语言模型的输出，采用所述训练数据对所述语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。

[0240] 可选地，所述装置还包括：

[0241] 待验证结果获取模块，用于根据所述待验证操作信息和所述目标文档对象集合确定待验证网页操作指令，根据所述待验证网页操作指令对所述大屏网页进行操作，得到待验证操作结果；

[0242] 奖励信号获取模块，用于根据预设的奖励函数或所述待验证操作结果确定所述待验证操作信息的奖励信号；

[0243] 三次模型训练模块，用于根据所述奖励信号对所述待验证的语言模型进行模型参数调整，并重新执行将所述训练意图信息作为所述语言模型的输入，将所述训练操作信息作为所述语言模型的输出，采用所述训练数据对所述语言模型进行训练的步骤，直至达到预设的停止条件，得到经过训练的语言模型。

[0244] 本申请一实施例还提供了一种电子设备，其特征在于，包括处理器、存储器及存储在所述存储器上并能够在所述处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现如上所述的大屏网页的控制方法。

[0245] 本申请一实施例还提供了一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储计算机程序，所述计算机程序被处理器执行时实现如上所述的大屏网页的控制方法。

[0246] 需要说明的是，本申请所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，并且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准，并提供有相应的操作入口，供用户选择授权或者拒绝。

[0247] 本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。

[0248] 本领域内的技术人员应明白，本申请实施例可提供为方法、装置、或计算机程序产品。因此，本申请实施例可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。

[0249] 本申请实施例是参照根据本申请实施例的方法、终端设备(系统)、和计算机程序产品的流程图和 / 或方框图来描述的。应理解可由计算机程序指令实现流程图和 / 或方框图中的每一流程和 / 或方框、以及流程图和 / 或方框图中的流程和 / 或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理终端设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理终端设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和 / 或方框图一个方框或多个方框中指定的功能的装置。

[0250] 这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理终端设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和 / 或方框图一个方框或多个方框中指定的功能。

[0251] 这些计算机程序指令也可装载到计算机或其他可编程数据处理终端设备上，使得在计算机或其他可编程终端设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程终端设备上执行的指令提供用于实现在流程图一个流程或多个流程和 / 或方框图一个方框或多个方框中指定的功能的步骤。

[0252] 尽管已描述了本申请实施例的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例做出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请实施例范围的所有变更和修改。

[0253] 最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语"包括”、"包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者终端设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者终端设备所固有的要素。在没有更多限制的情况下，由语句"包括一个……”限定的要素，并不排除在包括上述要素的过程、方法、物品或者终端设备中还存在另外的相同要素。

[0254] 以上对所提供的一种大屏网页的控制方法、装置、电子设备及介质，进行了详细介绍，本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本申请的限制。< / script>

Claims

1. A method for controlling a large-screen web page, characterized in that: The method comprises: In response to a webpage control instruction of a user, obtaining control intent information in the webpage control instruction and an original document object set of an original large-screen webpage; Using a pre-trained language model to identify web page operation information from the control intention information; Determine non-display objects in the original document object set of the original large-screen webpage, and remove the non-display objects from the original document object set to obtain a target document object set; Generate a web page operation instruction according to the web page operation information and the target document object set; A target large-screen webpage is generated in response to the webpage operation instruction, and the target large-screen webpage is displayed to the user.

2. The method according to claim 1, characterized in that The non-display object includes at least one of a functional object, a blank object, and a redundant object. The determining the non-display object in the original document object set of the original large-screen webpage and removing the non-display object from the original document object set to obtain a target document object set includes: Determine the functional object in the original document object set, and remove the functional object from the original document object set to obtain the target document object set; the functional object is an object that is irrelevant to the content display of the large-screen webpage; and / or, determining the blank objects in the original document object set, and removing the blank objects from the original document object set to obtain the target document object set; the blank objects are objects corresponding to blank characters in the large-screen web page; And / or, determining redundant objects in the original document object set, and removing the redundant objects from the original document object set to obtain the target document object set; the redundant objects are objects that are not used by the large-screen web page.

3. The method according to claim 1, characterized in that: The language model is trained in the following way: Acquire training data and test data; the training data includes training operation information and training intention information; the test data includes test operation information and test intention information; Using the training intention information as the input of the language model, using the training operation information as the output of the language model, and using the training data to train the language model to obtain a language model to be verified; Inputting the test intention information into the language model to be verified, and obtaining the operation information to be verified output by the language model to be verified; Determine a loss function corresponding to the language model according to the test operation information and the operation information to be verified; The loss function corresponding to the language model is used to adjust the model parameters of the language model to be verified, and the steps of taking the training intention information as the input of the language model, taking the training operation information as the output of the language model, and using the training data to train the language model are re-executed until a preset stopping condition is reached to obtain a trained language model.

4. The method according to claim 3, characterized in that The obtaining of training data and test data includes: Get the sample large screen web page; The sample large-screen web page is parsed using a preset web page parsing tool to obtain operable objects, operable object types and operable object codes in the sample large-screen web page; Obtain at least one sample operation instruction in the sample large-screen webpage and a sample operation instruction definition corresponding to the sample operation instruction; Generate at least one sample task using the sample operation instruction definition, the operable object and the operable object type, and obtain intent information corresponding to the sample task; Dividing the intention information corresponding to the sample task into the training intention information and the test intention information; Determine the operation information corresponding to the sample task according to the sample operation instruction corresponding to the sample operation instruction definition and the operable object code corresponding to the operable object; Dividing the operation information corresponding to the sample task into the training operation information and the test operation information; Using the training intention information and the training operation information as the training data; The test intention information and the test operation information are used as the test data.

5. The method according to claim 3, characterized in that: The obtaining of training data and test data further includes: Get several sample large-screen web pages; Using a preset task generation tool to generate at least one sample task for any of the sample large-screen web pages; In the process of executing the sample task, a preset operation recording tool is used to record the operation information sequence generated by executing the sample task; Dividing the operation information sequence into the training operation information and the test operation information; For any operation information in the operation information sequence, determine the intention information corresponding to the operation information, and obtain the intention information sequence corresponding to the operation information sequence; Dividing the intention information sequence corresponding to the operation information sequence into the training intention information and the test intention information; Using the training intention information and the training operation information as the training data; The test intention information and the test operation information are used as the test data.

6. The method according to claim 5, characterized in that After obtaining the trained language model, the method further includes: Acquire a number of the operation information sequences and a number of intention information sequences corresponding to the operation information sequences; Determine positive sample data and negative sample data according to the operation information sequence and the intention information sequence; Using the operation information sequence in the positive sample data as positive sample training operation information, and using the intention information sequence in the positive sample data as positive sample training intention information; Using the operation information sequence in the negative sample data as negative sample training operation information, and using the intention information sequence in the negative sample data as negative sample training intention information; Using the positive sample training operation information, the positive sample training intention information, the negative sample training operation information, and the negative sample training intention information as the training data; Re-execute the steps of taking the positive sample training intention information or the negative sample training intention information as the input of the language model, taking the positive sample training operation information or the negative sample training operation information as the output of the language model, and using the training data to train the language model until a preset stopping condition is reached, thereby obtaining a trained language model.

7. The method according to claim 3, characterized in that The method further comprises: Determine the webpage operation instruction to be verified according to the operation information to be verified and the target document object set, and operate the large-screen webpage according to the webpage operation instruction to be verified to obtain the operation result to be verified; Determine a reward signal for the operation information to be verified according to a preset reward function or the operation result to be verified; The model parameters of the language model to be verified are adjusted according to the reward signal, and the steps of taking the training intention information as the input of the language model, taking the training operation information as the output of the language model, and training the language model using the training data are re-executed until a preset stopping condition is reached, thereby obtaining a trained language model.

8. A control device for a large-screen web page, characterized in that: The device comprises: An information acquisition module, configured to respond to a webpage control instruction of a user and acquire control intent information in the webpage control instruction and an original document object set of an original large-screen webpage; An intention recognition module, used to recognize web page operation information from the control intention information using a pre-trained language model; A webpage simplification module, configured to determine non-display objects in the original document object set of the original large-screen webpage, and remove the non-display objects from the original document object set to obtain a target document object set; An operation instruction acquisition module, used to generate a webpage operation instruction according to the webpage operation information and the target document object set; The webpage control module is used to generate a target large-screen webpage in response to the webpage operation instruction, and display the target large-screen webpage to the user.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method for controlling a large-screen web page according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method for controlling a large-screen web page according to any one of claims 1 to 7 is implemented.

Images