Page labeling method and device based on large model and electronic equipment

Abstract

The invention discloses a page labeling method and device based on a large model and electronic equipment, and relates to the technical field of computers, in particular to the field of artificial intelligence such as deep learning and large models. According to the specific implementation scheme, the method comprises the steps of obtaining a page screenshot of a to-be-labeled page and a block screenshot of each page block in the to-be-labeled page; according to the page screenshot, integrally labeling the to-be-labeled page by using the large model to obtain integral labeling information; according to the block screenshot, labeling the page block by using a large model to obtain block labeling information; and obtaining target annotation information of the page to be annotated according to the overall annotation information and the block annotation information.

Description

Technical Field

[0001] This application relates to the field of computer technology, especially to artificial intelligence fields such as deep learning and large models. Specifically, it relates to a page annotation method, device, and electronic device based on a large model. Background Art

[0002] With the development of artificial intelligence, generative AI (Artificial Intelligence) applications have emerged. Generative AI applications can generate application pages through natural language interaction, improving development efficiency. Summary of the Invention

[0003] This application provides a page annotation method, device, and electronic device based on a large model. The specific solutions are as follows:

[0004] According to one aspect of this application, a page annotation method based on a large model is provided, including:

[0005] Obtain a page screenshot of the page to be annotated and block screenshots of each page block in the page to be annotated;

[0006] According to the page screenshot, use the large model to perform an overall annotation on the page to be annotated to obtain overall annotation information;

[0007] According to the block screenshots, use the large model to annotate the page blocks to obtain block annotation information;

[0008] According to the overall annotation information and the block annotation information, obtain the target annotation information of the page to be annotated.

[0009] According to another aspect of this application, a page annotation device based on a large model is provided, including:

[0010] A first acquisition module, configured to obtain a page screenshot of the page to be annotated and block screenshots of each page block in the page to be annotated;

[0011] An annotation module, configured to perform an overall annotation on the page to be annotated using the large model according to the page screenshot to obtain overall annotation information;

[0012] The annotation module is further configured to perform an annotation on the page blocks using the large model according to the block screenshots to obtain block annotation information;

[0013] A second acquisition module, configured to obtain the target annotation information of the page to be annotated according to the overall annotation information and the block annotation information.

[0014] According to another aspect of this application, an electronic device is provided, including:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the above embodiments.

[0018] According to another aspect of the present application, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method described in the above embodiments.

[0019] According to another aspect of the present application, there is provided a computer program product, including a computer program, where the computer program implements the steps of the method described in the above embodiments when executed by a processor.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings are used to better understand the solution and do not constitute a limitation to the present application. Among them:

[0022] Figure 1 is a schematic flowchart of a page annotation method based on a large model provided by an embodiment of the present application;

[0023] Figure 2 is a schematic flowchart of a page annotation method based on a large model provided by another embodiment of the present application;

[0024] Figure 3 is a schematic flowchart of a page annotation method based on a large model provided by another embodiment of the present application;

[0025] Figure 4 is a schematic flowchart of a page annotation method based on a large model provided by another embodiment of the present application;

[0026] Figure 5 is a schematic flowchart of a page annotation method based on a large model provided by another embodiment of the present application;

[0027] Figure 6 is a schematic structural diagram of a page annotation device based on a large model provided by an embodiment of the present application;

[0028] Figure 7It is a block diagram of an electronic device for implementing the page annotation method based on a large model according to an embodiment of the present application. Detailed implementation manners

[0029] The following describes exemplary embodiments of the present application with reference to the accompanying drawings. Various details of the embodiments of the present application are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, descriptions of well-known functions and structures are omitted below for clarity and conciseness.

[0030] It should be noted that the acquisition, storage, use, processing, etc. of data in the technical solution of the present application all comply with the relevant regulations of national laws and regulations and do not violate public order and good customs.

[0031] The following describes a page annotation method, device, electronic device, and storage medium based on a large model according to an embodiment of the present application with reference to the accompanying drawings.

[0032] In the related art, for the page generation scenario, common problems of generative AI are poor accuracy of page annotation results, low annotation efficiency, etc. Based on this, an embodiment of the present application provides a page annotation method based on a large model. Figure 1 It is a flowchart of a page annotation method based on a large model provided by an embodiment of the present application.

[0033] The page annotation method based on a large model according to an embodiment of the present application can be executed by a page annotation device based on a large model according to an embodiment of the present application, and this device can be configured in an electronic device.

[0034] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, etc., which are hardware devices with various operating systems, touch screens, and / or display screens.

[0035] As Figure 1 shown, the page annotation method based on a large model includes:

[0036] Step 101, obtain a page screenshot of the page to be annotated and block screenshots of each page block in the page to be annotated.

[0037] Exemplarily, the page to be annotated can be a web page of websites in various industries and scenarios.

[0038] In this application, the page to be annotated may include multiple page blocks. A page block can be a UI (User Interface) section in the page to be annotated that has independent functions and display logics, such as a header image block, company introduction, product advantages, featured functions, news list, product list, contact us, etc. Different page blocks can form a complete page, and page blocks of the same type may have different display styles, block elements, etc.

[0039] Exemplarily, the page to be annotated can be screenshot as a whole through a screenshot tool to obtain a page screenshot, and each page block can be screenshot separately to obtain a block screenshot of each page block.

[0040] Exemplarily, a headless browser can also be used to take a screenshot of the page to be annotated as a whole and by blocks to obtain a page screenshot and block screenshots of each page block.

[0041] It should be noted that the above screenshot methods are only examples, and other methods can also be used to obtain the page screenshot and block screenshots. This application does not make any limitations in this regard.

[0042] Step 102: Based on the page screenshot, use a large model to perform an overall annotation on the page to be annotated to obtain overall annotation information.

[0043] Among them, overall annotation can refer to comprehensively and comprehensively annotating the page, aiming to capture the overall characteristics and information of the page.

[0044] In this application, the multi-modal understanding ability of the large model can be used to analyze the page screenshot to obtain overall annotation information such as the type, layout, and style of the page to be annotated, so as to achieve the overall annotation of the page to be annotated.

[0045] Among them, the multi-modal understanding ability can include text understanding, image understanding, context understanding, logical inference, etc.

[0046] Exemplarily, based on the page reference type, page annotation examples, annotation information output requirements, etc., a large model can be used to perform an overall annotation on the page to be annotated to obtain overall annotation information.

[0047] Step 103: Based on the block screenshot, use a large model to annotate the page block to obtain block annotation information.

[0048] Since the functions of different page blocks in the same page may be different, in order to improve the accuracy of annotation, in this application, for each page block, the multi-modal understanding ability of the large model can be used to analyze the block screenshot to obtain block annotation information such as the name and type of the page block, so as to achieve the block-by-block annotation of the page to be annotated.

[0049] Step 104: Obtain the target annotation information of the page to be annotated according to the overall annotation information and the block annotation information.

[0050] In this application, the overall annotation information, block annotation information, etc. can be used as the target annotation information of the page to be annotated. The annotated page here can be used as the material support for generating the application page.

[0051] Exemplarily, according to the page annotated by the above method, an application page can be generated by using a generative AI application based on artificial intelligence. Among them, the application page can be understood as the page of an application program, such as the pages of websites, H5 (HyperText Markup Language 5), etc.

[0052] In the embodiment of this application, by taking a screenshot of the page to be annotated and using a large model to perform overall annotation on the page to be annotated to obtain the overall annotation information, and performing annotation on the page blocks according to the block screenshots of each page block in the page to be annotated to obtain the block annotation information, and obtaining the target annotation information of the page to be annotated according to the overall annotation information and the block annotation information. Thus, by using a large model to annotate the page to be annotated from two different granularities of the page as a whole and the page blocks, the accuracy of page annotation can be improved, and the annotation efficiency is high.

[0053] Figure 2 It is a schematic flowchart of a page annotation method based on a large model provided by another embodiment of this application.

[0054] As Figure 2 shown, the page annotation method based on a large model includes:

[0055] Step 201: Obtain the page screenshot of the page to be annotated and the block screenshots of each page block in the page to be annotated.

[0056] Since the description data of different web pages may be in different formats, for example, some pages are in HTML (HyperText Markup Language) format and some pages are in JSON format. In order to facilitate the subsequent generation of a new page based on the annotation, in this application, the format of the description data of the page to be annotated can be unified.

[0057] Since DSL (Domain-Specific Language) focuses on a specific domain and its grammar and structure can be closer to the natural language expression of that domain, which makes the DSL code easier to read and understand. Therefore, this application can unify the format of the original description data of the page to be annotated into the DSL format.

[0058] Exemplarily, after obtaining the original description data of the page to be annotated, it can be first determined whether the format of the original description data is the DSL format. If the format of the original description data is not the DSL format, the original description data is converted into page DSL data, and the page rendered by the page DSL data is respectively subjected to an overall screenshot and a block screenshot to obtain a page screenshot and each block screenshot. Among them, the non-DSL format can be understood as other formats except the DSL format, such as the HTML format, etc.

[0059] For example, a headless browser can be used to perform an overall screenshot and a block screenshot on the page rendered by the page DSL data.

[0060] Thus, by unifying the format of the description data of the page to be annotated into the DSL format, the readability and maintainability can be enhanced. Performing an overall screenshot and a block screenshot on the page rendered by the page DSL data can improve the consistency of page annotation and the accuracy of annotation.

[0061] Step 202: Generate a first prompt message according to the page screenshot, the first annotation dimension, and the page annotation example.

[0062] Among them, the first annotation dimension may include, but is not limited to, the type of the page, the theme color, the style, the layout, the block, etc.

[0063] It should be noted that the first annotation dimension may be one or more, and the present application does not make any limitation thereto.

[0064] Among them, the page annotation example may include the page screenshot of the reference page, the output example of the overall analysis result of the reference page, etc.

[0065] Exemplarily, the first annotation dimension and the page annotation example may be obtained according to the user input, or obtained from the annotation library according to the annotation requirements, or obtained by other means, and no limitation is made thereto.

[0066] In the present application, a first prompt message can be generated according to the page screenshot, the first annotation dimension, the page annotation example, etc.

[0067] Among them, the first prompt message may include the page screenshot, the first annotation dimension, the page annotation example, etc.

[0068] Optionally, the first prompt message may further include the reference categories of some annotation dimensions in the first annotation dimension, the requirements for the output of the large model, etc. Exemplary requirements for the output of the large model may include what output items there are, the requirements for some output items, the output form, the output format, etc.

[0069] For example, the first prompt message of a certain page screenshot is as follows:

[0070]

[0071]

[0072] Please directly return the strict JSON result according to the output format, without adding comments and extended descriptions, and without using Markdown syntax.

[0073] It can be seen that in the prompt information of the above example, the first annotation dimension includes type, theme color, style, layout, blocks, etc., and the output items of the large model include theme color, style, and page description, where the page description includes information such as the type, layout, and blocks of the page.

[0074] It should be noted that the number of the first annotation dimensions and the number of the output items of the large model can be the same or different, and this application does not make any limitations in this regard.

[0075] Step 203: Use a large model to annotate the page to be annotated according to the first annotation dimension based on the first prompt information, so as to obtain the overall annotation information.

[0076] In this application, the first prompt information can be input into the large model, and by using the multi-modal understanding ability of the large model, the page screenshot can be analyzed according to the first annotation dimension and the reference page annotation example, so as to annotate the page to be annotated according to the first annotation dimension and obtain the overall annotation information of the page to be annotated.

[0077] Exemplarily, the page screenshot can be recognized by the large model to obtain the text content in the page to be annotated, the annotation information of the annotation dimensions related to the text in the first annotation dimension can be determined according to the text content, such as the type and scenario of the page, etc., and the page screenshot can be recognized to determine the annotation information of the annotation dimensions related to the image in the first annotation dimension, such as the theme color of the page, etc.

[0078] Optionally, in order to improve the accuracy of the overall annotation, other large models can be used to verify the overall annotation information, and the overall annotation information of the page to be annotated can be determined according to the annotation results of multiple large models.

[0079] Step 204: Use the large model to annotate the page blocks according to the block screenshots to obtain the block annotation information.

[0080] Step 205: Obtain the target annotation information of the page to be annotated according to the overall annotation information and the block annotation information.

[0081] In this application, steps 204 - 205 can adopt any implementation manner in the embodiments of this application, so details are not described herein again.

[0082] In the embodiments of the present application, by generating a first prompt message based on a page screenshot, a first annotation dimension, and a page annotation example, and using a large model to process the first prompt message to annotate the page to be annotated according to the first annotation dimension, overall annotation information is obtained. Thus, by using a large model and analyzing the page screenshot with reference to the page annotation example to perform annotation on the page to be annotated in the corresponding dimension, the overall annotation accuracy can be improved, and further the page annotation accuracy can be improved.

[0083] Figure 3 FIG. 4 is a schematic flowchart of a page annotation method based on a large model provided by another embodiment of the present application.

[0084] As Figure 3 shown, the page annotation method based on a large model includes:

[0085] Step 301, obtain a page screenshot of the page to be annotated and block screenshots of each page block in the page to be annotated.

[0086] Step 302, use a large model to perform overall annotation on the page to be annotated according to the page screenshot to obtain overall annotation information.

[0087] In the present application, steps 301-step 302 can adopt any implementation manner in the embodiments of the present application, so details are not described herein again.

[0088] Step 303, generate a second prompt message according to the block screenshot, a second annotation dimension, and a block annotation example.

[0089] Among them, the second annotation dimension may include but is not limited to block name, block classification, block layout, block theme color, block style, block description, block serial number, etc.

[0090] Exemplarily, the block description may include dimensions such as block type, main content included in the block, picture or copywriting features, applicable scenarios, etc. Exemplarily, the block serial number may refer to the serial number of the page block among all page blocks in the page to be annotated.

[0091] It should be noted that the second annotation dimension may be one or more, and the present application does not limit this.

[0092] Among them, the block annotation example may include a block screenshot of a reference page, an output example of a block analysis result, etc.

[0093] Exemplarily, the second annotation dimension and the block annotation example may be obtained according to user input, or obtained from an annotation library according to annotation requirements, or obtained by other means, and this is not limited.

[0094] In this application, the corresponding prompt template can be filled according to the block screenshot, the second annotation dimension, the block annotation example, etc., to obtain the second prompt information.

[0095] Among them, the second prompt information may include the block screenshot, the second annotation dimension, the page annotation example, etc.

[0096] Optionally, the second prompt information may further include the reference categories of some annotation dimensions in the second annotation dimension, the requirements for the output of the large model, etc. Exemplary requirements for the output of the large model may include what output items there are, the requirements for some output items, the output form, the output format, etc.

[0097] For example, the second prompt information may be as follows:

[0098] Suppose you are a web designer. According to the screenshots of each block of the provided web page, please analyze the block name, block classification, block layout, theme color, style, description, and the serial number in all original block pictures in the order of the pictures. Keep the description of each block concise.

[0099]

[0100]

[0101] Please return the block descriptions in the order and number of the pictures. There should be as many items as there are block screenshots in the array. If the block type cannot be determined, it can be set to other. The final result should be directly returned as a strict JSON result in the output format without adding comments and extended descriptions.

[0102] It should be noted that the number of the second annotation dimensions and the number of output items of the large model may be the same or different, and this application does not make any restrictions on this.

[0103] In addition, the screenshots of each block of the page to be annotated Figure 1 can be input into the large model for annotation, and the large model outputs the block annotation information of each page block. Or, the screenshot of a single block can also be input into the large model for annotation. This application does not make any restrictions on this.

[0104] Step 304, using the large model, based on the second prompt information, annotate the page blocks according to the second annotation dimension to obtain the block annotation information.

[0105] In this application, the second prompt information can be input into the large model, and by using the multi-modal understanding ability of the large model, according to the second annotation dimension and the reference block annotation example, analyze the block screenshot to annotate the block screenshot according to the second annotation dimension, so as to obtain the block annotation information of the page block.

[0106] Optionally, in order to improve the accuracy of block annotation, other large models can be used to verify the block annotation information, and the block annotation information of the page block can be determined according to the annotation results of multiple large models.

[0107] Step 305: Obtain the target annotation information of the page to be annotated according to the overall annotation information and the block annotation information.

[0108] In this application, step 305 can adopt any implementation manner in the embodiments of this application, so it will not be elaborated here.

[0109] In the embodiments of this application, by generating the second prompt information according to the block screenshot, the second annotation dimension, and the block annotation example, and using the large model to process the second prompt information for guidance, the page block is annotated according to the second annotation dimension to obtain the block annotation information. Thus, by using the large model and analyzing the block screenshot with reference to the block annotation example to annotate the page block in the corresponding dimension, the accuracy of block-level annotation can be improved, and further the accuracy of page annotation can be improved.

[0110] Figure 4 It is a schematic flowchart of a page annotation method based on a large model provided by another embodiment of this application.

[0111] As Figure 4 shown, the page annotation method based on a large model includes:

[0112] Step 401: Obtain the page screenshot of the page to be annotated and the block screenshots of each page block in the page to be annotated.

[0113] Step 402: Use the large model to perform overall annotation on the page to be annotated according to the page screenshot to obtain the overall annotation information.

[0114] Step 403: Use the large model to annotate the page block according to the block screenshot to obtain the block annotation information.

[0115] In this application, steps 401 - 403 can adopt any implementation manner in the embodiments of this application, so it will not be elaborated here.

[0116] Step 404: Obtain the first block element in the page block.

[0117] Among them, the block element can refer to the elements in the page block, such as pictures, texts, etc. For example, the first block element can include the first picture, the first text, etc.

[0118] Exemplarily, the block DSL data of the page block can be obtained from the page DSL data of the page to be annotated, and the first picture, the first text, etc. in the page block can be obtained by traversing the block DSL data.

[0119] Step 405: Annotate the first block element to obtain element annotation information.

[0120] In this application, the annotation dimensions of different types of block elements may be different. The third annotation dimension and element annotation examples can be obtained according to the type of the first block element. Then, based on the first block element, the third annotation dimension, the element annotation examples, etc., the third prompt information is generated. Using a large model, based on the third prompt information, the first block element is annotated according to the third annotation dimension to obtain element annotation information.

[0121] Exemplarily, the first block element includes a first picture. The third annotation dimension for the first picture may include picture type, picture description, whether the background is transparent, etc. The element annotation example may refer to a picture annotation example, and the picture annotation example may include an output example of the picture analysis result.

[0122] For example, the third prompt information for the picture may be as follows:

[0123]

[0124] Exemplarily, the first block element includes a first text. The third annotation dimension for the first text may include the type of the text, such as the main title copy of the block, the subtitle copy of the block, the content summary copy of the block, etc.

[0125] Thus, by obtaining the third annotation dimension and element annotation examples according to the type of the first block element, and using a large model to perform corresponding dimension annotation on different block elements in the page block based on the third annotation dimension, element annotation examples, etc., the accuracy of element granularity annotation can be improved, and then the accuracy of page annotation can be improved.

[0126] Optionally, in order to improve the accuracy of element annotation, other large models can be used to verify the element annotation information, and the element annotation information of the block element is determined according to the annotation results of multiple large models.

[0127] Step 406: Obtain target annotation information according to the overall annotation information, block annotation information, and element annotation information.

[0128] In this application, the overall annotation information, block annotation information, element annotation information, etc. can be used as the target annotation information of the page to be annotated. It can be seen that the target annotation information can include annotation information with different granularities from the page as a whole to the page block and then to the page element, from coarse to fine.

[0129] Optionally, block annotation information, element annotation information, etc. can be added to the block DSL data to facilitate implementing detailed page retrieval, block combination, component modification, and other logics in combination with large models. Additionally, adding element annotation information to the block DSL data can prevent the loss of information of the block elements themselves. Herein, the block DSL data refers to the DSL data of page blocks.

[0130] For example, the annotation information of the first picture can be added to the DSL data of the component to which the first picture belongs in the block DSL data.

[0131] For another example, the annotation information of the first text can be added to the DSL data of the component corresponding to the first text in the block DSL data.

[0132] In the embodiments of the present application, by performing overall annotation on the page to be annotated based on the page screenshot to obtain overall annotation information, annotating the page blocks based on the block screenshots to obtain block annotation information, and annotating the block elements in the page blocks to obtain element annotation information. Thus, it is possible to perform annotations on the page to be annotated at three different granularities of the overall page, page blocks, and page elements, improving the accuracy of page annotation.

[0133] Figure 5 It is a schematic flowchart of a page annotation method based on a large model provided by another embodiment of the present application.

[0134] As Figure 5 shown, the page annotation method based on a large model further includes:

[0135] Step 501, obtain page generation requirement information.

[0136] In the present application, the user can input page generation requirement information in the chat interface of the generative AI application, and thus the page generation requirement information can be obtained.

[0137] Among them, the page generation requirement information can be used to describe the requirements for the page to be generated. For example, the page generation requirement information is "generate a promotional page for product A, and the information of product A is xxxx".

[0138] Step 502, retrieve in the page database according to the page generation requirement information to obtain a retrieval result.

[0139] Among them, the page database can be constructed based on the above-mentioned page to be annotated and the target annotation information of the page to be annotated.

[0140] Exemplarily, the page database can include the overall annotation information of the page to be annotated, page DSL data carrying block annotation information, element annotation information, etc.

[0141] In this application, the page generation requirement information can be matched with the target annotation information of each page in the page database, and according to the matching result, a retrieval result can be obtained.

[0142] Exemplarily, the page generation requirement information can be matched with the overall annotation information of each page in the page database. If the page generation requirement information matches the overall annotation information of any page in the page database, the retrieval result can be obtained according to this page. Among them, the retrieval result may include the page DSL data of this page.

[0143] Thus, by matching the page generation requirement information with the overall annotation information of each page in the page database to perform page retrieval, the retrieval efficiency can be improved.

[0144] If the page generation requirement information does not match the overall annotation information of all pages in the page database, the page generation requirement information can be matched with the block annotation information of the page blocks in each page. If the page generation requirement information matches the target page block, the retrieval result can be obtained according to the target page block.

[0145] Among them, the retrieval result may include the block DSL data of the target page block, etc. In addition, the target page block can be one or more, and there is no limitation on this.

[0146] Thus, if a page that meets the page generation requirements is not retrieved, a search is performed in the page blocks, so as to meet the user's needs as much as possible.

[0147] Step 503, generate an application page according to the retrieval result.

[0148] In this application, new block elements can be generated using a large model according to the page generation requirement information, and an application page can be generated according to the retrieval result and the new block elements.

[0149] As a possible implementation, if the page generation requirement information matches the overall annotation information of any page in the page database, second block elements corresponding to the page blocks in this page can be generated according to the page generation requirement information, and an application page can be generated according to the second block elements and this page.

[0150] Exemplarily, the second block elements can be used to replace the first block elements in the page blocks of this page to generate a new page, that is, the application page.

[0151] For example, the second block elements include a second picture, a second text, etc. The first picture in the page block of this page can be replaced with the second picture, and the first text in the page block of this page can be replaced with the second text, so as to generate an application page.

[0152] Exemplarily, new page DSL data can be generated based on the second block element and the page DSL data of the page, and an application page can be generated by rendering according to the new page DSL data. For example, the first block element in the page block of the page DSL data of the page can be replaced with the second block element to obtain new page DSL data.

[0153] It should be noted that the block elements to be replaced can be determined according to actual needs and are not limited in this regard.

[0154] Thus, if a page that meets the page generation requirements is matched in the page database, an application page can be generated based on the page and the block elements generated based on the page generation requirement information, which can not only improve the accuracy of the generated page, thereby improving the page generation effect, but also improve the page development efficiency.

[0155] As another possible implementation, if no matching page is retrieved in the page database but a target page block that matches the page generation requirement information is retrieved, a large model can be used to generate a third block element corresponding to the target page block according to the page generation requirement information, combine the target page blocks to obtain a combined page, and then generate an application page based on the third block element and the combined page.

[0156] Exemplarily, the third block element can be generated according to the page generation requirement information and the element annotation information of the block elements in the target page block. For example, if the background image in a certain target page block is a pink background image, which is inconsistent with the light blue background image desired by the user, then a light blue background image can be generated.

[0157] Exemplarily, according to the page generation requirement information, the page scenario can be determined, and the target page blocks can be combined according to the page layout corresponding to the page scenario to obtain a combined page. The combined page refers to the page obtained by combining the target page blocks.

[0158] For example, if the user wants to generate a product promotion page, then the target page blocks can be combined according to the preset promotion page layout method.

[0159] Exemplarily, if the block annotation information of the target page block includes a block serial number, the target page blocks can be combined according to the block serial numbers of the target page blocks.

[0160] Exemplarily, for any target page block, the first block element in the target page block in the combined page can be replaced with the generated third block element, thereby obtaining an application page.

[0161] Optionally, it is also possible to first replace the first block element in the target page block with the generated third block element, and then combine the replaced target page block to obtain an application page.

[0162] Thus, if a page block that meets the page generation requirements is matched, new block elements and page block combinations can be generated according to the page generation requirement information to generate an application page, thereby meeting different page generation requirements of users.

[0163] In the embodiments of the present application, by retrieving in the page database constructed based on the page to be annotated and its target annotation information according to the page generation requirement information, and generating an application page based on the retrieval result, the accuracy of page generation can be improved, and further the page generation effect can be improved.

[0164] To implement the above embodiments, the embodiments of the present application also propose a page annotation device based on a large model. Figure 6 It is a schematic structural diagram of a page annotation device based on a large model provided by an embodiment of the present application.

[0165] As Figure 6 shown, the page annotation device 600 based on a large model includes:

[0166] A first acquisition module 610, configured to acquire a page screenshot of the page to be annotated and block screenshots of each page block in the page to be annotated;

[0167] An annotation module 620, configured to perform overall annotation on the page to be annotated using a large model according to the page screenshot to obtain overall annotation information;

[0168] The annotation module 620 is further configured to perform annotation on the page block using the large model according to the block screenshot to obtain block annotation information;

[0169] A second acquisition module 630, configured to acquire target annotation information of the page to be annotated according to the overall annotation information and the block annotation information.

[0170] Optionally, the annotation module 620 is configured to:

[0171] Acquire a first annotation dimension and a page annotation example;

[0172] Generate first prompt information according to the page screenshot, the first annotation dimension, and the page annotation example;

[0173] Use the large model to perform annotation on the page to be annotated according to the first annotation dimension based on the first prompt information to obtain the overall annotation information.

[0174] Optionally, the annotation module 620 is configured to:

[0175] Obtain a second annotation dimension and a block annotation example;

[0176] Generate a second prompt message according to the block screenshot, the second annotation dimension, and the block annotation example;

[0177] Use the large model to annotate the page block according to the second annotation dimension based on the second prompt message to obtain block annotation information.

[0178] Optionally, the annotation module 620 is configured to:

[0179] Obtain a first block element in the page block;

[0180] Annotate the first block element to obtain element annotation information;

[0181] Obtain the target annotation information according to the overall annotation information, the block annotation information, and the element annotation information.

[0182] Optionally, the annotation module 620 is configured to:

[0183] Obtain a third annotation dimension and an element annotation example according to the type of the first block element;

[0184] Generate a third prompt message according to the first block element, the third annotation dimension, and the element annotation example;

[0185] Use the large model to annotate the first block element according to the third annotation dimension based on the third prompt message to obtain the element annotation information.

[0186] Optionally, the first acquisition module 610 is configured to:

[0187] In response to the format of the original description data of the page to be annotated being a non-domain-specific language DSL format, convert the original description data into page DSL data;

[0188] Perform an overall screenshot and a block-by-block screenshot on the page rendered using the page DSL data to obtain the page screenshot and each block screenshot.

[0189] Optionally, the device may further include:

[0190] A third acquisition module, configured to acquire page generation requirement information;

[0191] A retrieval module, configured to generate requirement information based on the page, and retrieve in the page database to obtain a retrieval result; wherein, the page database is constructed based on the page to be annotated and the target annotation information;

[0192] A generation module, configured to generate an application page based on the retrieval result.

[0193] Optionally, the retrieval module is configured to:

[0194] Match the requirement information generated from the page with the overall annotation information of each page in the page database;

[0195] In response to the requirement information generated from the page matching the overall annotation information of any page in the page database, obtain the retrieval result according to the any page.

[0196] Optionally, the generation module is configured to:

[0197] Generate second block elements corresponding to the page blocks in the any page according to the requirement information generated from the page;

[0198] Generate the application page according to the second block elements and the any page.

[0199] Optionally, the retrieval module is configured to:

[0200] In response to the requirement information generated from the page not matching the overall annotation information of all pages in the page database, match the requirement information generated from the page with the block annotation information of the page blocks in each page;

[0201] In response to the requirement information generated from the page matching a target page block, obtain the retrieval result according to the target page block.

[0202] Optionally, the generation module is configured to:

[0203] Generate third block elements corresponding to the target page block according to the requirement information generated from the page;

[0204] Combine the target page blocks to obtain a combined page;

[0205] Generate the application page according to the third block elements and the combined page.

[0206] It should be noted that the above explanation of the embodiment of the page annotation method based on the large model also applies to the page annotation device based on the large model in this embodiment, so it will not be elaborated here.

[0207] In the embodiments of the present application, by using the page screenshot of the page to be labeled and utilizing a large model to perform an overall label on the page to be labeled, overall label information is obtained. Based on the block screenshots of each page block in the page to be labeled, the page blocks are labeled to obtain block label information. According to the overall label information and the block label information, the target label information of the page to be labeled is obtained. Thus, by using a large model to label the page to be labeled from two different granularities of the overall page and page blocks, the accuracy of page labeling can be improved, and the labeling efficiency is high.

[0208] According to an embodiment of the present application, the present application further provides an electronic device, a readable storage medium, and a computer program product.

[0209] Figure 7 FIG. shows a schematic block diagram of an exemplary electronic device 700 that can be used to implement the embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0210] As Figure 7 shown, the device 700 includes a computing unit 701, which can execute various appropriate actions and processes according to the computer program stored in the ROM (Read-Only Memory) 702 or the computer program loaded from the storage unit 708 into the RAM (Random Access Memory) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The I / O (Input / Output) interface 705 is also connected to the bus 704.

[0211] Multiple components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disc, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0212] The computing unit 701 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above, such as the large model-based page annotation method. For example, in some embodiments, the large model-based page annotation method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the large model-based page annotation method described above can be executed. Alternatively, in other embodiments, the computing unit 701 can be configured to execute the large model-based page annotation method by any other suitable means (e.g., by means of firmware).

[0213] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chip), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0214] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0215] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory), or flash memory, optical fibers, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0216] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or an LCD (Liquid Crystal Display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0217] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0218] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services (Virtual Private Server). The server can also be a server of a distributed system, or a server combined with blockchain.

[0219] According to an embodiment of the present application, the present application also provides a computer program product, which, when executed by an instruction processor in the computer program product, executes the page annotation method based on a large model proposed in the above embodiments of the present application.

[0220] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and no limitation is imposed herein.

[0221] The above specific embodiments do not constitute a limitation to the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. A page annotation method based on a large model, comprising: Obtaining a page screenshot of the to-be-annotated page and a block screenshot of each page block in the to-be-annotated page; According to the page screenshot, the page to be annotated is annotated as a whole using a large model to obtain overall annotation information; According to the block screenshot, the page block is marked using the large model to obtain block marking information; According to the overall marking information and the block marking information, target marking information of the page to be marked is obtained.

2. The method of claim 1, wherein: The step of annotating the to-be-annotated page using a large model according to the page screenshot to obtain overall annotation information includes: Get the first annotation dimension and page annotation example; Generate first prompt information according to the page screenshot, the first annotation dimension and the page annotation example; The large model is used to annotate the to-be-annotated page according to the first annotation dimension based on the first prompt information to obtain the overall annotation information.

3. The method of claim 1, wherein: The step of labeling the page block using the large model according to the block screenshot to obtain the block labeling information includes: Obtain the second annotation dimension and block annotation examples; Generate second prompt information according to the block screenshot, the second annotation dimension and the block annotation example; The large model is used to annotate the page block according to the second annotation dimension based on the second prompt information to obtain block annotation information.

4. The method of claim 1, wherein: The step of acquiring target annotation information of the to-be-annotated page according to the overall annotation information and the block annotation information includes: Obtaining the first block element in the page block; Marking the first block elements to obtain element marking information; The target labeling information is acquired according to the overall labeling information, the block labeling information and the element labeling information.

5. The method of claim 4, wherein: The step of marking the first block element to obtain element marking information includes: According to the type of the first block element, obtain a third annotation dimension and an element annotation example; Generate third prompt information according to the first block element, the third annotation dimension and the element annotation example; The large model is used to label the first block elements according to the third labeling dimension based on the third prompt information to obtain the element labeling information.

6. The method according to any one of claims 1 to 5, wherein: The obtaining of a page screenshot of the to-be-annotated page and a block screenshot of each page block in the to-be-annotated page includes: In response to the format of the original description data of the to-be-annotated page being in a non-domain-specific language DSL format, converting the original description data into page DSL data; The page rendered using the page DSL data is respectively screenshotted as a whole and as a block, to obtain the page screenshot and each block screenshot.

7. The method according to any one of claims 1 to 5, further comprising: Obtain page generation requirement information; Generate demand information according to the page, and search in the page database to obtain search results; wherein the page database is constructed according to the page to be annotated and the target annotation information; An application page is generated according to the search results.

8. The method of claim 7, wherein: Generating demand information according to the page and searching the page database to obtain search results include: Matching the page generation requirement information with the overall annotation information of each page in the page database; In response to the page generation requirement information matching the overall annotation information of any page in the page database, the search result is obtained according to the any page.

9. The method of claim 8, wherein: Generating an application page according to the search result includes: Generate a second block element corresponding to a page block in any of the pages according to the page generation requirement information; The application page is generated according to the second block element and any one of the pages.

10. The method of claim 8, further comprising: In response to the page generation requirement information not matching the overall annotation information of all pages in the page database, matching the page generation requirement information with the block annotation information of the page blocks in each of the pages; In response to the page generation requirement information matching the target page block, the search result is obtained according to the target page block.

11. The method of claim 10, wherein: Generating an application page according to the search result includes: Generate a third block element corresponding to the target page block according to the page generation requirement information; Combining the target page blocks to obtain a combined page; The application page is generated according to the third block element and the combined page.

12. A page annotation device based on a large model, comprising: A first acquisition module is used to acquire a page screenshot of a page to be annotated and a block screenshot of each page block in the page to be annotated; A marking module, used to mark the to-be-marked page as a whole using a large model according to the page screenshot, to obtain overall marking information; The marking module is further used to mark the page block according to the block screenshot using the large model to obtain block marking information; The second acquisition module is used to acquire target annotation information of the to-be-annotated page according to the overall annotation information and the block annotation information.

13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

Images