Page code generation method, model training method, device and equipment

Abstract

The embodiment of the invention provides a page code generation method, a model training method, devices and equipment. The method comprises the steps of obtaining a page image of a target page; performing feature processing on the page image to obtain at least one element feature of the page image; determining coordinate information of the element features according to the element features and the page image; inputting the coordinate information of each element feature and each element feature into a cue word model to obtain at least one cue word of the target page; according to the cue word, generating and displaying a code result corresponding to the cue word; wherein each code result comprises a page code of the target page. According to the method, the page code generation efficiency and accuracy can be improved, and the maintenance cost of the page code is reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and particularly to a method for generating page code, a method for model training, an apparatus, and a device. Background Art

[0002] With the rapid development of Internet technology, Web page applications have become an important means for people to obtain information and communicate. By generating the page code of the Web page, the design and development of the Web page are carried out to build Web page applications.

[0003] In the prior art, based on the manual processing method, front-end developers manually write page codes such as HyperText Markup Language (HTML for short), Cascading Style Sheets (CSS for short), and JavaScript.

[0004] However, in the above method, the manual writing method has the problems of low efficiency, easy errors, and high maintenance costs. Summary of the Invention

[0005] Embodiments of this application provide a method for generating page code, a method for model training, an apparatus, and a device, which can improve the efficiency and accuracy of page code generation and reduce the maintenance cost of page code.

[0006] In a first aspect, an embodiment of this application provides a method for generating page code, including:

[0007] Obtain a page image of a target page; and perform feature processing on the page image to obtain at least one element feature of the page image; wherein, the element feature represents a user interface (UI) element in the target page;

[0008] According to the element feature and the page image, determine the coordinate information of the element feature; wherein, the coordinate information represents the position of the UI element represented by the element feature in the page image;

[0009] Input the coordinate information of each element feature and each element feature into a prompt model to obtain at least one prompt for the target page;

[0010] Generate and display a code result corresponding to the prompt according to the prompt; wherein, each code result includes the page code of the target page.

[0011] In a possible implementation manner, determining the coordinate information of the element feature according to the element feature and the page image includes:

[0012] Perform detection and calculation processing on the page image and the element features respectively to obtain at least one first key point in the page image and at least one second key point in the element features;

[0013] Determine at least one target matching item corresponding to the element feature from the at least one first key point and the at least one second key point in the element features; wherein, one first key point and one second key point are included in the target matching item; the similarity degree between the first key point and the second key point in each target matching item is greater than the similarity degree between each first key point and each second key point other than each target matching item;

[0014] Determine the coordinate information of the element feature from the page image according to the first key point and the second key point in each target matching item.

[0015] In a possible implementation manner, the first key point has a first descriptor; the first descriptor represents the feature description of the first key point; the second key point has a second descriptor; the second descriptor represents the feature description of the second key point;

[0016] Determining at least one target matching item corresponding to the element feature from the at least one first key point and the at least one second key point in the element features includes:

[0017] Perform matching processing on each first descriptor and each second descriptor to obtain a matching list; wherein, at least one matching item is included in the matching list, and a distance value between the first descriptor of each first key point and the second descriptor of each second key point is included in the matching item; the distance value represents the similarity degree between the first key point and the second key point corresponding to the matching item;

[0018] Determine each target matching item from the matching list; wherein, the distance value of each target matching item is less than the distance value of the matching item other than each target matching item in the matching list.

[0019] In a possible implementation manner, determining the coordinate information of the element feature from the page image according to the first key point and the second key point in each target matching item includes:

[0020] Determine the first key point in the target matching item as the third key point; and determine the key point index of the third key point in the page image;

[0021] Determine the coordinate information corresponding to the third key point in the page image according to each key point index and each first key point;

[0022] Among them, the coordinate information corresponding to each of the third key points constitutes the coordinate information of the element feature.

[0023] In a possible implementation manner, performing feature processing on the page image to obtain at least one element feature of the page image includes:

[0024] Performing feature processing on the page image based on a feature extraction model to obtain each of the element features.

[0025] In a possible implementation manner, generating and displaying a code result corresponding to the prompt word according to the prompt word includes:

[0026] Determining a code generation model corresponding to the prompt word based on a target interface; and generating a code result corresponding to the prompt word based on the code generation model;

[0027] Performing local deployment processing on each of the code results, and displaying each of the code results based on the user interface;

[0028] In response to a selection instruction of the user, determining the code result corresponding to the selection instruction as the page code of the target page; and displaying the page code.

[0029] In a possible implementation manner, the method further includes:

[0030] In response to a modification instruction of the user, performing modification processing on the prompt word corresponding to the code result indicated by the modification instruction to obtain and display the modified prompt word;

[0031] In response to the first confirmation instruction of the user, generating and displaying a code result corresponding to the modified prompt word according to the modified prompt word;

[0032] In response to the second confirmation instruction of the user, packing and storing the code result corresponding to the modified prompt word in a download list.

[0033] In a second aspect, an embodiment of the present application provides a model training method applied to page code generation, including:

[0034] Invoking an initial detection model; wherein, at least one preset bounding box is included in the initial detection model; the bounding box has width information and height information;

[0035] Optimizing the initial detection model according to the width information and height information of each of the bounding boxes to obtain a target detection model; wherein, the target detection model is used to process a page image of a target page as described in the first aspect to obtain the page code of the target page.

[0036] In a possible implementation manner, according to the width information and height information of each of the bounding boxes, the initial detection model is optimized to obtain a target detection model, including:

[0037] Determine at least one initial anchor box from each of the bounding boxes; wherein, the initial anchor box has width information and height information;

[0038] According to the width information and height information of the bounding boxes, and the width information and height information of each of the initial anchor boxes, determine the approximate anchor box corresponding to the bounding box from each of the initial anchor boxes; wherein, the approximate anchor box is the initial anchor box closest to the bounding box;

[0039] Optimize the initial detection model according to the width information and height information of each of the bounding boxes corresponding to the approximate anchor boxes to obtain the target detection model.

[0040] In a possible implementation manner, according to the width information and height information of the bounding boxes, and the width information and height information of each of the initial anchor boxes, determining the approximate anchor box corresponding to the bounding box from each of the initial anchor boxes includes:

[0041] According to the width information and height information of the bounding boxes, and the width information and height information of each of the initial anchor boxes, determine the area set corresponding to the bounding box; wherein, the area set includes the overlapping area and the combined area between the bounding box and each of the initial anchor boxes; the overlapping area is the area of the overlapping part between the bounding box and the initial anchor box; the combined area is the total area after combining the bounding box and the initial anchor box;

[0042] According to the area set corresponding to the bounding box, determine the ratio set corresponding to the bounding box; wherein, the ratio set includes the ratio values between the overlapping area and the combined area between the bounding box and each of the initial anchor boxes;

[0043] According to the ratio set corresponding to the bounding box, determine the initial anchor box with the largest ratio value among each of the initial anchor boxes as the approximate anchor box corresponding to the bounding box.

[0044] In a possible implementation manner, according to the width information and height information of each of the bounding boxes corresponding to the approximate anchor boxes, optimizing the initial detection model to obtain the target detection model includes:

[0045] Calculate the average value of the width information of each of the bounding boxes corresponding to the approximate anchor box to obtain the average width value corresponding to the approximate anchor box; and calculate the average value of the height information of each of the bounding boxes corresponding to the approximate anchor box to obtain the average height value corresponding to the approximate anchor box;

[0046] Update the proximity anchor boxes according to the average width and average height corresponding to the proximity anchor boxes to obtain the target detection model.

[0047] In a third aspect, an embodiment of the present application provides a page code generation device, including:

[0048] A processing module, configured to obtain a page image of a target page; and perform feature processing on the page image to obtain at least one element feature of the page image; wherein the element feature characterizes a user interface (UI) element in the target page;

[0049] A determination module, configured to determine coordinate information of the element feature according to the element feature and the page image; wherein the coordinate information characterizes the position of the UI element represented by the element feature in the page image;

[0050] A generation module, configured to input the coordinate information of each element feature and each element feature into a prompt model to obtain at least one prompt word for the target page;

[0051] A display module, configured to generate and display a code result corresponding to the prompt word according to the prompt word; wherein each code result includes the page code of the target page.

[0052] In a possible implementation manner, the determination module is specifically configured to: perform detection and calculation processing on the page image and the element feature respectively to obtain at least one first key point in the page image and at least one second key point in the element feature; determine at least one target matching item corresponding to the element feature from the at least one first key point and the at least one second key point in the element feature; wherein the target matching item includes a first key point and a second key point; the similarity degree between the first key point and the second key point in each target matching item is greater than the similarity degree between each first key point and each second key point other than each target matching item; determine the coordinate information of the element feature from the page image according to the first key point and the second key point in each target matching item.

[0053] In a possible implementation manner, the first key point has a first descriptor; the first descriptor characterizes the feature description of the first key point; the second key point has a second descriptor; the second descriptor characterizes the feature description of the second key point;

[0054] The determining module is specifically configured to: perform matching processing on each of the first descriptors and each of the second descriptors to obtain a matching list; wherein, the matching list includes at least one matching item, and each matching item includes a distance value between the first descriptor of each first key point and the second descriptor of each second key point; the distance value characterizes the similarity degree between the first key point and the second key point corresponding to the matching item; determine each target matching item from the matching list; wherein, the distance values of each target matching item are all smaller than the distance values of the matching items other than each target matching item in the matching list.

[0055] In a possible implementation manner, the determining module is further specifically configured to: determine the first key point in the target matching item as a third key point; and determine the key point index of the third key point in the page image; determine the coordinate information corresponding to the third key point in the page image according to each key point index and each first key point; wherein, the coordinate information corresponding to each third key point constitutes the coordinate information of the element feature.

[0056] In a possible implementation manner, the processing module is specifically configured to: perform feature processing on the page image based on a feature extraction model to obtain each element feature.

[0057] In a possible implementation manner, the display module is specifically configured to: determine a code generation model corresponding to the prompt word based on a target interface; and generate a code result corresponding to the prompt word based on the code generation model; perform local deployment processing on each code result, and display each code result based on the user interface; in response to a selection instruction of a user, determine the code result corresponding to the selection instruction as the page code of the target page; and display the page code.

[0058] In a possible implementation manner, the device is further configured to: in response to a modification instruction of a user, perform modification processing on the prompt word corresponding to the code result indicated by the modification instruction to obtain and display the modified prompt word; in response to the first confirmation instruction of the user, generate and display a code result corresponding to the modified prompt word according to the modified prompt word; in response to the second confirmation instruction of the user, package and store the code result corresponding to the modified prompt word in a download list.

[0059] In a fourth aspect, an embodiment of the present application provides a model training device applied to page code generation, including:

[0060] A calling module, configured to call an initial detection model; wherein, the initial detection model includes at least one preset bounding box; the bounding box has width information and height information;

[0061] An optimization module that optimizes the initial detection model according to the width information and height information of each of the bounding boxes to obtain a target detection model; wherein, the target detection model is used to process the page image of the target page as described in the third aspect to obtain the page code of the target page.

[0062] In a possible implementation manner, the optimization module is specifically configured to: determine at least one initial anchor box from each of the bounding boxes; wherein, the initial anchor box has width information and height information; according to the width information and height information of the bounding boxes and the width information and height information of each of the initial anchor boxes, determine a proximity anchor box corresponding to the bounding box from each of the initial anchor boxes; wherein, the proximity anchor box is the initial anchor box closest to the bounding box; optimize the initial detection model according to the width information and height information of each of the bounding boxes corresponding to each of the proximity anchor boxes to obtain the target detection model.

[0063] In a possible implementation manner, the optimization module is specifically configured to: determine an area set corresponding to the bounding box according to the width information and height information of the bounding box and the width information and height information of each of the initial anchor boxes; wherein, the area set includes the overlapping area and the combined area between the bounding box and each of the initial anchor boxes; the overlapping area is the area of the overlapping part between the bounding box and the initial anchor box; the combined area is the total area after the bounding box and the initial anchor box are combined; determine a ratio set corresponding to the bounding box according to the area set corresponding to the bounding box; wherein, the ratio set includes the ratio values between the overlapping area and the combined area between the bounding box and each of the initial anchor boxes; determine the initial anchor box with the largest ratio value among each of the initial anchor boxes according to the ratio set corresponding to the bounding box as the proximity anchor box corresponding to the bounding box.

[0064] In a possible implementation manner, the optimization module is further specifically configured to: calculate the average value of the width information of each of the bounding boxes corresponding to the proximity anchor box to obtain the average width value corresponding to the proximity anchor box; and calculate the average value of the height information of each of the bounding boxes corresponding to the proximity anchor box to obtain the average height value corresponding to the proximity anchor box; update the proximity anchor box according to the average width value and the average height value corresponding to the proximity anchor box to obtain the target detection model.

[0065] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0066] The memory stores computer execution instructions;

[0067] The processor executes the computer-executable instructions stored in the memory, such that the processor executes the method according to the first aspect and / or the second aspect as described above.

[0068] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to the first aspect and / or the second aspect as described above.

[0069] In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method according to the first aspect and / or the second aspect as described above.

[0070] The page code generation method, model training method, device, and equipment provided by the embodiments of the present application perform feature processing on the page image of a target page to obtain user interface (UI) elements in the target page, and based on each UI element and the page image, obtain the position of each UI element image in the page image, and input all the UI elements and the positions of all the UI element images in the page image into a prompt model to obtain a plurality of prompts, so as to generate and display respective code results including the page code of the target page; by identifying UI elements and their positions in the page image through image features, corresponding prompts and code results corresponding to the prompts can be quickly and automatically generated, thereby determining the page code of the target page, which can improve the efficiency and accuracy of page code generation and reduce the maintenance cost of page code. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0072] Figure 1 It is a schematic diagram of an application scenario provided by the present application;

[0073] Figure 2 It is a schematic flowchart of a page code generation method provided by an embodiment of the present application;

[0074] Figure 3 It is a schematic flowchart of another page code generation method provided by an embodiment of the present application;

[0075] Figure 4 It is a schematic diagram of a page code generation process provided by an embodiment of the present application;

[0076] Figure 5 It is a schematic flowchart of a model training method applied to page code generation provided by an embodiment of the present application;

[0077] Figure 6A schematic flowchart of another model training method for page code generation provided by an embodiment of the present application;

[0078] Figure 7 A schematic diagram of a model training process provided by an embodiment of the present application;

[0079] Figure 8 A schematic structural diagram of a page code generation device provided by an embodiment of the present application;

[0080] Figure 9 A schematic structural diagram of a model training device for page code generation provided by an embodiment of the present application;

[0081] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0082] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0083] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0084] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties. Moreover, the processing of relevant data such as collection, use, processing, transmission, provision, disclosure, and application complies with relevant laws, regulations, and standards, takes necessary confidentiality measures, does not violate public order and good customs, and provides corresponding operation entrances for users to choose to authorize or refuse.

[0085] Furthermore, for the technical solution of the present application that involves big data analysis of user information (including but not limited to personal biometric features, identity data, consumption data, asset data, electronic terminal operation data, etc.) and uses artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights and interests based on the results of automated decision-making, corresponding operation entrances are provided for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0086] It should be noted that this application can be used in the field of artificial intelligence or any field other than artificial intelligence, and the application field of this application is not limited.

[0087] Figure 1 As a schematic diagram of an application scenario provided for this application, as Figure 1 shown, the specific application scenarios of this application include: Based on the electronic device 101, the user 102 can generate the page code of the Web page to design and develop the Web page for building a Web page application.

[0088] Combined with the above scenarios, in the prior art, by manually writing page codes such as HTML, CSS, and JavaScript by front-end developers, there are problems of low efficiency, easy errors, and high maintenance costs.

[0089] The page code generation method provided by this application can identify UI elements and their positions in the page image through image features, quickly and automatically generate corresponding prompt words and the code results corresponding to the prompt words, so as to determine the page code of the target page, and solve the technical problems of low efficiency, easy errors, and high maintenance costs caused by manually writing page codes.

[0090] The technical solution of this application and how this technical solution solves the above technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0091] Figure 2 As a schematic flowchart of a page code generation method provided by an embodiment of this application, as Figure 2 shown, the method includes:

[0092] 201. Obtain the page image of the target page; and perform feature processing on the page image to obtain at least one element feature of the page image; wherein, the element feature represents the user interface (UI) element in the target page.

[0093] Exemplarily, the execution subject of this embodiment can be an electronic device. The electronic device is hereinafter referred to as the device. The user uploads the page image of the designed target page to the device. After the device obtains the page image, based on the image feature processing technology, the device performs feature processing on the page image to identify at least one user interface (User Interface, hereinafter referred to as UI) element in the target page, so as to obtain at least one element feature of the page image.

[0094] 202. Determine the coordinate information of the element features according to the element features and the page image; wherein, the coordinate information represents the position of the UI element represented by the element features in the page image.

[0095] Exemplarily, the device performs computational processing on the obtained page image and each element feature based on image processing technology to obtain the position of the UI element represented by each element feature in the page image, that is, the coordinate information of each element feature. For example, the coordinates (x, y) of each pixel point in the UI element in the corresponding image coordinate system of the page image.

[0096] 203. Input the coordinate information of each element feature and each element feature into the prompt word model to obtain at least one prompt word for the target page.

[0097] Exemplarily, call the preset prompt word model, and input the coordinate information of all element features and all element features into the prompt word model, so that the prompt word model performs prompt word generation processing on the coordinate information of all element features and all element features to obtain at least one prompt word for the target page, such as all prompt words of the entire web page Web.

[0098] For example, after obtaining the element features of the Web and their coordinates, preprocess these data and divide these data into a training data set and a validation data set. Define the training model Mi corresponding to the i-th accessed target code generation model, so that it can generate corresponding prompt words for different target code generation models. The present invention is based on the T5 model of Transformer and combines the multi-modal model ViT + BERT of visual and text information to ensure that the model can effectively process image and text data and generate high-quality prompt words. Among them,

[0099] ,

[0100] Then, define the loss function L to measure the difference between the prompt word predicted by the model and the true prompt word. , and select the optimizer Adam to update the model parameters optimizer. ; Use the preprocessed data set for model training, and through multiple epoch iterations, gradually optimize the model parameters until the predetermined convergence condition is reached. The training steps are as follows:

[0101] for epoch in range(num_epochs):

[0102] for(I latch ,L latch ,P latch ) in data_loader:

[0103] I batch_processed = preprocess_image(I batch , (W, H), interpolation = BILINEAR, method = LUMINANCE, σ = 1.0)

[0104] P batch_processed = preprocess_text(P batch , tokenizer = BERTTokenizer, stopwords = EnglishStopwords)

[0105] P predicted = M(I batch_processed , I batch_processed )

[0106] L = cross_entropy_loss(P predicted , P ground_truth )

[0107] optimizer.zero g rad()

[0108] L.backward()

[0109] optimizer.step().

[0110] Evaluate the model performance using the validation set or test set, calculate the evaluation metrics, and conduct manual evaluation to obtain the trained model; input the obtained element features and their coordinate information into each trained model, and different prompt phrases generated by different models will be obtained. Furthermore, through intelligent page image recognition, corresponding prompt words can be quickly and automatically generated.

[0111] 204. Generate and display the code results corresponding to the prompt words; among them, each code result includes the page code of the target page.

[0112] Exemplarily, set to call the corresponding target large model according to each obtained prompt word to perform code generation processing on each prompt word, obtain the code results corresponding to each prompt word, and integrate and display all the generated code results, then the page code of the target page can be obtained. For example, generate and display all the static codes of the Web page.

[0113] In this embodiment, a method for generating page code is provided. By recognizing UI elements and their positions in the page image through image features, corresponding prompt words and code results corresponding to the prompt words can be quickly and automatically generated, so as to determine the final page code, which can improve the efficiency and accuracy of page code generation and reduce the maintenance cost of page code.

[0114] Figure 3 As shown in the flowchart of another method for generating page code provided by the embodiment of the present application, as Figure 3 shown, the method includes:

[0115] 301. Obtain the page image of the target page; and based on the feature extraction model, perform feature processing on the page image to obtain each element feature.

[0116] Exemplarily, the user uploads the page image of the designed target page to the device. After the device obtains the page image, it calls the feature extraction model and, based on the feature extraction model, performs feature processing on the page image to identify at least one UI element in the target page, so as to obtain at least one element feature of the page image.

[0117] 302. Perform detection and calculation processing on the page image and the element features respectively to obtain at least one first key point in the page image and at least one second key point in the element features.

[0118] Exemplarily, the device matches the page image with each element feature and uses a selected feature detector to perform detection and calculation processing on the page image to obtain at least one key point in the page image, that is, the first key point, and based on the feature detector, perform detection and calculation processing on each element feature to obtain at least one key point in the element features, that is, the second key point.

[0119] 303. Determine at least one target matching item corresponding to the element feature from at least one first key point and at least one second key point in the element features; wherein, a target matching item includes a first key point and a second key point; the similarity degree between the first key point and the second key point in each target matching item is greater than the similarity degree between each first key point and each second key point other than each target matching item.

[0120] Exemplarily, the device uses a preset matcher to match each first key point with each second key point corresponding to each element feature to obtain the similarity degree between each first key point and each second key point, and determines multiple first key points and second key points whose similarity degrees are greater than the similarity degrees between each other first key point and each other second key point as multiple target matching items corresponding to each element feature.

[0121] For example, the device uses a preset matcher to match each of the first key points 1, 2, 3 and each of the second key points 4, 5, 6, and obtains the similarity degree between each first key point and each second key point, including 1-4, 2-4, 2-5, 3-4, 1-5, 1-6, 2-6, 3-5, 3-6 sorted from high to low similarity degree. Then, the top 3 combinations with the highest similarity degrees, namely 1-4, 2-4, and 2-5, can be determined as the 3 target matching items.

[0122] In one example, the first key point has a first descriptor; the first descriptor represents the feature description of the first key point; the second key point has a second descriptor; the second descriptor represents the feature description of the second key point; step 303 includes the following steps:

[0123] The first step is to perform a matching process on each first descriptor and each second descriptor to obtain a matching list; wherein, the matching list includes at least one matching item, and the matching item includes the distance value between the first descriptor of each first key point and the second descriptor of each second key point; the distance value represents the similarity degree between the first key point and the second key point corresponding to the matching item.

[0124] The second step is to determine each target matching item from the matching list; wherein, the distance value of each target matching item is less than the distance value of the matching items other than each target matching item in the matching list.

[0125] Exemplarily, each first key point has a first descriptor to represent the feature description of each first key point, such as a feature vector used to describe the area around the first key point; each second key point has a second descriptor to represent the feature description of each second key point, such as a feature vector used to describe the area around the second key point; wherein, each descriptor can be a 128-dimensional floating-point vector or a binary descriptor. The device uses a matcher to match the first descriptor of each first key point and the second descriptor of each second key point, obtains each matching item, and calculates the distance value between the first descriptor of each first key point and the second descriptor of each second key point in each matching item to represent the similarity degree between the first key point and the second key point corresponding to the matching item; according to the distance values in each matching item, a preset number of matching items with high distance values are determined as the target matching items, that is, the distance value of each target matching item is less than the distance value of the matching items other than each target matching item in the matching list.

[0126] For example, using a selected feature detector F, key points Ko are detected on the original image Io, and the descriptor Do of each key point is calculated through Ko, Do = F(Io). Similarly, key points Kt are detected on the elemental feature It, and the descriptor Dt of each key point is calculated through Kt, Dt = F(It). Using a matcher M, the descriptor Do of the original image is matched with the descriptor Dt of the feature through M list = M(Do, Dt) to generate a matching list M list . Then, according to the distance values of the matching items, the matching list M list is sorted to obtain a matching list M sorted in ascending order of distance sorted . The smaller the distance value of the matching item, the more similar the two key points are. Select the top N best matching items M sorted from the sorted matching list M best = M sorted [:N] to obtain M containing each target matching item best .

[0127] 304. According to the first key point and the second key point in each target matching item, determine the coordinate information of the elemental feature from the page image.

[0128] Exemplarily, the device processes the first key point and the second key point in all target matching items corresponding to each elemental feature through a preset detection algorithm. For example, through a preset deep learning model, the page image and the first descriptor of the first key point and the second descriptor of the second key point in all target matching items are processed to obtain the coordinate information of each elemental feature in the page image.

[0129] In one example, step 304 includes the following steps:

[0130] The first step: Determine the first key point in the target matching item as the third key point; and determine the key point index of the third key point in the page image.

[0131] The second step: According to each key point index and each first key point, determine the coordinate information corresponding to the third key point in the page image.

[0132] Among them, the coordinate information corresponding to each third key point constitutes the coordinate information of the elemental feature.

[0133] Exemplarily, the device determines the first key point in each target match item corresponding to each element feature as the third key point, and based on each third key point, determines the key point index of each third key point from the page image, such as position (coordinates (x, y) of the key point in the image), scale (scale information of the key point, indicating the size of the feature), orientation (main direction of the key point, for rotation invariance), response value (response intensity of the key point, indicating the significance of the point as a key point), and feature descriptor (feature vector for describing the area around the key point). The device performs calculation processing on the respective key point indexes and the respective first key points corresponding to each element feature according to the key point detection algorithm, and obtains the coordinate information of each third key point corresponding to each element feature in the page image, that is, the coordinate information of all the third key points corresponding to each element feature in the page image constitutes the coordinate information of each element feature.

[0134] 305. Based on the target interface, determine the code generation model corresponding to the prompt word; and based on the code generation model, generate the code result corresponding to the prompt word.

[0135] Exemplarily, the device calls the target interface corresponding to each prompt word according to each prompt word, and based on each target interface, determines the code generation model corresponding to each prompt word. According to each code generation model and the corresponding prompt word, the device can generate the code result corresponding to each prompt word, and this code result.

[0136] 306. Perform local deployment processing on each code result, and display each code result based on the user interface.

[0137] Exemplarily, the device deploys each generated code result to the local simulation environment respectively and displays them one by one on the user interface for the user to view and select.

[0138] 307. In response to the user's selection instruction, determine the code result corresponding to the selection instruction as the page code of the target page; and display the page code.

[0139] Exemplarily, the user operates the user interface according to the displayed code results or inputs a selection instruction to the device by voice input. In response to the selection instruction, the device determines the code result corresponding to the selection instruction as the final page code of the target page and displays the page code to the user based on the user interface of the device. Furthermore, the development efficiency of Web developers is greatly improved and the development cost is reduced.

[0140] In one example, it further includes:

[0141] Step 1: In response to the user's modification instruction, modify the prompt word corresponding to the code result indicated by the modification instruction, and obtain and display the modified prompt word.

[0142] Step 2: In response to the user's first confirmation instruction, generate and display the code result corresponding to the modified prompt word according to the modified prompt word.

[0143] Step 3: In response to the user's second confirmation instruction, package and store the code result corresponding to the modified prompt word in the download list.

[0144] Specifically, when displaying all code results to the user, after the user selects the page corresponding to the relatively satisfactory code result, the prompt word corresponding to this page appears below the system. The user can adjust and modify the prompt word according to the requirements, and then input a modification instruction to the device, enabling the device to modify the prompt word corresponding to the code result indicated by the modification instruction. For example, the user inputs the modified prompt word to the device, and the device updates the original prompt word to obtain and display the modified prompt word. After the user confirms that the modified prompt word is correct, the user inputs a first confirmation instruction to the device. For example, the user can click the submit button, and the device generates and displays the code result corresponding to the modified prompt word according to the modified prompt word. For example, according to the modified prompt word, the corresponding code generation model is called to generate the corresponding code result. After the device displays the page corresponding to the new code result to the user, if the user confirms that there is no modification, the user inputs a second confirmation instruction to the device. For example, the user can click the submit button, and the device packages and stores the code result corresponding to the modified prompt word in the download list for the user to download and process.

[0145] For example, Figure 4 is a schematic diagram of a page code generation process provided by an embodiment of the present application. As Figure 4 shown, the user uploads the pattern designed by the art designer to the optimized feature extraction model for Web UI element feature extraction. The extracted features and the original image are calculated to obtain the coordinate information of each feature in the image. The extracted features and coordinate information are input into the large model prompt word training model designed by the present invention to obtain all the prompt words of the entire web page. The system calls a third-party large model for corresponding code generation according to the obtained prompt words, and integrates and displays the generated code. The user judges whether adjustment is needed according to the displayed result. If adjustment is needed, the corresponding prompt word is modified, and then the third-party large model is called again for new code generation and display. After the user finishes adjusting, the user clicks to confirm, and the system automatically generates the corresponding Web page static code for the user to download. The user clicks to download the code and deploys it to their own development environment or running environment.

[0146] In this embodiment, based on the above embodiment, on the one hand, users do not need to reserve professional knowledge, and can adjust and modify the generated page code by modifying the prompt words. On the other hand, based on the interface call code generation model, the development cost can be effectively reduced, and at the same time, the latest code generation model can be accessed at any time for users to select and generate corresponding prompt words and corresponding page codes.

[0147] Figure 5 It is a schematic flow chart of a model training method applied to page code generation provided by an embodiment of the present application. As Figure 5 shown, the method includes:

[0148] 401. Call an initial detection model; wherein, the initial detection model includes at least one preset bounding box; the bounding box has width information and height information.

[0149] Exemplarily, in order to better identify each Web UI control element in the image, the device calls the initial detection model, which includes at least one preset bounding box generated based on the anchor box design algorithm, and each bounding box has width information and height information.

[0150] 402. Optimize the initial detection model according to the width information and height information of each bounding box to obtain a target detection model; wherein, the target detection model is used to process the page image of the target page in the page code generation method to obtain the page code of the target page.

[0151] Exemplarily, in view of the characteristics that the shapes and sizes of Web UI control elements are different, the algorithm for designing the anchor box in the initial model is optimized according to the width information and height information of each bounding box to obtain the optimized initial detection model, that is, the target detection model, which is used to process the page image of the target page in the page code generation method to obtain the page code of the target page.

[0152] In this embodiment, a model training method applied to page code generation is provided. The initial detection model is optimized according to the width information and height information of each bounding box in the initial detection model to obtain a target detection model, which is used to perform image features, identify UI elements and their positions in the page image, and then can quickly and automatically generate corresponding prompt words and code results corresponding to the prompt words, so as to determine the final page code, which can improve the efficiency and accuracy of page code generation and reduce the maintenance cost of page code.

[0153] Figure 6 It is a schematic flow chart of another model training method applied to page code generation provided by an embodiment of the present application. As Figure 6 shown, the method includes:

[0154] 501. Call the initial detection model, where the initial detection model includes at least one preset bounding box, and the bounding box has width information and height information.

[0155] Exemplarily, this step can refer to step 201, which will not be elaborated here.

[0156] 502. Determine at least one initial anchor box from each bounding box, where the initial anchor box has width information and height information.

[0157] Exemplarily, select a set of initial anchor boxes from each bounding box , where each ai represents an anchor box in the form of (w, h). k is a preset value, tentatively set to 9 in the present invention, where w is the width and h is the height.

[0158] 503. Determine the approximate anchor box corresponding to the bounding box from each initial anchor box according to the width information and height information of the bounding box and the width information and height information of each initial anchor box, where the approximate anchor box is the initial anchor box closest to the bounding box.

[0159] Exemplarily, calculate and process the width information and height information of each bounding box and the width information and height information of all initial anchor boxes to obtain the degree of closeness between each bounding box and each initial anchor box, and then the initial anchor box closest to each bounding box, that is, the approximate anchor box, can be determined from each initial anchor box.

[0160] In one example, step 503 includes the following steps:

[0161] The first step: Determine the area set corresponding to the bounding box according to the width information and height information of the bounding box and the width information and height information of each initial anchor box, where the area set includes the overlapping area and the combined area between the bounding box and each initial anchor box; the overlapping area is the area of the overlapping part between the bounding box and the initial anchor box; the combined area is the total area after the bounding box and the initial anchor box are combined.

[0162] The second step: Determine the ratio set corresponding to the bounding box according to the area set corresponding to the bounding box, where the ratio set includes the ratio values between the overlapping area and the combined area between the bounding box and each initial anchor box.

[0163] The third step: Determine the initial anchor box with the largest ratio value among each initial anchor box according to the ratio set corresponding to the bounding box as the approximate anchor box corresponding to the bounding box.

[0164] Exemplarily, the device performs area calculation processing on the width information and height information of each bounding box and the width information and height information of each initial anchor box according to a preset anchor box algorithm, and can calculate the area of the overlapping part between each bounding box and each initial anchor box, that is, the overlapping area, and can also calculate the total area after merging each bounding box and each initial anchor box, that is, the merging area, and further obtain the area set corresponding to each bounding box. According to the area set corresponding to each bounding box, determine the ratio value between the overlapping area and the merging area between each bounding box and each initial anchor box, and obtain the ratio set corresponding to each bounding box. According to the ratio set corresponding to each bounding box, determine the initial anchor box with the largest ratio value among the initial anchor boxes, and determine that the initial anchor box is the approximate anchor box corresponding to the bounding box.

[0165] For example, through the formula:

[0166] ;

[0167] Calculate the ratio (Intersection over Union, IoU) of the overlapping area of each bounding box and each initial anchor box to the total area after their merging to determine the closest anchor box; where w a and h a represent the width and height of box a, w b and h b represent the width and height of box b, the Intersection over Union. For each bounding box b j , find the initial anchor box a i closest to it, where "closest" means the largest IoU (Intersection over Union), and obtain , where C i represents the bounding box assigned to the initial anchor box a i .

[0168] 504. Optimize the initial detection model according to the width information and height information of each bounding box corresponding to each approximate anchor box to obtain the target detection model.

[0169] Exemplarily, after determining the approximate anchor box corresponding to each bounding box, that is, all the bounding boxes corresponding to each approximate anchor box can be determined. Based on a preset algorithm, optimize the initial detection model according to the width information and height information of all the bounding boxes corresponding to each approximate anchor box to obtain the target detection model.

[0170] In one example, step 504 includes the following steps:

[0171] The first step: Calculate the average value of the width information of each bounding box corresponding to the approximate anchor box to obtain the average width value corresponding to the approximate anchor box; and calculate the average value of the height information of each bounding box corresponding to the approximate anchor box to obtain the average height value corresponding to the approximate anchor box.

[0172] Second step: Update the proximity anchor boxes according to the average width and average height corresponding to the proximity anchor boxes to obtain the target detection model.

[0173] Exemplarily, after determining the proximity anchor box corresponding to each bounding box, all the bounding boxes corresponding to each proximity anchor box can be determined. Calculate the average value of the width information of each bounding box corresponding to each proximity anchor box to obtain the average width corresponding to each proximity anchor box. Calculate the average value of the height information of each bounding box corresponding to each proximity anchor box to obtain the average height corresponding to each proximity anchor box. Update the proximity anchor box according to the average width and average height corresponding to each proximity anchor box to obtain the updated proximity anchor box, and thus complete the training of the initial detection model to obtain the target detection model.

[0174] For example, Figure 7 is a schematic diagram of a model training process provided by an embodiment of the present application. As Figure 7 shown, select a set of initial anchor boxes , where each represents an anchor box in the form of (w, h). k is a preset value, which is tentatively set to 9 in the present invention, where w is the width and h is the height. For each bounding box b j , calculate its IoU with each anchor box, and assign b j to calculate the average width and height of all the bounding boxes in each set C i . Where C i represents the set of bounding boxes assigned to the anchor box with the largest IoU, forming each corresponding set C i . Calculate the average width and height of all the bounding boxes in each set C i , and update the position of each anchor box through the formula: , so that it becomes the average value of all the bounding boxes in its corresponding set C i , and thus the new anchor box can be obtained. The updated set of anchor boxes is denoted as ; where C i represents the bounding boxes assigned to the anchor box , |C i | represents the number of elements in the set C i , and b[0] and b[1] respectively represent the width and height of the bounding box b. If A' is the same as A in the previous iteration, stop the iteration; otherwise, use A' as the new set of anchor boxes to continue the iteration, and output the final set of anchor boxes.

[0175] It should be added that the above implementation process is described as follows:

[0176] # Initialization

[0177] k = 9 # Assume we need 9 anchor boxes

[0178] boxes = [Bounding boxes in all training sets] # Each bounding box is a (w, h) tuple

[0179] anchors = Randomly select k bounding boxes as initial center points

[0180] prev_assignments = None

[0181] assignments = None

[0182] while assignments != prev_assignments:

[0183] prev_assignments = assignments

[0184] assignments = []

[0185] # Assign the nearest anchor box to each bounding box

[0186] for box in boxes:

[0187] best_anchor, max_iou = None, 0

[0188] for anchor in anchors:

[0189] iou = calculate_iou(box, anchor)

[0190] if iou > max_iou:

[0191] max_iou = iou

[0192] best_anchor = anchor

[0193] assignments.append(best_anchor)

[0194] # Update the positions of the anchor boxes

[0195] for i in range(k):

[0196] assigned_boxes = [box for box, anchor in zip(boxes,assignments) if anchor == anchors[i]]

[0197] if assigned_boxes:

[0198] new_anchor = average(assigned_boxes)

[0199] anchors[i] = new_anchor

[0200] # Output the final anchor boxes

[0201] print("Final anchors:", anchors)。

[0202] In this embodiment, based on the above embodiment, by optimizing the anchor box algorithm in the initial detection model according to the width information and height information of each bounding box and the width information and height information of each initial anchor box, a target detection model can be obtained for page code generation, which can improve the efficiency and accuracy of page code generation and reduce the maintenance cost of page code.

[0203] Figure 8 The following is a schematic structural diagram of a page code generation device provided by an embodiment of the present application. As Figure 8 shown, the device includes:

[0204] A processing module 601, configured to obtain a page image of a target page; and perform feature processing on the page image to obtain at least one element feature of the page image; wherein, the element feature represents a user interface (UI) element in the target page;

[0205] A determination module 602, configured to determine the coordinate information of the element feature according to the element feature and the page image; wherein, the coordinate information represents the position of the UI element represented by the element feature in the page image;

[0206] A generation module 603, configured to input the coordinate information of each element feature and each element feature into a prompt word model to obtain at least one prompt word of the target page;

[0207] A display module 604, configured to generate and display a code result corresponding to the prompt word according to the prompt word; wherein, each code result includes the page code of the target page.

[0208] In a possible implementation, the determining module 602 is specifically configured to: perform detection and calculation processing on the page image and the element features respectively to obtain at least one first key point in the page image and at least one second key point in the element features; determine at least one target matching item corresponding to the element features from at least one first key point and at least one second key point in the element features; wherein, each target matching item includes a first key point and a second key point; the similarity degree between the first key point and the second key point in each target matching item is greater than the similarity degree between each first key point and each second key point other than each target matching item; determine the coordinate information of the element features from the page image according to the first key point and the second key point in each target matching item.

[0209] In a possible implementation, the first key point has a first descriptor; the first descriptor characterizes the feature description of the first key point; the second key point has a second descriptor; the second descriptor characterizes the feature description of the second key point.

[0210] The determining module 602 is specifically configured to: perform matching processing on each first descriptor and each second descriptor to obtain a matching list; wherein, the matching list includes at least one matching item, and each matching item includes the distance value between the first descriptor of each first key point and the second descriptor of each second key point; the distance value characterizes the similarity degree between the first key point and the second key point corresponding to the matching item; determine each target matching item from the matching list; wherein, the distance value of each target matching item is less than the distance value of the matching items other than each target matching item in the matching list.

[0211] In a possible implementation, the determining module 602 is further specifically configured to: determine the first key point in the target matching item as the third key point; and determine the key point index of the third key point in the page image; determine the coordinate information corresponding to the third key point in the page image according to each key point index and each first key point; wherein, the coordinate information corresponding to each third key point constitutes the coordinate information of the element features.

[0212] In a possible implementation, the processing module 601 is specifically configured to: perform feature processing on the page image based on a feature extraction model to obtain each element feature.

[0213] In a possible implementation, the display module 604 is specifically configured to: determine a code generation model corresponding to the prompt word based on a target interface; and generate a code result corresponding to the prompt word based on the code generation model; perform local deployment processing on each code result, and display each code result based on a user interface; in response to a selection instruction of a user, determine the code result corresponding to the selection instruction as the page code of a target page; and display the page code.

[0214] In a possible implementation, the device is further configured to: in response to a modification instruction from the user, perform a modification process on the prompt word corresponding to the code result indicated by the modification instruction to obtain and display the modified prompt word; in response to the user's first confirmation instruction, generate and display the code result corresponding to the modified prompt word according to the modified prompt word; and in response to the user's second confirmation instruction, package and store the code result corresponding to the modified prompt word in the download list.

[0215] The device in this embodiment can execute the technical solutions in the above method, and the specific implementation process and technical principle are the same, which will not be elaborated here.

[0216] Figure 9 FIG. is a schematic structural diagram of a model training device for page code generation provided by an embodiment of the present application. As Figure 9 shown, the device includes:

[0217] A calling module 701, configured to call an initial detection model; wherein, the initial detection model includes at least one preset bounding box; the bounding box has width information and height information;

[0218] An optimization module 702, which optimizes the initial detection model according to the width information and height information of each bounding box to obtain a target detection model; wherein, the target detection model is used to process the page image of the target page in the page code generation device to obtain the page code of the target page.

[0219] In a possible implementation, the optimization module 702 is specifically configured to: determine at least one initial anchor box from each bounding box; wherein, the initial anchor box has width information and height information; according to the width information and height information of the bounding box and the width information and height information of each initial anchor box, determine the approximate anchor box corresponding to the bounding box from each initial anchor box; wherein, the approximate anchor box is the initial anchor box closest to the bounding box; and optimize the initial detection model according to the width information and height information of each bounding box corresponding to each approximate anchor box to obtain a target detection model.

[0220] In a possible implementation, the optimization module 702 is specifically configured to: determine an area set corresponding to the bounding box according to the width information and height information of the bounding box, and the width information and height information of each initial anchor box; wherein, the area set includes the overlapping area and the combined area between the bounding box and each initial anchor box; the overlapping area is the area of the overlapping part between the bounding box and the initial anchor box; the combined area is the total area after combining the bounding box and the initial anchor box; determine a ratio set corresponding to the bounding box according to the area set corresponding to the bounding box; wherein, the ratio set includes the ratio values between the overlapping area and the combined area between the bounding box and each initial anchor box; determine the initial anchor box with the largest ratio value among the initial anchor boxes according to the ratio set corresponding to the bounding box, and use it as the approximate anchor box corresponding to the bounding box.

[0221] In a possible implementation, the optimization module 702 is further specifically configured to: calculate the average value of the width information of each bounding box corresponding to the approximate anchor box to obtain the average width corresponding to the approximate anchor box; and calculate the average value of the height information of each bounding box corresponding to the approximate anchor box to obtain the average height corresponding to the approximate anchor box; update the approximate anchor box according to the average width and average height corresponding to the approximate anchor box to obtain the target detection model.

[0222] The device in this embodiment can execute the technical solutions in the above method. The specific implementation process and technical principle are the same, and will not be elaborated here.

[0223] Figure 10 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present application, as Figure 5 shown, the electronic device includes: a memory 801 and a processor 802; the memory 801 is a memory for storing executable instructions of the processor 802.

[0224] Wherein, the processor 802 is configured to execute the method provided in the above embodiment.

[0225] The electronic device further includes a receiver 803 and a transmitter 804. The receiver 803 is used to receive instructions and data sent by other devices, and the transmitter 804 is used to send instructions and data to external devices.

[0226] For the specific implementation process of the processor, reference can be made to the above method embodiment. The implementation principle and technical effects are similar, and will not be elaborated here in this embodiment.

[0227] An embodiment of the present application further provides a chip for running instructions. The chip is used to execute the technical solutions of the processing method in the above embodiment.

[0228] An embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, which, when running on a computer, cause the computer to execute the technical solutions of the processing method in the above embodiment.

[0229] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0230] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application-specific integrated circuit. Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0231] An embodiment of the present application further provides a computer program product including a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when at least one processor executes the computer program, the technical solutions of the processing method in the above embodiment can be implemented.

[0232] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments.

[0233] Finally, it should be noted that those skilled in the art will readily think of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for generating page code, characterized in that Including: Obtain a page image of a target page; and perform feature processing on the page image to obtain at least one element feature of the page image; wherein, the element feature characterizes a user interface (UI) element in the target page; Determine coordinate information of the element feature according to the element feature and the page image; wherein, the coordinate information characterizes the position of the UI element represented by the element feature in the page image; Input the coordinate information of each element feature and each element feature into a prompt word model to obtain at least one prompt word of the target page; Generate and display code results corresponding to the prompt words according to the prompt words; wherein, each code result includes page code of the target page.

2. The method according to claim 1, characterized in that, Determine coordinate information of the element feature according to the element feature and the page image, including: Perform detection and calculation processing on the page image and the element feature respectively to obtain at least one first key point in the page image and at least one second key point in the element feature; Determine at least one target matching item corresponding to the element feature from the at least one first key point and the at least one second key point in the element feature; wherein, the target matching item includes one first key point and one second key point; the similarity degree between the first key point and the second key point in each target matching item is greater than the similarity degree between each first key point and each second key point other than each target matching item; Determine the coordinate information of the element feature from the page image according to the first key point and the second key point in each target matching item.

3. The method according to claim 2, characterized in that, The first key point has a first descriptor; the first descriptor characterizes the feature description of the first key point; the second key point has a second descriptor; the second descriptor characterizes the feature description of the second key point; Determine at least one target matching item corresponding to the element feature from the at least one first key point and the at least one second key point in the element feature, including: Perform matching processing on each first descriptor and each second descriptor to obtain a matching list; wherein, the matching list includes at least one matching item, and the matching item includes a distance value between the first descriptor of each first key point and the second descriptor of each second key point; the distance value characterizes the similarity degree between the first key point and the second key point corresponding to the matching item; Determine each target matching item from the matching list; wherein, the distance value of each target matching item is less than the distance value of the matching item other than each target matching item in the matching list.

4. The method according to claim 2, characterized in that, Determine the coordinate information of the element feature from the page image according to the first key point and the second key point in each target matching item, including: Determine the first key point in the target matching item as a third key point; and determine the key point index of the third key point in the page image; Determine the coordinate information corresponding to the third key point in the page image according to each key point index and each first key point. Among them, the coordinate information corresponding to each of the third key points constitutes the coordinate information of the element feature.

5. The method according to claim 1, characterized in that Performing feature processing on the page image to obtain at least one element feature of the page image, including: Performing feature processing on the page image based on a feature extraction model to obtain each of the element features.

6. The method according to claim 1, wherein Generating and displaying a code result corresponding to the prompt word according to the prompt word, including: Based on a target interface, determining a code generation model corresponding to the prompt word; and based on the code generation model, generating a code result corresponding to the prompt word; Performing local deployment processing on each of the code results, and displaying each of the code results based on the user interface; In response to a user's selection instruction, determining the code result corresponding to the selection instruction as the page code of the target page; and displaying the page code.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: In response to a user's modification instruction, performing modification processing on the prompt word corresponding to the code result indicated by the modification instruction to obtain and display the modified prompt word; In response to the user's first confirmation instruction, generating and displaying a code result corresponding to the modified prompt word according to the modified prompt word; In response to the user's second confirmation instruction, packing and storing the code result corresponding to the modified prompt word in a download list.

8. A model training method applied to page code generation, characterized in that, Including: Invoking an initial detection model; wherein, at least one preset bounding box is included in the initial detection model; the bounding box has width information and height information; According to the width information and height information of each of the bounding boxes, performing optimization processing on the initial detection model to obtain a target detection model; wherein, the target detection model is used to process the page image of the target page as described in any one of claims 1-7 to obtain the page code of the target page.

9. The method according to claim 8, wherein According to the width information and height information of each of the bounding boxes, performing optimization processing on the initial detection model to obtain a target detection model, including: Determining at least one initial anchor box from each of the bounding boxes; wherein, the initial anchor box has width information and height information; According to the width information and height information of the bounding box and the width information and height information of each of the initial anchor boxes, determining a close anchor box corresponding to the bounding box from each of the initial anchor boxes; wherein, the close anchor box is the initial anchor box closest to the bounding box; According to the width information and height information of each of the bounding boxes corresponding to each of the close anchor boxes, performing optimization processing on the initial detection model to obtain the target detection model.

10. The method according to claim 9, wherein According to the width information and height information of the bounding box and the width information and height information of each of the initial anchor boxes, determining a close anchor box corresponding to the bounding box from each of the initial anchor boxes, including: Determine the area set corresponding to the bounding box according to the width information and height information of the bounding box, and the width information and height information of each of the initial anchor boxes; wherein, the area set includes the overlapping area and the combined area between the bounding box and each of the initial anchor boxes; the overlapping area is the area of the overlapping part between the bounding box and the initial anchor box; the combined area is the total area after combining the bounding box and the initial anchor box. Determine the ratio set corresponding to the bounding box according to the area set corresponding to the bounding box; wherein, the ratio set includes the ratio values between the overlapping area and the combined area between the bounding box and each of the initial anchor boxes. Determine the initial anchor box with the largest ratio value among the initial anchor boxes according to the ratio set corresponding to the bounding box as the close anchor box corresponding to the bounding box.

11. The method according to claim 9, characterized in that, Optimize the initial detection model according to the width information and height information of the bounding boxes corresponding to each of the close anchor boxes to obtain the target detection model, including: Calculate the average value of the width information of the bounding boxes corresponding to the close anchor box to obtain the average width value corresponding to the close anchor box; and calculate the average value of the height information of the bounding boxes corresponding to the close anchor box to obtain the average height value corresponding to the close anchor box. Update the close anchor box according to the average width value and average height value corresponding to the close anchor box to obtain the target detection model.

12. A page code generation device, characterized in that, Including: A processing module, configured to obtain a page image of a target page; and perform feature processing on the page image to obtain at least one element feature of the page image; wherein, the element feature characterizes a user interface (UI) element in the target page. A determination module, configured to determine the coordinate information of the element feature according to the element feature and the page image; wherein, the coordinate information characterizes the position of the UI element represented by the element feature in the page image. A generation module, configured to input the coordinate information of each of the element features and each of the element features into a prompt model to obtain at least one prompt word of the target page. A display module, configured to generate and display a code result corresponding to the prompt word according to the prompt word; wherein, each of the code results includes the page code of the target page.

13. A model training device applied to page code generation, characterized in that, Including: An invocation module, configured to invoke an initial detection model; wherein, the initial detection model includes at least one preset bounding box; the bounding box has width information and height information. An optimization module, configured to optimize the initial detection model according to the width information and height information of each of the bounding boxes to obtain a target detection model; wherein, the target detection model is used to process the page image of the target page as described in claim 12 to obtain the page code of the target page.

14. An electronic device, characterized in that, Including: A memory, a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1-11.

16. A computer program product, characterized in that, It includes a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-11.

Images