Interface generation method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, in particular to an interface generation method and device, electronic equipment and a storage medium, and aims to improve the interface generation efficiency. The method comprises: presenting an initial interface based on an initial code; for each style graph in the initial interface, inputting the style graph into a pre-trained component identification model, determining a functional component corresponding to the style graph, and determining position information of the style graph in the initial interface; for each style graph in the initial interface, determining a first code block corresponding to each style graph based on the position information of the style graph; on the basis of a second code block of a functional component corresponding to the style graph, a first code block corresponding to the style graph in the initial code is replaced, and a target code is obtained; and generating a target interface containing the plurality of functional components based on the target code. According to the method and the device, the functional component can be automatically identified and replaced to generate the required interface, so that the interface generation efficiency is improved.

Description

Background Art

[0002] In the front - end development process, usually, graphic designers first design user interface drafts, and then front - end developers manually identify several user interface (UI) components from the drafts, such as buttons, input boxes, tables, dropdown menus, etc. After that, they use code to restore the structure, style, and interaction functions of each UI component. However, this development method requires front - end developers to spend a lot of time on UI component identification and code writing, and they cannot invest more energy in business understanding and interface optimization.

[0003] Therefore, in order to reduce the burden on front - end developers, an image - to - code technology has emerged, that is, based on the user interface draft, the structure and style of UI components are automatically generated. However, this method still has many limitations. For example, this method does not understand the functions of UI components. It simply draws the above - mentioned UI components in the interface through code according to the appearance styles of the UI components presented in the user interface draft. However, the "drawn UI components" are essentially just a graphic with the same appearance as the UI component and do not have the functions of the UI component. Therefore, front - end developers still need to manually replace the code corresponding to these graphics with real UI components with interactive functions. That is to say, this method still requires front - end developers to perform a large number of repetitive tasks of manual replacement, which reduces the development efficiency. Summary of the Invention

[0004] Embodiments of the present application provide an interface generation method, device, electronic device, and storage medium to improve the efficiency of interface generation.

[0005] An interface generation method provided by embodiments of the present application includes:

[0006] Presenting an initial interface based on initial code; the initial interface includes multiple style graphics, each style graphic is drawn based on the appearance of a different functional component, and each style graphic corresponds to a first code block in the initial code;

[0007] For each style graphic in the initial interface, input the style graphic into a pre - trained component recognition model to determine the functional component corresponding to the style graphic and the position information of the style graphic in the initial interface;

[0008] For each style graphic in the initial interface, based on the position information of the style graphic, determine the corresponding first code block for each style graphic; and replace the first code block corresponding to the style graphic in the initial code with the second code block of the functional component corresponding to the style graphic to obtain the target code; the second code block is pre-written and is used to implement the interaction function and component appearance of the functional component.

[0009] Generate a target interface including multiple functional components based on the target code.

[0010] An interface generation device provided by an embodiment of the present application includes:

[0011] A presentation unit for presenting an initial interface based on initial code; the initial interface includes multiple style graphics, and each style graphic is drawn based on the appearance of a different functional component, and each style graphic corresponds to a first code block in the initial code.

[0012] A first determination unit for, for each style graphic in the initial interface, input the style graphic into a pre-trained component recognition model to determine the functional component corresponding to the style graphic and determine the position information of the style graphic in the initial interface.

[0013] A second determination unit for, for each style graphic in the initial interface, based on the position information of the style graphic, determine the corresponding first code block for each style graphic; and replace the first code block corresponding to the style graphic in the initial code with the second code block of the functional component corresponding to the style graphic to obtain the target code; the second code block is pre-written and is used to implement the interaction function and component appearance of the functional component.

[0014] A generation unit for generating a target interface including multiple functional components based on the target code.

[0015] Optionally, the first determination unit is specifically configured to:

[0016] Frame the style graphic based on a recognition frame;

[0017] Determine the first coordinate information of the edge points of the recognition frame in the initial interface, and obtain the second coordinate information of the center point of the recognition frame based on the first coordinate information;

[0018] Use the first coordinate information and the second coordinate information as the position information of the style graphic in the initial interface.

[0019] Optionally, each of the style graphics includes a plurality of style elements, and in each of the style graphics, there is a hierarchical relationship between the first sub-code blocks corresponding to the respective style elements; the first sub-code block corresponding to a superior style element includes the first sub-code block corresponding to an inferior style element.

[0020] Optionally, the second determination unit is specifically configured to:

[0021] Based on the respective first coordinate information, obtain the graphic size information of the style graphic;

[0022] Based on the preset probe and the second coordinate information, in the hierarchical order from bottom to top, sequentially obtain the first sub-code blocks corresponding to the respective style elements included in the style graphic, and the element size information of the respective style elements;

[0023] Based on the hierarchical order, sequentially compare the differences between the element size information of the respective style elements and the graphic size information of the style graphic, and determine the target style element based on the respective differences;

[0024] Use the first sub-code block corresponding to the target style element as the first code block corresponding to the style graphic.

[0025] Optionally, the second determination unit is specifically configured to:

[0026] Use the style element currently detected by the probe as the candidate style element, and compare the candidate element size information of the candidate style element with the graphic size information of the style graphic;

[0027] If the difference between the candidate element size information and the graphic size information is not greater than a preset threshold, use the candidate style element as the target style element;

[0028] If the difference between the candidate element size information and the graphic size information is greater than the preset threshold, use the next style element as the candidate style element detected by the probe.

[0029] Optionally, the device further includes:

[0030] A display unit, configured to, after determining the position information of the style graphic in the initial interface and before determining the first code block corresponding to the style graphic based on the position information of the style graphic, generate a prompt floating layer corresponding to the component type at the corresponding position of the style graphic based on the position information of the style graphic and the component type of the functional component corresponding to the style graphic, so as to highlight the style graphic.

[0031] An electronic device provided by an embodiment of the present application includes a processor and a memory. Among them, the memory stores a computer program. When the computer program is executed by the processor, the processor is caused to execute the steps of any one of the above interface generation methods.

[0032] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program runs on an electronic device, the computer program is used to cause the electronic device to execute the steps of any one of the above interface generation methods.

[0033] An embodiment of the present application provides a computer program product. The computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of any one of the above interface generation methods.

[0034] The beneficial effects of the present application are as follows:

[0035] An embodiment of the present application provides an interface generation method, device, electronic device, and storage medium. The present application can identify style graphics in an initial interface that only have appearance but no functions, so as to determine the functional components corresponding to each style graphic and the position information of the style graphic in the initial interface; then, based on the position information of the style graphic, find the first code block corresponding to each style graphic, and based on the second code block of the functional component corresponding to the style graphic, replace the first code block corresponding to the style graphic in the initial code to obtain a target code; finally, generate a target interface including multiple functional components based on the target code; since the present application can automatically identify the appearance of style components and realize automatic replacement of components based on the component positions, the whole process saves time and effort, solves the low-efficiency problem caused by manual component identification in related technologies, and can improve the accuracy of component replacement.

[0036] Other features and advantages of the present application will be described in the subsequent specification, and, in part, will become apparent from the specification, or will be understood by implementing the present application. The objectives and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written specification, claims, and drawings. Description of the Drawings

[0037] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0038] Figure 1Schematic diagram of an application scenario of an interface generation method provided by an embodiment of the present application;

[0039] Figure 2 Overall flowchart of an interface generation method provided by an embodiment of the present application;

[0040] Figure 3 Initial interface diagram provided by an embodiment of the present application;

[0041] Figure 4 Schematic diagram of an identification box provided by an embodiment of the present application;

[0042] Figure 5 Schematic diagram of a DOM tree provided by an embodiment of the present application;

[0043] Figure 6 Overall flowchart of another interface generation method provided by an embodiment of the present application;

[0044] Figure 7 Schematic diagram of the composition structure of an interface generation device provided by an embodiment of the present application;

[0045] Figure 8 Schematic diagram of a hardware composition structure of an electronic device provided by an embodiment of the present application;

[0046] Figure 9 Schematic diagram of a hardware composition structure of another electronic device provided by applying an embodiment of the present application. Detailed implementation manners

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the technical solutions of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments recorded in this application document without creative efforts shall fall within the scope of the technical solutions protected by the present application.

[0048] Some concepts involved in the embodiments of the present application are introduced below.

[0049] Functional component: Also known as a user interface (UI) component, it is the basic element when constructing a user interface, encapsulating specific functions and interaction behaviors so that developers can efficiently construct complex user interfaces. Functional components are usually reusable and configurable.

[0050] DOM: Document Object Model. It is a cross - platform and language - neutral interface that allows programs and scripts to dynamically access and update the content, structure, and style of a document.

[0051] The preferred embodiments of the present application will be described below with reference to the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present application, and are not used to limit the present application. And without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0052] As Figure 1 shown, it is a schematic diagram of the application scenario of the embodiment of the present application. The application scenario diagram includes two terminal devices 110 and a server 120.

[0053] In the embodiment of the present application, the terminal device 110 includes, but is not limited to, devices such as mobile phones, tablet computers, laptop computers, desktop computers, e - book readers, intelligent voice interaction devices, smart home appliances, in - vehicle terminals, etc.; a client related to interface generation can be installed on the terminal device, and the client can be software (such as a browser, interface generation software, etc.), or a web page, a small program, etc. The server 120 is the background server corresponding to the software, web page, small program, etc., or a server dedicated to interface generation. The present application does not make specific limitations. The server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0054] It should be noted that the interface generation method in each embodiment of the present application can be executed by an electronic device, which can be a terminal device 110 or a server 120. That is, this method can be executed by the terminal device 110 or the server 120 alone, or can be executed jointly by the terminal device 110 and the server 120. For example, when executed by the terminal device 110, the terminal device 110 presents an initial interface based on an initial code; then, for each style graphic, the terminal device 110 determines the functional component corresponding to the style graphic and the position information of the style graphic in the initial interface based on a pre-trained component recognition model; the terminal device 110 determines the first code block corresponding to each style graphic based on the position information of the style graphic, and replaces the first code block with the second code block of the functional component corresponding to the style graphic to obtain a complete target code including the functional component. Finally, the terminal device 110 generates a target interface including multiple functional components based on the target code.

[0055] In an alternative embodiment, the terminal device 110 and the server 120 can communicate through a communication network.

[0056] In an alternative embodiment, the communication network is a wired network or a wireless network.

[0057] It should be noted that Figure 1 The above is only an example. In fact, the number of terminal devices and servers is not limited and is not specifically defined in the embodiments of the present application.

[0058] In addition, the embodiments of the present application can be applied to various scenarios, including but not limited to scenarios such as cloud technology, artificial intelligence, and intelligent transportation.

[0059] Next, in combination with the above-described application scenarios, the interface generation method provided by the exemplary embodiment of the present application will be described with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown for the convenience of understanding the spirit and principle of the present application, and the embodiments of the present application are not limited in this regard.

[0060] Refer to Figure 2 As shown, it is a flowchart of the implementation of an interface generation method provided by an embodiment of the present application. The specific implementation process of this method is as follows S201 - S204:

[0061] S201: Present an initial interface based on an initial code.

[0062] S202: For each style graphic in the initial interface, input the style graphic into a pre-trained component recognition model to determine the functional component corresponding to the style graphic and the position information of the style graphic in the initial interface.

[0063] Among them, the initial interface contains multiple style graphics, each of which is drawn based on the appearance of different functional components, and each style graphic corresponds to a first code block in the initial code. Functional components, which can also be referred to as UI components, are the basic elements for building user interfaces. Specific functions and interaction behaviors are encapsulated in UI components so that developers can efficiently build complex user interfaces, which can specifically include selector components, text input box components, drop-down menu components, progress bar components, and so on.

[0064] In the front-end development process, the interface design process usually starts with a UI designer drawing an interface design diagram, and then through a third-party diagram transcoding tool, the interface design diagram is converted into corresponding initial code, that is, HyperText Markup Language (html) code.

[0065] The specific implementation of this process can be to create a file upload component in the browser <input type＝"file”> , which can receive a picture in png or jpg format, such as the above-mentioned interface design diagram. Then, call the interface of the third-party diagram transcoding tool to convert the interface design diagram into an html file, that is, a file containing the above-mentioned initial code. According to this file, an initial interface can be generated. There are multiple style graphics in the initial interface, and each style graphic corresponds to a corresponding first code block in the initial code.

[0066] As Figure 3 shown, it is an initial interface diagram provided by an embodiment of the present application. The figure contains five style graphics: (a), (b1), (b2), (c), and (d).

[0067] However, the style graphics obtained based on html only have the same appearance as the functional components, but do not have the corresponding functions of the functional components, that is, the style graphics cannot interact or be operated. For example Figure 3 (a) has the same appearance as the functional component "drop-down menu", but it does not have the function of displaying a set of options, and users cannot select one or more options from it; (b1) and (b2) have the same appearance as the functional component "radio button", but users cannot click to select; (c) has the same appearance as the functional component "text box", but users cannot enter information in it; (d) has the same appearance as the functional component "confirm button", but users cannot click.

[0068] In traditional methods, only by manually identifying which functional components these style graphics correspond to and then replacing them one by one, the process is repetitive and the workload is large, increasing the workload of developers. Moreover, the process of manual replacement is extremely prone to errors and reduces work efficiency.

[0069] Therefore, this application proposes to call a pre-trained component recognition model to identify the appearance of the style graphics, so as to determine the corresponding functional components of the style graphics and the position information of the style graphics in the initial interface.

[0070] The component recognition model can be trained based on training samples. The training samples can be pictures of style graphics, attached with sample labels. The sample labels represent which functional components the corresponding style graphics correspond to. Then, the training samples are input into the component recognition model, and the component recognition model is adjusted according to the difference between the output result of the component recognition model and the sample label. Repeat the above process until the difference between the output result of the component recognition model and the sample label meets the preset conditions, or the number of training times reaches the preset threshold, and a trained component recognition model is obtained, that is, the "pre-trained component recognition model" in S202.

[0071] The component recognition model can be deployed in a terminal device or a server.

[0072] When the component recognition model recognizes the style graphics in the initial interface, it will frame the style graphics based on the recognition frame; then determine the first coordinate information of the edge points of the recognition frame in the initial interface, and obtain the second coordinate information of the center point of the recognition frame based on each first coordinate information; use each first coordinate information and the second coordinate information as the position information of the corresponding style graphics in the initial interface.

[0073] Exemplarily, the component recognition model will identify the style graphics it detects through a recognition frame (also called a bounding box). As Figure 4 shown, it is a schematic diagram of a recognition frame provided by an embodiment of this application. The several thick black rectangular frames in the figure are the recognition frames, which can mark the positions of the style graphics. It can also further display the recognition result of the component recognition model (that is, which functional component it corresponds to), and the confidence level.

[0074] The recognition frame can provide the position of the style graphics in the initial interface. For example, the position of the style graphics is located through the point coordinates of the four corners of the recognition frame; in addition, the position of the style graphics can also be determined through the center coordinates (x, y) of the recognition frame and the width and height (w, h).

[0075] Specifically, after the component recognition model recognizes, it can obtain which functional component the style graphics corresponds to, and can also obtain the coordinate array of the upper left corner point, lower left corner point, upper right corner point, and lower right corner point (that is, the first coordinate information) of the recognition frame, and calculate the width targetWidth of the recognition frame and put it into the array typeList, as follows:

[0076]

[0077] Among the above, bbox is an array storing the upper-left point, lower-left point, upper-right point, and lower-right point. label is the type of functional component corresponding to the recognized style graphic, and score is the score, which can be regarded as the probability of the style graphic corresponding to this functional component. Through the methods const centerX = (xmin + xmax) / 2 and const centerY = (ymin + ymax) / 2, the center coordinates of the recognition frame (i.e., the second coordinate information) can be calculated, and the center coordinates centerPoint: [centerX, centerY] are stored in the array typeList.

[0078] Finally, the coordinates of the upper-left point, lower-left point, upper-right point, and lower-right point of the recognition frame in the above, as well as the center coordinates, are used as the position information of the corresponding style graphic in the initial interface.

[0079] After determining the positions of various style graphics, based on the position information of the style graphics and the component type of the functional component corresponding to the style graphic, a hint floating layer corresponding to the component type can be generated at the corresponding position of the style graphic to highlight the style graphic.

[0080] That is, according to the upper-left point, lower-left point, upper-right point, and lower-right point of the recognition frame corresponding to the style graphic, and the component type of the functional component corresponding to the style graphic, a semi-transparent floating layer is created using div. Different colors can be used to represent the component types of different functional components, and it is supported to prompt the specific component type when the cursor moves to the floating layer.

[0081] Specifically, the initial interface is displayed in the browser, the array typeList is traversed, a rectangular block is created according to the bbox point information of each item. This rectangular block uses position:absolute absolute positioning and is filled with different colors according to different labels.

[0082] The browser can also include a floating button "Batch Replace". After clicking this button, the style graphics in the initial interface start to be replaced with the corresponding functional components.

[0083] S203: For each style graphic in the initial interface, based on the position information of the style graphic, determine the respective first code blocks corresponding to the style graphics; and based on the second code blocks of the functional components corresponding to the style graphics, replace the first code blocks corresponding to the style graphics in the initial code to obtain the target code.

[0084] S204: Generate a target interface containing multiple functional components based on the target code.

[0085] Among the above, the second code block is pre-written and is used to implement the interaction function and component appearance of the functional component.

[0086] In this application, the component type of the functional component can be a default functional component, such as Element ui, AntDesign, Vant, etc., or a custom functional component.

[0087] In this application, each style graphic contains multiple style elements. That is to say, each style graphic is generally composed of multiple style elements. In each style graphic, there is a hierarchical relationship between the first sub-code blocks corresponding to the respective style elements; the first sub-code block corresponding to the superior style element contains the first sub-code block corresponding to the inferior style element.

[0088] If you want to replace a style graphic, you first need to determine the first code block corresponding to the style graphic.

[0089] Exemplarily, the terminal device obtains the graphic size information of the style graphic based on each first coordinate information; based on the preset probe and the second coordinate information, in the hierarchical order from bottom to top, sequentially obtains the first sub-code blocks corresponding to the respective style elements included in the style graphic, and the element size information of the respective style elements; then, based on the hierarchical order, sequentially compares the difference between the element size information of the respective style elements and the graphic size information of the style graphic, and determines the target style element based on each difference; finally, takes the first sub-code block corresponding to the target style element as the first code block corresponding to the style graphic.

[0090] For example, taking a style graphic as an example, use the document.elementFromPoint(x,y) method to create a probe and substitute the center point coordinates of the style graphic into (x,y). The probe can obtain the topmost DOM element at the specified point x,y.

[0091] Suppose the first code block corresponding to a style graphic (corresponding to a drop-down menu) is as follows:

[0092]

[0093] It contains three style elements (also called html elements), namely div, span, and img. Among them, div is the superior style element, and span and img are the inferior style elements of div, or rather, span and img are the child elements of div. span and img are at the same level and are juxtaposed in div. However, when the above style elements are presented in the initial interface, the lowermost style elements are presented on the topmost layer, that is, span and img will be presented on the upper layer of div.

[0094] group_12, flex-row, and justify-between define the specific styles of the div, such as color, margin, arrangement, etc.; class="thumbnail_23", referrerpolicy="no-referrer", and src=". / img / SketchPng4fd45dacf2115681feb12c149e46204c1159ee7ef149b4b5c504763cf0be7332.png" are the attributes and path of the img (image).

[0095] The probe detects elements in the DOM tree. Convert the first code block above into a DOM tree, as Figure 5 shown, which is a schematic diagram of a DOM tree provided by an embodiment of the present application. div, span, and img are called DOM elements in the DOM tree. Among them, div is the root element and also the parent element of span and img, and span and img are the child elements of div.

[0096] It should be noted that span, img, and div are both style elements and DOM elements. Style elements and DOM elements are different names in different environments.

[0097] Since the probe detects the top-level DOM element (or style element), the probe may finally return a span element or an img element (if the span element and the img element do not exactly cover the center point of the style graph, the probe returns div). If a replacement is performed at this time, it may occur that the first sub-code block corresponding to the span element or the img element is replaced, rather than the first code block corresponding to the entire style graph being replaced.

[0098] Therefore, the present application proposes to traverse all style elements in the hierarchical order from bottom to top (that is, from the lower-level style elements to the upper-level style elements).

[0099] Exemplarily, take the style element currently detected by the probe as the candidate style element, and compare the candidate element size information of the candidate style element with the graph size information of the style graph; if the difference between the candidate element size information and the graph size information is not greater than the preset threshold, then take the candidate style element as the target style element; if the difference between the candidate element size information and the graph size information is greater than the preset threshold, then take the next style element (that is, the upper-level style element of the candidate element) as the candidate style element detected by the probe.

[0100] Specifically, still taking span, img, and div in the above as examples, the DOM element obtained by the probe from the specified center point is denoted as currentDom. The width of this DOM element, currentWidth, is obtained through currentDom.offsetWidth. Compare currentWidth with the width of the previously recognized style graphic (i.e., the width of the recognition box). If currentWidth is less than the width of the style graphic, then currentDom may be just the DOM element corresponding to a subordinate style element of the style graphic, such as span or img mentioned above. However, since in this application, the width of the recognition box is used as the width of the style graphic, and the recognition box is often larger than the style graphic, so currentDom may also be div. Therefore, a preset threshold can be set to determine the target style element based on whether the difference between currentWidth and the width of the style graphic is greater than the preset threshold.

[0101] In addition, height, or a combination of width and height can also be used for comparison, and this application does not make specific limitations.

[0102] In addition to the preset threshold, the target style element can also be determined in another way. Specifically, after determining that currentWidth is less than the width of the style graphic, the parent element of currentDom, parentDom, is obtained through currentDom.parentElement. Then compare the width of parentDom with the width of the style graphic. If it is still less, then obtain the parent element of parentDom again and conduct a width comparison. If the width of the parent element of parentDom is greater than the width of the style graphic, then determine parentDom as the target style element; otherwise, continue to obtain the parent element's parent element of parentDom upwards.

[0103] After determining the target style element, the first sub-code block corresponding to the target style element (such as the first sub-code block corresponding to div) is the first code block corresponding to the style graphic, which contains the first sub-code blocks corresponding to all sub-elements (such as the first sub-code blocks corresponding to span and img).

[0104] Finally, according to the functional component corresponding to the style graphic, use the replaceWidth method to replace the first code block with the second code block of the functional component in this step.

[0105] After all style graphics are replaced, the initial code is transformed into the target code, and a target interface containing multiple functional components can be generated based on the target code.

[0106] Based on the above process, as Figure 6As shown in the figure, it is the overall flowchart of another interface generation method provided by the embodiments of the present application. The server specifically performs the following steps:

[0107] Step 601: The terminal device converts the interface design diagram into corresponding initial code.

[0108] Step 602: The terminal device presents the initial interface based on the initial code and determines the functional components corresponding to the style graphics.

[0109] The functional components corresponding to the style graphics are essentially a recognition process, which can be completed based on a pre-trained component recognition model.

[0110] Step 603: The terminal device determines the position information of the style graphics in the initial interface based on the recognition frame.

[0111] The first coordinate information of the edge points of the recognition frame in the initial interface and the second coordinate information of the center point of the recognition frame are used as the position information of the corresponding style graphics in the initial interface.

[0112] Step 604: The terminal device generates a semi-transparent floating layer corresponding to the component type at the corresponding position of the style graphics based on the position information of the style graphics and the functional components corresponding to the style graphics.

[0113] Step 605: The terminal device starts to execute the replacement process in response to the operation triggered by the developer for the "batch replacement" floating button.

[0114] Step 606: The terminal device creates a probe.

[0115] Step 607: The terminal device determines the first code block corresponding to each style graphic based on the position information of the style graphics.

[0116] Since the probe detects the topmost DOM element, the method of comparing the element size with the style graphic size is adopted, and the target style element is determined by comparing from the lower-level style elements one by one upward. The first sub-code block corresponding to the target style element is the first code block corresponding to the style graphic.

[0117] Step 608: The terminal device replaces each first code block with the second code block of the functional component corresponding to the corresponding style graphic to obtain the target code.

[0118] Step 609: The terminal device generates the target interface based on the target code.

[0119] Based on the same inventive concept, the embodiments of the present application also provide an interface generation device. As Figure 7 shown, it is the structural schematic diagram of the interface generation device, which may include:

[0120] A presentation unit 701 for presenting an initial interface based on an initial code; the initial interface includes multiple style graphics, each style graphic is drawn based on the appearance of a different functional component, and each style graphic corresponds to a first code block in the initial code;

[0121] A first determination unit 702 for, for each style graphic in the initial interface, inputting the style graphic into a pre-trained component recognition model to determine the functional component corresponding to the style graphic and the position information of the style graphic in the initial interface;

[0122] A second determination unit 703 for, for each style graphic in the initial interface, determining the first code block corresponding to each style graphic based on the position information of the style graphic; and replacing the first code block corresponding to the style graphic in the initial code with a second code block corresponding to the functional component of the style graphic to obtain a target code; the second code block is pre-written and is used to implement the interaction function and component appearance of the functional component;

[0123] A generation unit 704 for generating a target interface including multiple functional components based on the target code.

[0124] Optionally, the first determination unit 702 is specifically configured to:

[0125] Frame the style graphic based on a recognition frame;

[0126] Determine the first coordinate information of the edge points of the recognition frame in the initial interface, and obtain the second coordinate information of the center point of the recognition frame based on each first coordinate information;

[0127] Use each first coordinate information and the second coordinate information as the position information of the style graphic in the initial interface.

[0128] Optionally, each style graphic includes multiple style elements, and in each style graphic, there is a hierarchical relationship between the first sub-code blocks corresponding to each style element; the first sub-code block corresponding to the upper-level style element includes the first sub-code block corresponding to the lower-level style element.

[0129] Optionally, the second determination unit 703 is specifically configured to:

[0130] Obtain the graphic size information of the style graphic based on each first coordinate information;

[0131] Based on a preset probe and the second coordinate information, in the hierarchical order from bottom to top, sequentially obtain the first sub-code blocks corresponding to each style element included in the style graphic and the element size information of each style element;

[0132] Based on the hierarchical order, the differences between the element size information of each style element and the graphic size information of the style graphic are compared in sequence, and the target style element is determined based on each difference;

[0133] Use the first sub-code block corresponding to the target style element as the first code block corresponding to the style graphic.

[0134] Optionally, the second determination unit 703 is specifically configured to:

[0135] Use the style element currently detected by the probe as the candidate style element, and compare the candidate element size information of the candidate style element with the graphic size information of the style graphic;

[0136] If the difference between the candidate element size information and the graphic size information is not greater than the preset threshold, use the candidate style element as the target style element;

[0137] If the difference between the candidate element size information and the graphic size information is greater than the preset threshold, use the next style element as the candidate style element detected by the probe.

[0138] Optionally, the device further includes:

[0139] A display unit 705, configured to, after determining the position information of the style graphic in the initial interface and before determining the first code block corresponding to each style graphic based on the position information of the style graphic, generate a prompt floating layer corresponding to the component type at the corresponding position of the style graphic based on the position information of the style graphic and the component type of the functional component corresponding to the style graphic, so as to highlight the display of the style graphic.

[0140] After introducing the interface generation method and device of the exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0141] Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method, or a program product. Therefore, various aspects of the present application can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "system" here.

[0142] Based on the same inventive concept as the above method embodiment, an electronic device is further provided in the embodiment of the present application. In one embodiment, the electronic device can be a server, such as Figure 1 the server 120 shown. In this embodiment, the structure of the electronic device can be as Figure 8 shown, including a memory 801, a communication module 803, and one or more processors 802.

[0143] A memory 801 for storing computer programs executed by a processor 802. The memory 801 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and programs required for running an instant messaging function, etc.; the data storage area can store various instant messaging information and operation instruction sets, etc.

[0144] The memory 801 can be a volatile memory, such as a random-access memory (RAM); the memory 801 can also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 801 is any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 801 can be a combination of the above memories.

[0145] The processor 802 can include one or more central processing units (CPUs) or be a digital processing unit, etc. The processor 802 is used to implement the above interface generation method when calling the computer program stored in the memory 801.

[0146] A communication module 803 is used to communicate with a terminal device and other servers.

[0147] In the embodiments of the present application, the specific connection medium between the above-mentioned memory 801, communication module 803 and processor 802 is not limited. In the embodiments of the present application Figure 8 it is described that the memory 801 and the processor 802 are connected through a bus 804, and the bus 804 is described in thick lines in Figure 8 The connection manners between other components are only for illustrative purposes and are not to be taken as limitations. The bus 804 can be divided into an address bus, a data bus, a control bus, etc. For the sake of description, Figure 8 only one thick line is used to describe it in

[0148] The memory 801 stores a computer storage medium, and the computer storage medium stores computer-executable instructions for implementing the interface generation method of the embodiments of the present application. The processor 802 is used to execute the above interface generation method, as Figure 2 shown.

[0149] In another embodiment, the electronic device may also be other electronic devices, such as Figure 1 the terminal device 110 shown. In this embodiment, the structure of the electronic device may be as Figure 9 shown, including components such as a communication component 910, a memory 920, a display unit 930, a camera 940, a sensor 950, an audio circuit 960, a Bluetooth module 970, a processor 980, etc.

[0150] The communication component 910 is used to communicate with the server. In some embodiments, it may include a Wireless Fidelity (WiFi) module. The WiFi module belongs to short-range wireless transmission technology, and the electronic device can help users send and receive information through the WiFi module.

[0151] The memory 920 can be used to store software programs and data. The processor 980 executes various functions and data processing of the terminal device 110 by running the software programs or data stored in the memory 920. The memory 920 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. The memory 920 stores an operating system that enables the terminal device 110 to run. In this application, the memory 920 can store the operating system and various application programs, and can also store the computer program for implementing the interface generation method of the embodiments of this application.

[0152] The display unit 930 can also be used to display information input by the user or information provided to the user, as well as the graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 930 may include a display screen 932 disposed on the front of the terminal device 110. Among them, the display screen 932 can be configured in the form of a liquid crystal display, a light-emitting diode, etc.

[0153] The display unit 930 can also be used to receive input digital or character information, and generate signal inputs related to the user settings and function controls of the terminal device 110. Specifically, the display unit 930 may include a touch screen 931 disposed on the front of the terminal device 110, which can collect touch operations of the user on or near it, such as clicking buttons, dragging scroll boxes, etc.

[0154] Among them, the touch screen 931 can cover the display screen 932, or the touch screen 931 and the display screen 932 can be integrated to implement the input and output functions of the terminal device 110. After integration, it can be simply referred to as a touch display screen. In this application, the display unit 930 can display application programs and corresponding operation steps.

[0155] The camera 940 can be used to capture static images, and users can publish the images captured by the camera 940 through an application. There can be one or more cameras 940. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the processor 980 to be converted into a digital image signal.

[0156] The terminal device may further include at least one sensor 950, such as an acceleration sensor 951, a distance sensor 952, a fingerprint sensor 953, and a temperature sensor 954. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0157] The audio circuit 960, the speaker 961, and the microphone 962 can provide an audio interface between the user and the terminal device 110. The audio circuit 960 can transmit the electrical signal converted from the received audio data to the speaker 961, and the speaker 961 converts it into a sound signal for output. The terminal device 110 may also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 962 converts the collected sound signal into an electrical signal, which is received by the audio circuit 960, converted into audio data, and then the audio data is output to the communication component 910 to be sent to, for example, another terminal device 110, or the audio data is output to the memory 920 for further processing.

[0158] The Bluetooth module 970 is used to interact with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module to perform data interaction.

[0159] The processor 980 is the control center of the terminal device, connecting various parts of the entire terminal through various interfaces and lines. By running or executing software programs stored in the memory 920 and calling data stored in the memory 920, it performs various functions of the terminal device and processes data. In some embodiments, the processor 980 may include one or more processing units; the processor 980 may also integrate an application processor and a baseband processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the baseband processor mainly processes wireless communication. It can be understood that the above baseband processor may not be integrated into the processor 980. In this application, the processor 980 can run the operating system, application programs, user interface display and touch response, as well as the interface generation method of the embodiments of this application. In addition, the processor 980 is coupled to the display unit 930.

[0160] In some possible implementation manners, various aspects of the interface generation method provided in this application can also be implemented in the form of a program product, which includes a computer program. When the program product runs on an electronic device, the computer program is used to cause the electronic device to execute the steps in the interface generation method according to various exemplary embodiments of this application described above in this specification. For example, the electronic device can execute the steps as shown in Figure 2 the figure.

[0161] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0162] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a system, or a computer program product. Therefore, this application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0163] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to produce a machine, so that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0164] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including instruction means, and the instruction means implements the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0166] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. A method for generating an interface, characterized in that: The method comprises: Presenting an initial interface based on the initial code; the initial interface includes a plurality of style graphics, each style graphic is drawn based on the appearance of a different functional component, and each style graphic corresponds to a first code block in the initial code; For each style graphic in the initial interface, input the style graphic into a pre-trained component recognition model, determine the functional component corresponding to the style graphic, and determine the position information of the style graphic in the initial interface; For each style graphic in the initial interface, based on the position information of the style graphic, determine the first code block corresponding to each style graphic; and based on the second code block of the functional component corresponding to the style graphic, replace the first code block corresponding to the style graphic in the initial code to obtain the target code; the second code block is pre-written, and the second code block is used to realize the interactive function and component appearance of the functional component; A target interface including multiple functional components is generated based on the target code.

2. The method according to claim 1, characterized in that The determining the position information of the style graphic in the initial interface includes: Selecting the style graphic based on the identification frame; Determine first coordinate information of edge points of the identification frame in the initial interface, and acquire second coordinate information of a center point of the identification frame based on each first coordinate information; The first coordinate information and the second coordinate information are used as the position information of the style graphic in the initial interface.

3. The method according to claim 2, characterized in that Each of the style graphics includes multiple style elements. In each of the style graphics, there is a hierarchical relationship between the first sub-code blocks corresponding to the respective style elements; the first sub-code block corresponding to the upper-level style element includes the first sub-code block corresponding to the lower-level style element.

4. The method according to claim 3, characterized in that The determining, based on the position information of the style graphics, the first code blocks corresponding to the style graphics respectively includes: Based on the first coordinate information, obtaining the graphic size information of the style graphic; Based on the preset probe and the second coordinate information, first sub-code blocks corresponding to the respective style elements included in the style graphic and element size information of the respective style elements are sequentially acquired in a hierarchical order from bottom to top; Based on the hierarchical order, sequentially comparing the differences between the element size information of each of the style elements and the graphic size information of the style graphic, and determining a target style element based on the differences; The first sub-code block corresponding to the target style element is used as the first code block corresponding to the style graphic.

5. The method according to claim 4, characterized in that The step of sequentially comparing the element size information of each of the style elements with the graphic size information of the style graphic based on the hierarchical order, and determining the target style element based on the differences, includes: taking the style element currently detected by the probe as a candidate style element, and comparing the candidate element size information of the candidate style element with the graphic size information of the style graphic; If the difference between the candidate element size information and the graphic size information is not greater than a preset threshold, taking the candidate style element as the target style element; If the difference between the candidate element size information and the graphic size information is greater than a preset threshold, the next pattern element is taken as the candidate pattern element detected by the probe.

6. The method according to any one of claims 1 to 5, characterized in that: After determining the position information of the style graphics in the initial interface, and before determining the first code blocks corresponding to the style graphics based on the position information of the style graphics, the method further includes: Based on the position information of the style graphic and the component type of the functional component corresponding to the style graphic, a prompt floating layer corresponding to the component type is generated at the corresponding position of the style graphic to highlight the style graphic.

7. An interface generation device, characterized in that: include: A presentation unit, configured to present an initial interface based on the initial code, wherein the initial interface includes a plurality of style graphics; each style graphic is drawn based on the appearance of a different functional component, and each style graphic corresponds to a first code block in the initial code; A first determining unit is used to input each style graphic in the initial interface into a pre-trained component recognition model, determine the functional component corresponding to the style graphic, and determine the position information of the style graphic in the initial interface; A second determining unit, configured to determine, for each style graphic in the initial interface, a first code block corresponding to each style graphic based on position information of the style graphic; and based on the second code block of the functional component corresponding to the style graphic, replacing the first code block corresponding to the style graphic in the initial code to obtain the target code; The second code block is pre-written, and the second code block is used to implement the interactive function and component appearance of the functional component; A generating unit is used to generate a target interface including a plurality of functional components based on the target code.

8. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the methods of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: It includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any method described in claims 1 to 6.

10. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of any one of the methods described in claims 1 to 6.

Images