Front-end page building method and related product

Abstract

The embodiment of the invention discloses a front-end page building method and a related product. The method comprises the steps of extracting multi-modal features of a target design draft through an identification model, and identifying component attributes of components in the target design draft and a hierarchical relationship between the components based on the multi-modal features; converting the target design draft into a first code of a first programming language based on the component attributes of the components and the hierarchical relationship between the components; based on the mapping relation between the first programming language and a second programming language supported by a low-code platform, the first code is converted into a second code of the second programming language, and the second code is used for generating a page corresponding to the target design draft in the low-code platform. Therefore, end-to-end automatic conversion from the design draft to the codes supported by the low-code platform can be realized, so that the low-code platform can reuse the design draft, and the page development efficiency is improved.

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a front-end page building method and related products. Background Art

[0002] In the process of front-end page development, developers are usually required to write code and build pages according to the design draft to obtain the front-end page. However, this method relies on manual operation, which is cumbersome, and the design draft cannot be reused. Once the page development requirements change, the design draft needs to be re-created and code written, page built, etc., resulting in low development efficiency. Summary of the invention

[0003] The purpose of the embodiments of this specification is to provide a front-end page building method and related products to achieve end-to-end automated conversion from design drafts to codes suitable for low-code platforms, so that low-code platforms can reuse design drafts and thus improve page development efficiency.

[0004] In order to achieve the above purpose, the embodiments of this specification adopt the following technical solutions: In a first aspect, a front-end page construction method is provided, comprising: Extracting multimodal features of the target design draft through the recognition model, and identifying component attributes of components in the target design draft and hierarchical relationships between components based on the multimodal features; Based on the component properties of the components and the hierarchical relationship between the components, converting the target design draft into a first code in a first programming language; Based on the mapping relationship between the first programming language and the second programming language supported by the low-code platform, the first code is converted into a second code of the second programming language, and the second code is used to generate a page corresponding to the target design draft in the low-code platform.

[0005] In a second aspect, a front-end page building device is provided, comprising: An identification module, used to extract multimodal features of a target design draft through a recognition model, and identify component attributes of components in the target design draft and hierarchical relationships between components based on the multimodal features; A first conversion module, configured to convert the target design draft into a first code in a first programming language based on the component attributes of the components and the hierarchical relationship between the components; The second conversion module is used to convert the first code into a second code of the second programming language based on a mapping relationship between the first programming language and a second programming language supported by the low-code platform, and the second code is used to generate a page corresponding to the target design draft in the low-code platform.

[0006] According to a third aspect, an electronic device is provided, including: processor; a memory for storing instructions executable by the processor; Among them, the processor is configured to execute the instructions to implement the front-end page building method provided in the first aspect.

[0007] In a fourth aspect, a computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the front-end page building method provided in the first aspect.

[0008] In a fifth aspect, a computer program product is provided, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to execute part or all of the steps in the front-end page building method provided in the first aspect.

[0009] The scheme of the embodiments of this specification takes into account the differences in features of different components in the design draft, extracts multimodal features of the target design draft through an identification model, and can accurately identify component properties of components and hierarchical relationships between components in the target design draft based on the multimodal features; then, based on the component properties of components and hierarchical relationships between components in the target design draft, the target design draft is converted into a first code in a first programming language, the first code is used to describe the component properties of components and the hierarchical relationships between components, and has the advantage of being maintainable, and can be used to be modified and reused to adapt to changes in page development requirements, which is beneficial to improving page development efficiency; further, according to the mapping relationship between the first programming language and the second programming language supported by the low-code platform, the first code is converted into a second code in the second programming language, thereby realizing end-to-end automated conversion of codes from the design draft to the low-code platform, and the code can be used to generate a page corresponding to the target design draft in the low-code platform, so that the low-code platform can reuse the target design draft and automatically convert it into draggable components and configurations to adapt to changes in page development requirements, thereby improving page development efficiency. In addition, since the programming languages ​​supported by low-code platforms are usually closed source, the effect of directly converting the target design draft into the code of the programming language is not good. By first converting the target design draft into the first code of the open source first programming language, and then converting from the first code to the second code of the second programming language, the conversion effect is better and the conversion accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings: Figure 1 A flowchart of a front-end page building method provided for an embodiment of this specification; Figure 2 A schematic diagram of a disassembly result of a first-level component provided for an embodiment of the present specification; Figure 3 A schematic diagram of a disassembly result of a second-level component provided for an embodiment of the present specification; Figure 4 A flowchart of a method for generating a target design draft provided in one embodiment of this specification; Figure 5 A flowchart of a design draft generation method provided for another embodiment of this specification; Figure 6 A schematic diagram of the structure of a front-end page building device provided for one embodiment of this specification; Figure 7 A schematic diagram of the structure of an electronic device provided for one embodiment of the present specification. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solutions and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0012] The term "including" and its variations used in this document are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. The term "responsive to" is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be real-time or have a set delay. Unless otherwise specified, there is no restriction on the order of the multiple operations executed.

[0013] It should be noted that the concepts such as "first" and "second" mentioned in this document are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0014] It should be noted that modifiers such as "one", "multiple", etc. mentioned in this document are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly stated in the context, they should be understood as "one or more".

[0015] The names of the messages or information exchanged between multiple devices in the implementation manner of this document are for illustrative purposes only and are not used to limit the scope of these messages or information.

[0016] As mentioned above, existing design draft to code tools use CNN (Convolutional Neural Network) to parse the design layer, generate static HTML (HyperText Markup Language) or CSS (Cascading Style Sheets) code, and then generate the front-end page based on the HTML or CSS code. However, the code output by this solution lacks a componentized structure, such as being unable to generate React or Vue components. Once the front-end page needs to be modified, a new design draft needs to be designed and then the code converted, resulting in low development efficiency. In addition, this solution only focuses on the generation of static pages, without considering the integration with the back-end data, and therefore does not consider the data binding requirements of the low-code platform, which means that developers need to manually associate the API (Application Programming Interface) of the low-code platform to bind the back-end data to the generated HTML or CSS code, thereby increasing the development workload.

[0017] Although developers can currently develop front-end pages by dragging and dropping components in a low-code platform based on design drafts, this approach still has certain limitations. For example, this approach relies on manual operations, and low-code platforms usually cannot directly reuse these design drafts, resulting in low development efficiency.

[0018] To this end, the embodiments of this specification aim to provide a front-end page building method. Taking into account the differences in features of different components in a design draft, the multimodal features of a target design draft are extracted through an identification model, and based on the multimodal features, the component properties of the components in the target design draft and the hierarchical relationships between the components can be accurately identified; then, based on the component properties of the components in the target design draft and the hierarchical relationships between the components, the target design draft is converted into a first code in a first programming language, the first code is used to describe the component properties of the components and the hierarchical relationships between the components, and has the advantage of being maintainable, and can be used to be modified and reused to adapt to changes in page development requirements, which is beneficial to improving page development efficiency; further, according to the mapping relationship between the first programming language and the second programming language supported by the low-code platform, the first code is converted into a second code in the second programming language, thereby realizing end-to-end automated conversion of codes from the design draft to the low-code platform, and the code can be used to generate a page corresponding to the target design draft in the low-code platform, so that the low-code platform can reuse the target design draft and automatically convert it into draggable components and configurations to adapt to changes in page development requirements, thereby improving page development efficiency. In addition, since the programming languages ​​supported by low-code platforms are usually closed source, the effect of directly converting the target design draft into the code of the programming language is not good. By first converting the target design draft into the first code of the open source first programming language, and then converting from the first code to the second code of the second programming language, the conversion effect is better and the conversion accuracy is higher.

[0019] It should be understood that the front-end page building method provided in the embodiments of this specification can be executed by an electronic device or software installed in an electronic device, and can be specifically executed by a terminal device or a server device.

[0020] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0021] Please refer to Figure 1 , is a flow chart of a front-end page building method provided by an embodiment of this specification, and the method may include: S102, extracting multimodal features of the target design draft through the recognition model, and identifying component attributes of components in the target design draft and hierarchical relationships between components based on the multimodal features.

[0022] The target design draft refers to the design draft to be converted. The design draft can have various appropriate formats, such as but not limited to Figma design draft, Sketch design draft, etc.

[0023] The multimodal features of the target design draft may include the multimodal features of different areas of the target design draft. Multimodal features may include, for example, but are not limited to: visual features, structural features, and semantic features. Among them, visual features refer to features in visual presentation, which may specifically include element spacing, color values ​​(such as RGB values, HEX values, etc.), font size and style, line thickness, shape and pattern, light and shadow effects, and overall design style (such as flattening, three-dimensionalization), etc. Structural features are used to reflect layout and organizational forms, and may specifically include layout methods (such as top-down layout, left-right layout), element arrangement methods, etc. Semantic features are used to reflect the information content and intention conveyed by the target design draft or part of its area, and may specifically include text content, the meaning of icon symbols, the narrative of images, and the interactive information of elements.

[0024] In the embodiments of this specification, the recognition model may adopt various pre-trained artificial intelligence (AI) models, such as but not limited to multimodal models, image recognition models, deep learning networks, etc., which are not limited in the embodiments of this specification.

[0025] The target design draft may include multiple components, which may be located at different levels. For example, a larger-grained container component usually contains a composite component, and a composite component is usually composed of multiple atomic components with the smallest granularity. In other words, a composite component is a lower-level component of a container component, and multiple atomic components are lower-level components of a composite component. Each component has corresponding component properties. Component properties define the appearance, behavior, and interaction mode of a component with other components. Component properties may include, for example, but are not limited to, appearance properties (such as size, color and style, layout, etc.), behavior properties (such as interactivity, responsiveness, state management), and data properties (such as data source, data binding, etc.).

[0026] Since there are differences in at least one of the visual features, structural features, and semantic features of the areas where different components are located in the target design draft, based on the multimodal features of the target design draft and using graph computer vision recognition (Computer Vision) technology, the components in the target design draft can be accurately identified, and the component properties of the components and the hierarchical relationships between the components can be determined.

[0027] In the above S102, the component attributes of the components in the target design draft and the hierarchical relationship between the components may be identified in various appropriate ways, which are not limited in the embodiments of this specification.

[0028] In one implementation, the recognition model uses the extracted multimodal features to classify the components in the target design draft through a classification algorithm to identify the type of components (such as buttons, text boxes, pictures, etc.); then, for each component, further extract its attribute information, such as the size, position, color, font, etc. of the component through image processing and text analysis technology; further, by analyzing the spatial position relationship of the components in the target design draft, and combining the semantic information of the target design draft, further analyze the topological structure and dependency relationship between components, so as to accurately identify the hierarchical relationship between components. For complex relationships between components, models such as Graph Neural Network (GNN) can be used for modeling and analysis to capture the topological structure and dependency relationship between components, so as to more accurately identify the hierarchical relationship between components.

[0029] In another implementation, the step of identifying the component attributes of the components in the target design draft and the hierarchical relationship between the components based on the multimodal features in the S102 package includes the following steps: S121, determining the target design draft as a first-level component.

[0030] Specifically, the target design draft is determined as a container component, which can nest other components.

[0031] S122, starting from the first level component, for each level component, based on the multimodal features of the area where the component of this level is located, determine the component attributes of the component of this level.

[0032] Specifically, for each level of components, the appearance attributes of the components at this level can be determined based on the visual features and structural features of the area where the components at this level are located; the behavioral attributes and data attributes of the components at this level can be determined based on the semantic features of the area where the components at this level are located.

[0033] S123, disassemble the component at this level based on the component attributes of the component at this level to obtain the next level component of the component at this level.

[0034] Specifically, if the multimodal features of the region where the component of this level is located include multiple visual features, the region where the component of this level is located is divided into multiple sub-regions based on the structural features of the region where the component of this level is located; and each sub-region is determined as the next-level component.

[0035] For example, Figure 2 The target design shown is a design of a company details page, and the entire design is used as the first-level component. According to the visual and structural features of the design, the design contains 6 sub-areas with similar visual features, which are arranged in an up-down manner. Therefore, the design can be divided into 6 sub-areas, and each sub-area is determined as a second-level component.

[0036] Then, for each second-level component, repeat the above process to disassemble until the next-level component obtained contains only one visual feature. Figure 3 As shown, taking the first sub-region as an example, according to the visual features and structural features of the sub-region, the sub-region adopts a left-right layout, with a picture on the left and text on the right. The visual features of the two are different, so the sub-region is divided into two smaller sub-regions on the left and right, and these two smaller sub-regions are determined as third-level components.

[0037] Since the third level component on the left (i.e. Figure 3 The sub-region to which the third-level component belongs ("Picture 1" in the figure) contains only one visual feature as a whole, so the third-level component is determined to be an atomic component, that is, the component with the smallest functional granularity in the target design draft, and the third-level component is no longer disassembled. The sub-region to which the third-level component belongs on the right contains a variety of texts with different visual features, and the sub-region adopts a top-to-bottom layout, that is, the top is the bold text "A Enterprise" and "Information Technology", and the bottom is the non-bold text "Introduction to A Enterprise", so the sub-region is divided into two smaller sub-regions, and the two smaller sub-regions are determined as fourth-level components, that is, the upper fourth-level component contains the text "A Enterprise" and "Information Technology", and the lower fourth-level component contains the text "Introduction to A Enterprise".

[0038] The sub-region to which the fourth-level component at the top belongs contains two texts with different visual features, and the sub-region adopts a left-right layout, that is, the text "A Enterprise" on the left does not have a solid border, and the text "Information Technology" on the right has a solid border, and then the sub-region is divided into two smaller sub-regions on the left and right, and each of the divided sub-regions is determined as a fifth-level component, that is, the fifth-level component on the left contains the text "A Enterprise", and the fifth-level sub-component on the right contains the text "Information Technology". The sub-region to which the fourth-level component at the bottom belongs contains text with one visual feature, so the fourth-level component is determined to be an atomic component, and then the fourth-level component is no longer disassembled.

[0039] By decomposing the target design draft layer by layer in the above manner, a complex design draft can be decomposed into a series of clear and specific components, and the component properties and the hierarchical relationship between components can be accurately determined for each component, which helps to ensure that the second code obtained based on the final conversion can highly restore the target design draft.

[0040] S104, based on the component attributes of the components and the hierarchical relationship between the components, convert the target design draft into a first code in a first programming language.

[0041] The first programming language can be various open source programming languages, which can be set according to actual needs, and the embodiments of this specification do not limit this. Since the programming languages ​​supported by the low-code platform are usually closed source, the effect of directly converting the target design draft into the code of the programming language is not good. By first converting the target design draft into the first code of the open source first programming language, and then converting from the first code to the second code of the second programming language, the conversion effect is better and the conversion accuracy is higher.

[0042] In one implementation, the first programming language may be JSX (JavaScript XML). JSX is a syntax extension mainly used in the React framework, which allows HTML tags to be embedded in JavaScript code, thereby simplifying the writing of React components. Since low-code platforms generally support JavaScript, by converting the target design draft into the first code of the JSX language, not only can the first code accurately describe the components in the target design draft, but also be more intuitive, easy to understand, maintain and reuse, and the content of the target design draft can also be accurately converted into code supported by the low-code platform.

[0043] In the above S104, the target design draft may be converted into the first code of the first programming language by various appropriate methods, which is not limited in the embodiments of this specification.

[0044] In one implementation, the above S104 includes the following steps: generating a code template corresponding to the target design draft based on the hierarchical relationship between components and the grammatical specification of the first programming language; generating a first component code corresponding to the component based on the component properties of the component and the grammatical specification of the first programming language; filling the first component code corresponding to the component into the code template to obtain the first code in the first programming language.

[0045] For example, Figure 2 and Figure 3 Taking the hierarchical relationship between components shown in the figure as an example, assuming that the first programming language is JSX, then the code template corresponding to the target design draft is as follows: <container> <block1 / > <block2 / > <block3 / > <block4 / > <block5 / > <block6 / > <container> In the above code template, <container>represents the first level component, <block1 / > to <block6 / > Respectively represent a second-level component, which is used to describe the component properties of the second-level component and the subcomponents it contains.

[0046] by Figure 3 Taking the second-level component shown as an example, the field "layout" can be used to describe the layout of the second-level component. The second-level component contains two third-level components. The third-level component on the left contains an image, and the third-level component on the right contains text. For the third-level component on the left, the field "Image src" is used to describe the content of the third-level component; for the third-level component on the right, the field "layout" is used to describe the layout of the third-level component, and the field "title" is used to describe the text "A Enterprise" in the third-level component, and the field "content" is used to describe the text "Information Technology" and "Introduction to A Enterprise" in the third-level component. In this way, the component code corresponding to the second-level component and the component code corresponding to the sub-component in the second-level component are obtained; further, the component codes corresponding to these components are used to replace the fields in the above code template. <block1 / > .

[0047] The other second-level components are processed in a similar manner to obtain the first code.

[0048] In another implementation, the target design draft is converted into a first code in a first programming language through a pre-trained AI model based on the component attributes of the components in the target design draft and the hierarchical relationship between the components. The AI ​​model here may include but is not limited to LLM (Large Language Model), multimodal model, deep learning network with code conversion capability, etc., which is not limited in the embodiments of this specification.

[0049] S106, based on the mapping relationship between the first programming language and the second programming language supported by the low-code platform, convert the first code into a second code in the second programming language.

[0050] The second code is used to generate a page corresponding to the target design draft in the low-code platform. Different low-code platforms support different second programming languages. For example, some low-code platforms support JSON Schema as the second programming language.

[0051] The mapping relationship between the first programming language and the second programming language describes the rules for converting the code of the first programming language into the code of the second programming language. The mapping relationship may, for example, include but is not limited to the mapping relationship in the description of component types, sub-component structures and interactive events. For example, assuming that the first programming language is JSX and the second programming language is JSON Schema, then the mapping relationship between the two includes: using tags in JSX to describe component types, and using the type attribute in JSON Schema to specify the type of component; using nested structures in JSX to describe the nested relationship between components, and using the child attribute in JSON Schema to describe the array of sub-components, each sub-component is an object with its own type and props attributes; using onClick={handleSubmit} in JSX to describe the event triggered after the component is clicked, and using "actionType":"ajax" in JSON Schema to describe the event.

[0052] In the above S106 , the first code in the first programming language may be converted into the second code in the second programming language in various appropriate ways, which is not limited in this embodiment of the present specification.

[0053] In one implementation, S106 includes the following steps: parsing the first code to obtain an abstract syntax tree corresponding to the first code; and converting the abstract syntax tree into a second code in the second programming language based on the abstract syntax tree and a mapping relationship between the first programming language and the second programming language.

[0054] The abstract syntax tree describes the grammatical structure of the first code. The abstract syntax tree includes multiple nodes, and each node of the abstract syntax tree is traversed using a recursive method to extract information according to the node type, such as extracting tag names and attributes according to element nodes, extracting attribute names and values ​​according to attribute nodes, and extracting text content according to text nodes, etc.; further, according to the syntax specification of the second programming language, based on the extracted information, the nodes are mapped to data of the second programming language, such as mapping tag names to tagName or similar fields, mapping attributes to key-value pairs in objects, mapping sub-elements to children arrays, and each item in the array is a JSON Schema of a sub-element, etc., to obtain the second code.

[0055] It can be understood that the abstract syntax tree, as an abstract representation of the first code, can accurately reflect the components, attributes, nested relationships, etc. in the first code, and can retain data binding semantics. Code conversion based on the abstract syntax tree can significantly improve the accuracy and reliability of the conversion.

[0056] In another implementation, the first code includes a first component code corresponding to a component in the target design draft, and the component attributes of the component include the type of the component. The above S106 includes the following steps: based on the type of the component in the target design draft and the component type supported by the low-code platform, determine the first component and the second component in the target design draft, the first component is a component supported by the low-code platform, and the second component is a component not supported by the low-code platform; based on the mapping relationship between the first programming language and the second programming language, convert the first component code corresponding to the first component to obtain the second component code corresponding to the first component; create a third component with a code compilation function in the low-code platform, and compile the first component code corresponding to the second component through the third component to obtain the second component code corresponding to the second component; based on the hierarchical relationship between components in the target design draft, combine the second component code corresponding to the first component and the second component code corresponding to the second component to obtain the second code in the second programming language.

[0057] In the application, the component types supported by the low-code platform can be pre-configured according to actual needs. For example, the component types supported by the low-code platform include: tables, input boxes, calendars, bar charts, line charts, pie charts, etc. The third component refers to the component with code compilation function supported by the low-code platform, also known as the bottom-up component.

[0058] For the first component, the first component code of the first component can be parsed to obtain the abstract syntax tree of the first component code, and then the abstract syntax tree can be converted into the second component code of the second programming language. For the second component, the third component can be loaded in the preview area of ​​the low-code platform, and the first component code of the second component can be compiled by the third component to obtain the compatibility shim code, which is the second component code corresponding to the second component. Finally, according to the hierarchical relationship between components, the second component codes corresponding to all components in the target design draft are combined to obtain the second code of the second programming language.

[0059] Since the component types supported by the low-code platform are limited in actual applications, the above-mentioned conversion method can effectively avoid the problem of unsuccessful conversion of some components caused by the components in the target design draft exceeding the scope of components supported by the low-code platform, thereby ensuring that the second code can accurately restore the complete content of the target design draft.

[0060] In another embodiment of the present specification, the following steps may be included after the above S106: based on the second code, displaying the page corresponding to the target design draft in the low-code platform; detecting whether the page meets the preset design specifications through the optimization model, and if the page does not meet the design specifications, correcting the page.

[0061] Specifically, the page includes components in the target design draft. If the layer spacing of the page is less than or equal to the preset spacing, based on the difference between the layer spacing and the preset spacing, a fourth component is added to the page, and a first prompt message is output, where the fourth component is a component supported by the low-code platform for increasing the layer spacing; and / or, if there is a fifth component with an undefined interactive state in the page, a preset interactive style template is added to the page for the fifth component, and a second prompt message is output.

[0062] For example, the page's layer spacing is 5px grid, and the preset design specification requires the layer spacing to be 8px grid. In this case, add a 3px grid between two adjacent layers on the page. <space>Components are added to increase the layer spacing to 8px grid. Further, a first prompt message may be output to prompt the developer that the layer spacing of the page does not meet the specification. The first prompt message may include, but is not limited to, at least one of the following information: the layer spacing of the page, the layer spacing required by the design specification, the added spacing component and the size of the spacing component, etc.

[0063] For another example, there is a button component in the page, but the style of the button component when hovering or focusing is not specified. In this regard, a preset hover style template or focus style template is added to the button component. For example, when the mouse hovers or focuses on the button component, the color or style of the mouse is changed, thereby improving the visual experience of the page. Furthermore, a second prompt information can be output to prompt the developer that there is a component with an undefined interactive state in the page. The second prompt information can include, for example, but is not limited to, at least one of the following information: the location of the component, the hover style template or focus style template added for the component, etc.

[0064] In the application, the optimization model can be trained in the following manner: obtaining the page information of the sample page corresponding to the sample design draft and the modification record corresponding to the sample page; and training the optimization model based on the page information and the modification record.

[0065] The page information may include, but is not limited to, the layer tree structure of the page, the performance tracking data of the page, etc. The performance tracking data may include, but is not limited to: LCP (Largest Contentful Paint), CLS (Cumulative Layout Shift), FID (First Input Delay), etc.

[0066] Exemplarily, the page information of the sample page can be used as input, and the modification record corresponding to the sample page can be used as a label, and the optimization model can be trained using a supervised training method, so that the trained optimization model can accurately identify whether any page meets the preset design specifications.

[0067] By displaying, testing and correcting the page corresponding to the target design draft, a high-quality page can be generated even if the target design draft has deficiencies.

[0068] In another embodiment of this specification, Figure 4 As shown, before the above S102, the following steps may also be included: in response to receiving a target text indicating generation of a design draft, extracting requirement information for the design draft from the target text; generating a candidate design draft based on the recognition model and the requirement information; in response to receiving negative feedback information on the candidate design draft, outputting a third prompt information, the third prompt information being used to instruct correction of the requirement information; in response to receiving correction information of the requirement information, correcting the candidate design draft through the recognition model to obtain the target design draft.

[0069] For example, if the target text is "help me generate a work order management system, expand the requirements first, find out what functions the common work order systems on the market have, list the tasks, and then convert the tasks into JSX pages, and finally to DSL pages", the requirements information extracted from the target text includes: work order management system, common on the market, functions of work order systems, task lists, JSX pages, DSL (Domain-Specific Language) pages. Based on this requirement information, the recognition model searches for the functions of the current common work order systems on the Internet, generates a candidate design draft with these functions and displays it.

[0070] If positive feedback information is received for the candidate design draft, the candidate design draft is determined as the target design draft, and then according to the above S102~S106, the target design draft is first converted into JSX code, and then the JSX code is converted into DSL code supported by the low-code platform. Among them, the positive feedback information may include, for example, but is not limited to: negative evaluation text of the candidate design draft, adding an unqualified label to the candidate design draft, and the candidate design draft is not adopted.

[0071] If negative feedback information is received for the candidate design draft, a third prompt information "Please list the areas that need to be improved in the above design draft" is output. Then, a task list is generated according to the correction information of the demand information, and a new candidate design draft is generated according to the task list. Among them, the negative feedback information may include, for example, but is not limited to: positive evaluation text of the candidate design draft, adding a qualified label to the candidate design draft, the candidate design draft being adopted, etc.

[0072] If positive feedback is received for a new candidate design draft, the new candidate design draft is determined as the target design draft, and then according to the above S102 to S106, the target design draft is first converted into JSX code, and then the JSX code is converted into DSL code supported by the low-code platform. The task list is as follows: Ticket creation page: problem type selection (drop-down menu / category icon), problem description editor (supports uploading attachments), priority / urgency mark, and related product / service selection; Work order tracking also includes: personal work order list (sorted by status / time), work order progress bar visualization, historical response record viewing, and satisfaction rating entry; Knowledge base search page: intelligent search box (supports association of related words), classification navigation of popular questions, solution voting mechanism, and recommendation of similar questions. Through the above embodiment, the user only needs to enter a concise and clear text description, and the system can quickly capture its needs and automatically generate a design draft that matches it. In this way, the source of the design draft is expanded, that is, the design draft can be a picture designed by a designer using professional design software, or it can be automatically generated based on the description text provided by the user. This method not only greatly enriches the application scenarios of the above-mentioned front-end page building method, but also significantly improves the efficiency of page development.

[0073] The front-end page building method provided in the embodiment of this specification takes into account the differences in features of different components in the design draft, extracts the multimodal features of the target design draft through the recognition model, and can accurately identify the component properties of the components in the target design draft and the hierarchical relationship between the components based on the multimodal features; then, based on the component properties of the components in the target design draft and the hierarchical relationship between the components, the target design draft is converted into a first code in a first programming language, the first code is used to describe the component properties of the components and the hierarchical relationship between the components, and has the advantage of being maintainable, and can be used to be modified and reused to adapt to changes in page development requirements, which is beneficial to improving page development efficiency; further, according to the mapping relationship between the first programming language and the second programming language supported by the low-code platform, the first code is converted into a second code in the second programming language, thereby realizing end-to-end automated conversion of the code from the design draft to the low-code platform, and the code can be used to generate a page corresponding to the target design draft in the low-code platform, so that the low-code platform can reuse the target design draft and automatically convert it into draggable components and configurations to adapt to changes in page development requirements, thereby improving page development efficiency. In addition, since the programming languages ​​supported by low-code platforms are usually closed source, the effect of directly converting the target design draft into the code of the programming language is not good. By first converting the target design draft into the first code of the open source first programming language, and then converting from the first code to the second code of the second programming language, the conversion effect is better and the conversion accuracy is higher. This method has cross-platform compatibility and supports the output of multi-terminal codes such as mini-programs, Taro, and React Native.

[0074] In order to facilitate the understanding of the above front-end page construction method, the following is combined Figure 5 , using a specific embodiment to illustrate the page design draft conversion process.

[0075] like Figure 5 As shown, the user inputs a page design draft or describes the page design draft to be generated in natural language text.

[0076] When the user inputs a page design draft, the page design draft is determined as the target design draft.

[0077] When the user inputs a natural language text, the recognition model recognizes the user's needs based on the natural language text. If the needs are unclear, the user is prompted to continue to supplement the needs information, or a candidate design draft is generated based on the existing needs, and if the user rejects the candidate design draft, the user is prompted to continue to revise the needs information. Then, based on the revised needs information, a new candidate design draft is generated, and if the user confirms the new candidate design draft, the new candidate design draft is determined as the target design draft.

[0078] Furthermore, the recognition type identifies the multimodal features of the page design draft, and identifies the component properties of the components in the page design draft and the hierarchical relationships between the components based on the multimodal features; then, based on the component properties of the components and the hierarchical relationships between the components, the page design draft is converted into JSX code.

[0079] Furthermore, according to the mapping relationship between the JSX code and the JSON Schema supported by the low-code platform, the JSX code is converted into the JSON Schema code.

[0080] Furthermore, based on the JSON Schema code, a page corresponding to the target design draft is generated in the low-code platform, and the page is tested to see if it meets the preset design specifications. If the page meets the design specifications, the JSON Schema code and page are adopted; if the page does not meet the design specifications, the page and the JSON Schema code are corrected.

[0081] In addition, Figure 1 Corresponding to the front-end page building method shown, the embodiment of this specification also provides a front-end page building device. Figure 6 6 is a schematic diagram of the structure of a front-end page building device 600 provided in an embodiment of the present specification, including: an identification module 610, a first conversion module 620 and a second conversion module 630.

[0082] The recognition module 610 is used to extract multimodal features of the target design draft through a recognition model, and to recognize component attributes of components in the target design draft and hierarchical relationships between components based on the multimodal features.

[0083] The first conversion module 620 is used to convert the target design draft into a first code in a first programming language based on the component properties of the components and the hierarchical relationship between the components.

[0084] The second conversion module 630 is used to convert the first code into a second code of the second programming language based on a mapping relationship between the first programming language and a second programming language supported by the low-code platform, and the second code is used to generate a page corresponding to the target design draft in the low-code platform.

[0085] The front-end page building device provided in the embodiment of the present specification takes into account the differences in features of different components in the design draft, extracts the multimodal features of the target design draft through the recognition model, and can accurately identify the component properties of the components in the target design draft and the hierarchical relationship between the components based on the multimodal features; then, based on the component properties of the components in the target design draft and the hierarchical relationship between the components, the target design draft is converted into a first code in a first programming language, the first code is used to describe the component properties of the components and the hierarchical relationship between the components, and has the advantage of being maintainable, and can be used to be modified and reused to adapt to changes in page development requirements, which is beneficial to improving page development efficiency; further, according to the mapping relationship between the first programming language and the second programming language supported by the low-code platform, the first code is converted into a second code in the second programming language, thereby realizing end-to-end automatic conversion of the code from the design draft to the low-code platform, and the code can be used to generate a page corresponding to the target design draft in the low-code platform, so that the low-code platform can reuse the target design draft and automatically convert it into draggable components and configurations to adapt to changes in page development requirements, thereby improving page development efficiency. In addition, since the programming languages ​​supported by low-code platforms are usually closed source, the effect of directly converting the target design draft into the code of the programming language is not good. By first converting the target design draft into the first code of the open source first programming language, and then converting from the first code to the second code of the second programming language, the conversion effect is better and the conversion accuracy is higher.

[0086] In another embodiment, the multimodal features of the target design draft include multimodal features of different regions in the target design draft; The identification module comprises: A first determination submodule, used to determine the target design draft as a first-level component; A second determination submodule is used to determine, starting from the first-level components, for each level of components based on the multimodal features of the area where the components of the current level are located, the component attributes of the current level components; The disassembly submodule is used to disassemble the current level component based on the component attributes of the current level component to obtain the next level component of the current level component.

[0087] In another embodiment, the multimodal features include visual features and structural features; The disassembly submodule is used for: If the multimodal features of the region where the component of the current level is located include multiple visual features, then based on the structural features of the region where the component of the current level is located, the region where the component of the current level is located is divided into multiple sub-regions; Each sub-area is identified separately as a next-level component.

[0088] In another embodiment, the first conversion module includes: A first generating submodule, configured to generate a code template corresponding to the target design draft based on the hierarchical relationship between the components and the grammatical specification of the first programming language; A second generating submodule, configured to generate a first component code corresponding to the component based on a component attribute of the component and a grammatical specification of the first programming language; The filling submodule is used to fill the first component code corresponding to the component into the code template to obtain the first code in the first programming language.

[0089] In another embodiment, the second conversion module includes: A parsing submodule, used to parse the first code to obtain an abstract syntax tree corresponding to the first code; The first conversion submodule is configured to convert the abstract syntax tree into a second code in the second programming language based on the abstract syntax tree and a mapping relationship between the first programming language and the second programming language.

[0090] In another embodiment, the first code includes a first component code corresponding to a component in the target design draft, and the component attribute of the component includes a type of the component; The second conversion module comprises: A third determination submodule is used to determine a first component and a second component in the target design draft based on the type of the component in the target design draft and the type of the component supported by the low-code platform, wherein the first component is a component supported by the low-code platform, and the second component is a component not supported by the low-code platform; A second conversion submodule, configured to convert a first component code corresponding to the first component based on a mapping relationship between the first programming language and the second programming language to obtain a second component code corresponding to the first component; A compilation submodule, used to create a third component with a code compilation function in the low-code platform, and compile the first component code corresponding to the second component through the third component to obtain the second component code corresponding to the second component; A combination submodule is used to combine the second component code corresponding to the first component and the second component code corresponding to the second component based on the hierarchical relationship between the components in the target design draft to obtain the second code of the second programming language.

[0091] In another embodiment, the front-end page building device further includes: A display module, used to display a page corresponding to the target design draft in the low-code platform based on the second code; The correction module is used to detect whether the page meets the preset design specifications through the optimization model, and if the page does not meet the design specifications, correct the page.

[0092] In another embodiment, the page includes components in the target design draft; The correction module is used for: If the layer spacing of the page is less than or equal to the preset spacing, based on the difference between the layer spacing and the preset spacing, a fourth component is added to the page, and a first prompt message is output, wherein the fourth component is a component supported by the low-code platform for increasing the layer spacing; and / or, If there is a fifth component with an undefined interaction state in the page, a preset interaction style template is added to the page for the fifth component, and a second prompt information is output.

[0093] In another embodiment, the optimization model is trained in the following manner: Obtaining page information of a sample page corresponding to the sample design draft and a revision record corresponding to the sample page; The optimization model is trained based on the page information and the correction record.

[0094] In another embodiment, the front-end page building device further includes: A first response module, configured to extract requirement information for the design draft from the target text in response to receiving a target text indicating generation of a design draft; A generation module, used to generate candidate design drafts based on the recognition model and the demand information; A second response module, configured to output third prompt information in response to receiving negative feedback information on the candidate design draft, wherein the third prompt information is used to instruct to modify the requirement information; The third response module is used to modify the candidate design draft by using the recognition model in response to receiving the correction information of the requirement information to obtain the target design draft.

[0095] Obviously, the front-end page building device of the embodiment of this specification can be used as the above Figure 1 The execution body of the front-end page construction method shown in FIG. Figure 1 The functions realized are as follows. Since the principles are the same, they will not be described here.

[0096] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an embodiment of this specification. Figure 7 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0097] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0098] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0099] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a front-end page building device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: Extracting multimodal features of the target design draft through the recognition model, and identifying component attributes of components in the target design draft and hierarchical relationships between components based on the multimodal features; Based on the component properties of the components and the hierarchical relationship between the components, converting the target design draft into a first code in a first programming language; Based on the mapping relationship between the first programming language and the second programming language supported by the low-code platform, the first code is converted into a second code of the second programming language, and the second code is used to generate a page corresponding to the target design draft in the low-code platform.

[0100] The above is as in this manual Figure 1 The method performed by the front-end page building device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of this specification can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of this specification can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0101] It should be understood that the electronic device of the embodiment of this specification can implement the front-end page building device in Figure 1 Functions of the embodiments shown. Since the principles are the same, the embodiments of this specification will not be described in detail here.

[0102] Of course, in addition to software implementation, the electronic device of this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0103] The embodiment of the present specification also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method of the embodiment shown is specifically used to perform the following operations: Extracting multimodal features of the target design draft through the recognition model, and identifying component attributes of components in the target design draft and hierarchical relationships between components based on the multimodal features; Based on the component properties of the components and the hierarchical relationship between the components, converting the target design draft into a first code in a first programming language; Based on the mapping relationship between the first programming language and the second programming language supported by the low-code platform, the first code is converted into a second code of the second programming language, and the second code is used to generate a page corresponding to the target design draft in the low-code platform.

[0104] The embodiments of this specification also provide a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps in the front-end page building method provided in the embodiments of this specification.

[0105] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0106] In short, the above description is only a preferred embodiment of this specification and is not intended to limit the protection scope of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the protection scope of this specification.

[0107] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0108] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0109] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, commodity or device including the element.

[0110] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.< / space> < / container> < / container> < / container>

Claims

1. A front-end page construction method, characterized in that: include: Extracting multimodal features of the target design draft through the recognition model, and identifying component attributes of components in the target design draft and hierarchical relationships between components based on the multimodal features; Based on the component properties of the components and the hierarchical relationship between the components, converting the target design draft into a first code in a first programming language; Based on the mapping relationship between the first programming language and the second programming language supported by the low-code platform, the first code is converted into a second code of the second programming language, and the second code is used to generate a page corresponding to the target design draft in the low-code platform.

2. The method according to claim 1, characterized in that The multimodal features of the target design draft include multimodal features of different regions in the target design draft; The identifying, based on the multimodal features, component attributes of the components in the target design draft and the hierarchical relationships between the components includes: Determine the target design draft as a first-level component; Starting from the first-level components, for each level of components, based on the multimodal features of the region where the components of this level are located, determine the component attributes of the components of this level; The components at this level are disassembled based on the component properties of the components at this level to obtain the next level components of the components at this level.

3. The method according to claim 2, characterized in that The multimodal features include visual features and structural features; The step of disassembling the component at the current level based on the component attribute of the component at the current level to obtain the next level component of the component at the current level includes: If the multimodal features of the region where the component of the current level is located include multiple visual features, then based on the structural features of the region where the component of the current level is located, the region where the component of the current level is located is divided into multiple sub-regions; Each sub-area is identified separately as a next-level component.

4. The method according to claim 1, characterized in that: The converting the target design draft into a first code in a first programming language based on the component attributes of the component and the hierarchical relationship between the components includes: Based on the hierarchical relationship between the components and the grammatical specification of the first programming language, a code template corresponding to the target design draft is generated; Based on the component attributes of the component and the grammatical specification of the first programming language, generate a first component code corresponding to the component; Fill the code template with a first component code corresponding to the component to obtain a first code in the first programming language.

5. The method according to claim 1, characterized in that The converting the first code into a second code in the second programming language based on a mapping relationship between the first programming language and a second programming language supported by the low-code platform includes: Parsing the first code to obtain an abstract syntax tree corresponding to the first code; Based on the abstract syntax tree and a mapping relationship between the first programming language and the second programming language, the abstract syntax tree is converted into a second code of the second programming language.

6. The method according to claim 1, characterized in that The first code includes a first component code corresponding to a component in the target design draft, and the component attribute of the component includes a type of the component; The converting the first code into a second code in the second programming language based on a mapping relationship between the first programming language and a second programming language supported by the low-code platform includes: Based on the types of components in the target design draft and the types of components supported by the low-code platform, determine the first component and the second component in the target design draft, wherein the first component is a component supported by the low-code platform, and the second component is a component not supported by the low-code platform; Based on the mapping relationship between the first programming language and the second programming language, convert the first component code corresponding to the first component to obtain the second component code corresponding to the first component; Creating a third component with a code compilation function in the low-code platform, and compiling the first component code corresponding to the second component through the third component to obtain the second component code corresponding to the second component; Based on the hierarchical relationship between components in the target design draft, the second component code corresponding to the first component and the second component code corresponding to the second component are combined to obtain a second code in the second programming language.

7. The method according to any one of claims 1 to 6, characterized in that After converting the first code into a second code in the second programming language based on a mapping relationship between the first programming language and a second programming language supported by the low-code platform, the method further includes: Based on the second code, displaying a page corresponding to the target design draft in the low-code platform; By optimizing the model, it is detected whether the page meets the preset design specifications, and if the page does not meet the design specifications, the page is corrected.

8. The method according to claim 7, characterized in that The page includes components in the target design draft; If the page does not meet the design specification, the page is corrected, including: If the layer spacing of the page is less than or equal to the preset spacing, based on the difference between the layer spacing and the preset spacing, a fourth component is added to the page, and a first prompt message is output, wherein the fourth component is a component supported by the low-code platform for increasing the layer spacing; and / or, If there is a fifth component with an undefined interaction state in the page, a preset interaction style template is added to the page for the fifth component, and a second prompt information is output.

9. The method according to claim 8, characterized in that The optimization model is trained in the following way: Obtaining page information of a sample page corresponding to the sample design draft and a revision record corresponding to the sample page; The optimization model is trained based on the page information and the correction record.

10. The method according to any one of claims 1 to 6, characterized in that Before extracting the multimodal features of the target design draft through the recognition model, the method further includes: In response to receiving a target text indicating generation of a design draft, extracting requirement information for the design draft from the target text; Based on the recognition model and the demand information, generating a candidate design draft; In response to receiving negative feedback information on the candidate design draft, outputting third prompt information, wherein the third prompt information is used to instruct to revise the requirement information; In response to receiving the correction information of the requirement information, the candidate design draft is corrected by using the recognition model to obtain the target design draft.

11. A front-end page building device, characterized in that: include: An identification module, used to extract multimodal features of a target design draft through a recognition model, and identify component attributes of components in the target design draft and hierarchical relationships between components based on the multimodal features; A first conversion module, configured to convert the target design draft into a first code in a first programming language based on the component attributes of the components and the hierarchical relationship between the components; The second conversion module is used to convert the first code into a second code of the second programming language based on a mapping relationship between the first programming language and a second programming language supported by the low-code platform, and the second code is used to generate a page corresponding to the target design draft in the low-code platform.

12. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the front-end page building method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the front-end page building method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the front-end page building method as described in any one of claims 1 to 10.

Images