Automatic generation method and device of interactive design prototype

Abstract

The invention discloses an automatic generation method and device for an interactive design prototype, and the method comprises the steps: analyzing a graphic layer structure of a selected design file in a canvas page, and automatically recognizing and extracting interactive elements in the graphic layer structure; determining a logic sequence of the interaction elements according to the layer names and the arrangement sequence in the layer structure; inputting the interaction elements and the logic sequence thereof into a pre-trained interaction design network model, and deducing an interaction relationship and an interaction path between the interaction elements; and generating an interaction prototype of the design file according to the interaction relationship and the interaction path between the interaction elements. According to the method, the generation speed of the interactive design prototype is increased, and meanwhile, the consistency of interactive prototype design is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of software design, and particularly to a method and device for automatically generating an interactive design prototype. Background Art

[0002] When generating an interactive prototype with existing interactive design tools, designers usually need to manually define the association relationships and jump paths between each interactive element. For complex and multi-step interactive processes, it is not only time-consuming but also error-prone. Designers need to spend a lot of effort on adjustment and testing. Especially when requirements change or modifications are needed, the maintenance cost is relatively high, and it is difficult to meet complex interactive requirements. In addition, due to the lack of built-in detection functions in existing interactive design tools, designers need to manually analyze and detect potential interactive problems, resulting in low efficiency.

[0003] To solve the above problems, the present invention develops a method and device for automatically generating an interactive design prototype to improve the generation speed of the interactive design prototype. Summary of the Invention

[0004] In view of the above problems, a method and device for automatically generating an interactive design prototype are proposed to facilitate improving the generation speed of the interactive design prototype.

[0005] According to one aspect of the present invention, a method for automatically generating an interactive design prototype is provided, including:

[0006] Analyze the layer structure of the selected design file in the canvas page, and automatically identify and extract the interactive elements in the layer structure;

[0007] Determine the logical order of the interactive elements according to the layer names and arrangement order in the layer structure;

[0008] Input the interactive elements and their logical order into a pre-trained interactive design network model to infer the interactive relationships and interactive paths between the interactive elements;

[0009] Generate an interactive prototype of the design file according to the interactive relationships and interactive paths between the interactive elements.

[0010] In an alternative manner, the method further includes:

[0011] Customize and adjust the interactive relationships and interactive paths between the interactive elements;

[0012] Wherein, the interactive relationships are modified through interactive logic rules, and the interactive logic rules include conditional judgment rules and loop rules;

[0013] The interaction path modifies the connection relationship of the selected interaction path through the path attribute panel, or creates the connection relationship between the interaction elements through gestures.

[0014] In an alternative approach, the step of parsing the layer structure of the selected design file in the parsing canvas page and automatically identifying and extracting the interaction elements in the layer structure further includes:

[0015] Extract the parent-child relationships between the individual layers from the design file and construct a layer tree network structure, where the nodes of the layer tree are the layers and the edges of the layer tree are the layer attributes, and the layer attributes include the name, type, position, size, style, visibility, constraints, and links of the layer;

[0016] Identify and extract the interaction elements in the layer structure according to layer attribute rules or a trained interaction element prediction model.

[0017] In an alternative approach, the step of determining the logical order of the interaction elements according to the layer names and arrangement order in the layer structure further includes:

[0018] Infer the interaction trigger points and the interaction states of the interaction elements contained in the layer according to the layer naming rules and the layer names;

[0019] Determine the logical order of the interaction trigger points according to the interaction states of the interaction elements and the position and arrangement of the layer on the canvas page.

[0020] In an alternative approach, the step of inputting the interaction elements and their logical order into a pre-trained interaction design network model to infer the interaction relationships and interaction paths between the interaction elements further includes:

[0021] Encode the interaction elements according to the layer attributes to obtain interaction element encoding vectors;

[0022] Assign a logical order encoding combination to each interaction element, where the logical order encoding combination includes an absolute position number and a relative position encoding; where the absolute position number is used to represent the order in the interaction sequence; the relative position encoding is used to record the relative order relationship between the interaction elements;

[0023] Input the interaction element encoding vectors and the logical order encoding combinations into the interaction design network model, and output the multiple possible interaction paths and interaction relationship probabilities between each interaction element and other interaction elements;

[0024] Input the interaction paths and interaction relationship probabilities into the Beam Search algorithm to calculate the best interaction path for each interaction element.

[0025] In an alternative approach, the interactive design network model includes a layer attribute encoding layer, a UI interface context encoding layer, an interaction reasoning layer, and an interaction path decoding layer;

[0026] Among them, the layer attribute encoding layer is used to encode layer name, type, position, size, style, visibility, constraints, and link information to obtain a layer attribute encoding vector;

[0027] The UI interface context encoding layer is used to encode interface type, screen size, theme color, and visual complexity, where the visual complexity includes the number of colors, edge density, and information entropy, to obtain a UI interface context encoding vector;

[0028] The interaction reasoning layer is used to output an interaction relationship probability matrix based on the layer attribute encoding vector, the UI interface context encoding vector, and the logical order encoding combination;

[0029] The interaction path decoding layer is used to generate multiple possible interaction paths for each interaction element according to the interaction relationship probability matrix.

[0030] In an alternative approach, generating the interaction prototype of the design document based on the interaction relationship and interaction path between the interaction elements further includes:

[0031] Selecting the interaction relationships with probabilities higher than a preset threshold according to the interaction relationship probabilities between each interaction element and other interaction elements;

[0032] Establishing connections between the interaction elements according to the selected interaction relationships, and binding corresponding types of interaction actions to the interaction elements according to the types of the interaction relationships;

[0033] Generating and previewing the interaction prototype of the design document according to the interaction actions.

[0034] In an alternative approach, binding corresponding types of interaction actions to the interaction elements according to the types of the interaction relationships further includes:

[0035] Determining the roles of each interaction element in the interaction prototype, where the roles include the source element that triggers the interaction and the target element that responds to the interaction;

[0036] Searching for interaction actions that match the roles and types of the interaction relationships of the interaction elements according to a predefined interaction action mapping table;

[0037] Automatically generating corresponding interaction event listening codes according to the found interaction actions and their parameter configuration items, and binding the listening codes to the interaction elements to achieve the interaction effect.

[0038] In an alternative approach, the method further includes:

[0039] Perform experience analysis and automatic detection on the generated interaction prototype, where the metrics for experience analysis include interaction completion time, number of clicks, error rate, and user satisfaction score; automatic detection includes the loop status, reachable status of the interaction process, and the consistency of operation logic.

[0040] According to another aspect of the present application, there is provided an automated generation device for an interaction design prototype, including:

[0041] An interaction element extraction module for parsing the layer structure of the selected design file in the canvas page and automatically identifying and extracting the interaction elements in the layer structure;

[0042] A logical order determination module for determining the logical order of the interaction elements according to the layer names and arrangement order in the layer structure;

[0043] An interaction relationship inference module for inputting the interaction elements and their logical order into a pre-trained interaction design network model to infer the interaction relationships and interaction paths between the interaction elements;

[0044] An interaction prototype generation module for generating the interaction prototype of the design file according to the interaction relationships and interaction paths between the interaction elements.

[0045] The solution provided in the above embodiments of the present invention parses the layer structure of the selected design file in the canvas page, automatically identifies and extracts the interaction elements in the layer structure; determines the logical order of the interaction elements according to the layer names and arrangement order in the layer structure; inputs the interaction elements and their logical order into a pre-trained interaction design network model to infer the interaction relationships and interaction paths between the interaction elements; and generates the interaction prototype of the design file according to the interaction relationships and interaction paths between the interaction elements. The present invention improves the generation speed of the interaction design prototype and at the same time ensures the consistency of the interaction prototype design.

[0046] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above description and other purposes, features and advantages of the present invention more obvious and understandable, the following specifically illustrates the specific embodiments of the present invention. Brief Description of the Drawings

[0047] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. And throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0048] Figure 1 Shows a schematic flowchart of a method for automatically generating an interaction design prototype according to an embodiment of the present invention;

[0049] Figure 2 Shows a schematic diagram of a design prototype according to an embodiment of the present invention;

[0050] Figure 3 Shows a schematic diagram of an interaction design prototype area according to an embodiment of the present invention;

[0051] Figure 4 Shows a schematic diagram of path generation and optimization according to an embodiment of the present invention;

[0052] Figure 5 Shows a schematic functional structure diagram of an apparatus for a method for automatically generating an interaction design prototype according to an embodiment of the present invention. Detailed implementation manners

[0053] Hereinafter, exemplary embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be completely conveyed to those skilled in the art.

[0054] The following describes in detail the method and apparatus for automatically generating an interaction design prototype proposed by the present invention through specific embodiments.

[0055] Embodiment 1:

[0056] Figure 1 Shows a schematic functional structure diagram of a method for automatically generating an interaction design prototype according to an embodiment of the present invention. Specifically, as Figure 1 shown, the following steps are included:

[0057] Step S101, parsing the layer structure of the selected design file in the canvas page, and automatically identifying and extracting the interaction elements in the layer structure.

[0058] Traditional prototype production relies on manual marking and defining of interaction elements. Due to the increasing complexity of UI design and the more cumbersome layer structure, it is easy to neglect and make mistakes through manual marking. By parsing the layer structure, potential interaction elements can be more accurately identified, more complex layer structures can be processed, and thus designers can devote more energy to creativity and user experience optimization.

[0059] In an alternative manner, the parsing of the layer structure of the selected design file in the canvas page and automatically identifying and extracting the interaction elements in the layer structure further includes:

[0060] Extracting the parent-child relationship between each layer from the design file, and constructing a layer tree network structure, wherein the nodes of the layer tree are layers, the edges of the layer tree are layer attributes, and the layer attributes include the name, type, position, size, style, visibility, constraints and links of the layer;

[0061] The interactive elements in the layer structure are identified and extracted according to the layer attribute rules or the trained interactive element prediction model.

[0062] In this embodiment, the design file is parsed using the API or SDK provided by the design tool, and the layers are constructed into a tree structure according to the parent-child relationship between the layers. The layer tree is traversed to extract the properties of each layer (including name, type, position, size, style, visibility, constraints, and links, etc.). The defined layer attribute rules are used to determine whether it is an interactive element. For example, if the layer name contains keywords such as "Button", "Btn", "Button", etc., it is considered to be a button, and if the layer has a link attribute, it is considered to be a link.

[0063] Step S102: determining the logical order of the interactive elements according to the layer names and arrangement order in the layer structure.

[0064] When creating an interface, designers usually sort and name the layers according to the operation process or importance. Based on the above information, the logical order of interaction can be inferred more accurately. For example, layers named "Step1Button", "Step2Input" and layers arranged at the top usually indicate higher priority and earlier interaction order. For another example, layer names containing "Step1", "Step2", "Step3" and so on usually indicate step numbers, layer names containing "_active", "_disabled", "_hover" usually indicate element states, and layer names containing "ClickButton", "SubmitForm" and so on usually indicate triggering events. The layer names can be parsed according to the naming rules to extract interaction information. For example, "Step1Button" is before "Step2Input", and the layer name "Step2_SubmitButton" indicates that the element is the submit button of the second step. Since the layer names and arrangement order are ready-made information in the design file, there is no need for complex analysis or user behavior simulation. The interaction order can be determined only by parsing the layer information.

[0065] In an optional manner, determining the logical order of the interactive elements according to the layer names and arrangement order in the layer structure further includes:

[0066] According to the layer naming rules and layer names, infer the interaction trigger points and interaction status of the interactive elements contained in the layer;

[0067] Determine the logical order of the interaction trigger points according to the interaction state of the interaction elements and the position and arrangement of the layers on the canvas page.

[0068] In this embodiment, as Figure 2 , Figure 3 shown, the interaction elements, interaction states, and priority information represented by the layers are indicated by the layer naming convention. For example, BTN_Primary_Normal represents the default state of the button, and Modal_Overlay represents the mask layer of the modal box. Parse the layer name to extract key information, and infer the interaction trigger points corresponding to the layers and the interaction states of the interaction elements according to the parsed key information (interaction element type, interaction state, and priority). For example, from BTN_Primary_Normal, it is inferred that the interaction trigger point is button click and the interaction state is the default state. Combine the interaction state of the interaction elements, the position and arrangement of the layers on the canvas page, and determine the logical order of the interaction trigger points. For example, if the hover state layer of the button (BTN_Primary_Hover) is above the normal state layer (BTN_Primary_Normal), the priority of the hover state is higher than that of the normal state. If the mask layer of the modal box (Modal_Overlay) is above all other elements, the interaction priority of this mask layer is the highest (preventing the user from interacting with the lower-level elements). Determine the display and interaction order of the elements according to the Z-axis position of the layers on the canvas, and the higher the layer is on the Z-axis, the higher its priority.

[0069] For example, the login interface contains the following layers: BG_Login (background image), Input_Username_Normal (username input box - default state), Input_Username_Focused (username input box - focused state), Input_Password_Normal (password input box - default state), Input_Password_Focused (password input box - focused state), BTN_Login_Normal (login button - default state), BTN_Login_Hover (login button - hover state), BTN_Login_Pressed (login button - pressed state), Text_Error (error prompt text, hidden in the initial state), Modal_Loading (loading modal box, hidden in the initial state), Modal_Loading_Overlay (overlay of the loading modal box). Analyze each layer name to extract the element type and state. Input_Username_Normal -> username input box, default state, trigger point: getting focus; BTN_Login_Hover -> login button, hover state, trigger point: mouse hovering. Priority: Modal_Loading_Overlay > Modal_Loading > Text_Error > BTN_Login_Pressed > BTN_Login_Hover > BTN_Login_Normal > Input_Password_Normal > Input_Username_Focused > Input_Username_Normal > BG_Login. The reason is that Modal_Loading_Overlay is on the top layer, covering all elements, with the highest priority to prevent users from performing other operations during loading. Text_Error is only displayed when there is an error, so its display priority is higher than other elements. The pressed state of the button has a higher priority than the hover state and the default state, and the background image has the lowest priority. Layers with a higher Z-axis position have a higher priority. The final interaction logic is as follows: Click on the username input box to display Input_Username_Focused, click on the password input box to display Input_Password_Focused. Mouse hover over the login button to display BTN_Login_Hover. Click on the login button to display BTN_Login_Pressed. If the login fails, display Text_Error. If the login is successful, display Modal_Loading and Modal_Loading_Overlay.

[0070] Step S103: Input the interactive elements and their logical order into a pre-trained interactive design network model to infer the interaction relationships and interaction paths between the interactive elements.

[0071] In this embodiment, by using a pre-trained interactive design network model, the relationships and interaction paths between interactive elements are automatically inferred, reducing the workload of manual design.

[0072] In an alternative approach, the step of inputting the interactive elements and their logical order into a pre-trained interactive design network model to infer the interaction relationships and interaction paths between the interactive elements further includes:

[0073] Encode the interactive elements according to the layer attributes to obtain an interactive element encoding vector;

[0074] Assign a logical order encoding combination to each interactive element, where the logical order encoding combination includes an absolute position number and a relative position encoding; the absolute position number is used to represent the order in the interaction sequence; the relative position encoding is used to record the relative order relationship between interactive elements;

[0075] Input the interactive element encoding vector and the logical order encoding combination into the interactive design network model, and output multiple possible interaction paths and interaction relationship probabilities between each interactive element and other interactive elements;

[0076] Input the interaction paths and interaction relationship probabilities into the Beam Search algorithm to calculate the best interaction path for each interactive element.

[0077] In this embodiment, as Figure 4As shown, through absolute position numbering and relative position encoding, the interaction design network model can understand the sequential relationship between interaction elements, thereby more reasonably inferring the interaction path. While exploring multiple possibilities for the multiple possible interaction paths and probabilities output by the Beam Search algorithm, it can efficiently find an approximately optimal interaction path and avoid the computational complexity of exhaustive search. For example, the login page includes BG_Login (background image), Input_Username_Normal (username input box - default state), Input_Password_Normal (password input box - default state), BTN_Login_Normal (login button - default state), Text_Error (error prompt text, initially hidden). Among them, the encoding vector of Input_Username_Normal includes: type: "text box", size: (200, 30), color: (255, 255, 255), font: "Arial", position: (100, 100), text: (encoding of the pre-trained word vector pair "username"); assume the interaction sequence is: Input_Username_Normal -> Input_Password_Normal -> BTN_Login_Normal. The absolute position number of Input_Username_Normal: 1, relative position encoding: {Input_Password_Normal: 1, BTN_Login_Normal: 2}. The absolute position number of Input_Password_Normal: 2, relative position encoding: {Input_Username_Normal: -1, BTN_Login_Normal: 1}. Inputting the encoding vector and logical order encoding into the model, the interaction probability between each interaction element is output: P(Input_Username_Normal -> Input_Password_Normal) = 0.8, P(Input_Username_Normal -> BTN_Login_Normal) = 0.1, P(Input_Password_Normal -> BTN_Login_Normal) = 0.9, P(BTN_Login_Normal -> Text_Error) = 0.5 (if the login fails). Through the Beam Search algorithm, the best interaction path is found among the multiple possible paths output by the model.Assume beam size = 2, initial path: Input_Username_Normal, extended paths: Input_Username_Normal -> Input_Password_Normal (probability 0.8), Input_Username_Normal -> BTN_Login_Normal (probability 0.1). Prune and retain the two paths with the highest probabilities: Input_Username_Normal -> Input_Password_Normal, and continue to extend until a complete path is found: Input_Username_Normal -> Input_Password_Normal -> BTN_Login_Normal. If the login fails, the path is: Input_Username_Normal -> Input_Password_Normal -> BTN_Login_Normal -> Text_Error.

[0078] In an alternative approach, the interaction design network model includes a layer attribute encoding layer, a UI interface context encoding layer, an interaction reasoning layer, and an interaction path decoding layer;

[0079] Among them, the layer attribute encoding layer is used to encode layer name, type, location, size, style, visibility, constraints, and link information to obtain a layer attribute encoding vector;

[0080] The UI interface context encoding layer is used to encode interface type, screen size, theme color, and visual complexity. The visual complexity includes the number of colors, edge density, and information entropy, to obtain a UI interface context encoding vector;

[0081] The interaction reasoning layer is used to output an interaction relationship probability matrix based on the layer attribute encoding vector, the UI interface context encoding vector, and the logical order encoding combination;

[0082] The interaction path decoding layer is used to generate multiple possible interaction paths for each interaction element according to the interaction relationship probability matrix.

[0083] In this embodiment, through the UI interface context encoding layer, the model can understand the interface type, screen size, and theme color environment information, so as to make an interaction design that better conforms to user habits and usage scenarios. By analyzing the interaction relationship probability matrix, designers can understand the reasons for the model's interaction predictions. The generated multiple possible interaction paths provide more inspiration for designers. For example, for the design interaction of the product details page of an e-commerce App, the attributes of the product image layer are extracted through the layer attribute encoding layer, including type (image), position, size, link (large product details image), etc. The attributes of the "Add to Cart" button are extracted, including type (button), text ("Add to Cart"), color, position, etc., and the attributes of the "Buy Now" button are extracted, including type (button), text ("Buy Now"), color, position, etc. Through the UI interface context encoding layer, the interface type is obtained as the product details page, the screen size is the mobile phone screen size, the theme color is the main color of the App, and the visual complexity is the number of colors, edge density, and information entropy. Through the interaction reasoning layer, it is learned that after the user clicks on the product image, it is more likely to view the large product details image, and after clicking on the "Add to Cart" or "Buy Now" button, it is more likely to enter the shopping cart or settlement page. An interaction relationship probability matrix is generated by the model. For example, P(click on product image -> view large product details image) = 0.8, P(click on "Add to Cart" -> enter shopping cart) = 0.9, P(click on "Buy Now" -> enter settlement page) = 0.7. The possible interaction paths generated through the interaction path decoding layer include: Path 1: User clicks on "product image" -> views "large product details image" -> returns -> clicks on "Add to Cart"; Path 2: User clicks on "product image" -> views "large product details image" -> swipes to view product description -> clicks on "Buy Now"; Path 3: User directly clicks on "Add to Cart".

[0084] Step S104, generate an interaction prototype of the design document according to the interaction relationship and interaction path between the interaction elements.

[0085] In this embodiment, the generated interaction prototype intuitively displays the interaction relationship and path. Designers can quickly generate multiple interaction prototypes for testing and optimization, thus accelerating the design iteration process.

[0086] In an alternative manner, the generating an interaction prototype of the design document according to the interaction relationship and interaction path between the interaction elements further includes:

[0087] Select the interaction relationships with probabilities higher than a preset threshold according to the interaction relationship probabilities between each interaction element and other interaction elements;

[0088] Establish connections between the interaction elements according to the selected interaction relationships, and bind corresponding types of interaction actions to the interaction elements according to the types of the interaction relationships.

[0089] Generate and preview an interaction prototype of the design file according to the interaction action.

[0090] In this embodiment, by setting a preset threshold to filter out low-probability interaction relationships, more important and more likely interaction behaviors of users are retained, making the generated interaction prototype closer to actual user operations. Automatically bind corresponding interaction actions according to the type of interaction relationship, reducing the workload of manual configuration. For example, the following information in the interaction relationship probability matrix: P(click on the "search box" -> enter text in the "search box") = 0.95, P(click on the "search box" -> click on the "search button") = 0.8, P(click on the "search button" -> display the "search result list") = 0.9, P(click on an item in the "search result list" -> jump to the "product details page") = 0.7, P(click on the "search box" -> display "popular searches") = 0.6, and the preset threshold is 0.75. The filtered set of interaction relationships: {("search box", "search box"): 0.95, ("search button", "search result list"): 0.9}. Establish a connection between the "search box" and the "search result list" according to the filtered interaction relationships. The "search box" is bound to the "click" action, triggering the "display input method keyboard" event and the "enter text" action. The "search button" is bound to the "click" action, triggering the "display search result list" event. Generate and preview the interaction prototype on the canvas page. The designer can enter text in the "search box" and then click the "search button" to view the display effect of the "search result list".

[0091] In an alternative approach, the step of binding corresponding types of interaction actions to interaction elements according to the type of interaction relationship further includes:

[0092] Determine the role of each interaction element in the interaction prototype, where the role includes the source element that triggers the interaction and the target element that responds to the interaction;

[0093] According to a predefined interaction action mapping table, find the interaction action that matches the role of the interaction element and the type of interaction relationship;

[0094] Automatically generate corresponding interaction event listening code according to the found interaction action and its parameter configuration items, and bind the listening code to the interaction element to achieve the interaction effect.

[0095] In this embodiment, through a predefined interaction action mapping table and parameter configuration items, it is possible to ensure that the generated code style is consistent. The interaction actions and code logic are associated with the interaction prototype. After modifying the interaction prototype, the relevant interaction actions and code can be quickly updated, reducing the maintenance cost. At the same time, there is no need to deeply understand various interaction events and code implementation details, and only the interaction prototype itself needs to be concerned about, reducing the learning cost or curve.

[0096] Specifically, at the stage of interaction prototype design, clarify the roles of each interaction element, and define them as "source elements that trigger interactions" and "target elements that respond to interactions", and the role definition can be carried out through methods such as interface annotation and attribute configuration. For example, when a button is clicked and a dialog box pops up, the button is the source element and the dialog box is the target element. Identify the types of interaction relationships between interaction elements. For example, click includes single click and double click, hover includes mouse over and mouse out, slide includes page slide and element slide, and form includes form submission and reset. Design an interaction action mapping table, including: source element role: such as Button, Link, Image, Input, target element role: such as Dialog, Container, Element, interaction relationship type: such as Click, Hover, Change, interaction action: such as Show, Hide, Navigate, UpdateText, parameter configuration items: such as Show(Dialog ID), Navigate(URL), UpdateText(Target Element ID,New Text), etc. According to the roles of the interaction elements and the types of interaction relationships, find the matching interaction actions in the interaction action mapping table, and automatically generate the corresponding interaction event listening code according to the found interaction actions and their parameter configuration items. For example, use addEventListener to bind event listeners, and use attributes such as onclick and onmouseover to bind event handling functions.

[0097] In an optional manner, the method further includes:

[0098] Customize and adjust the interaction relationships and interaction paths between the interaction elements;

[0099] Among them, the interaction relationship is modified through interaction logic rules, and the interaction logic rules include conditional judgment rules and loop rules;

[0100] The interaction path modifies the connection relationship of the selected interaction path through a path attribute panel, or creates the connection relationship between the interaction elements through gestures.

[0101] In this embodiment, connection relationships are created through the path attribute panel and gestures, which further enables more intuitive management and modification of interaction paths and reduces manual workload. Specifically, in an interaction prototype design tool, user-defined conditional judgments and loop rules are set through a rule editor, and the defined interaction logic rules are applied to the interaction relationships between interaction elements to modify the execution conditions and order of interaction actions. The connection relationships of the selected interaction paths are modified through the path attribute panel, and interaction paths are directly created and modified on the interface through gestures such as dragging and clicking. For example, the connection relationships between source elements and target elements are modified, intermediate nodes in the interaction paths are added or deleted, and the order and priority of the interaction paths are modified.

[0102] In an alternative approach, the method further includes:

[0103] Performing experience analysis and automatic detection on the generated interaction prototype, where the metrics for experience analysis include interaction completion time, number of clicks, error rate, and user satisfaction score; automatic detection includes the loop status, reachability status of the interaction process, and consistency of operation logic.

[0104] In this embodiment, the automatic detection function replaces part of the manual testing, which reduces the workload of testers and discovers logical errors, loop problems, and reachability problems in the prototype as early as possible, avoiding affecting subsequent development.

[0105] The solution provided in the above embodiments of the present invention parses the layer structure of the selected design file in the canvas page, automatically identifies and extracts the interaction elements in the layer structure; determines the logical order of the interaction elements according to the layer names and arrangement order in the layer structure; inputs the interaction elements and their logical order into a pre-trained interaction design network model to infer the interaction relationships and interaction paths between the interaction elements; and generates an interaction prototype of the design file according to the interaction relationships and interaction paths between the interaction elements. The present invention improves the generation speed of the interaction design prototype and ensures the consistency of the interaction prototype design.

[0106] Embodiment 2:

[0107] Figure 5 Shows a functional structure diagram of an apparatus for an automatic generation method of an interaction design prototype according to an embodiment of the present invention. As Figure 5 shown, the apparatus includes:

[0108] An interaction element extraction module 501, configured to parse the layer structure of the selected design file in the canvas page, and automatically identify and extract the interaction elements in the layer structure;

[0109] A logical order determination module 502, configured to determine the logical order of the interaction elements according to the layer names and arrangement order in the layer structure;

[0110] An interaction relationship inference module 503, configured to input the interaction elements and their logical order into a pre-trained interaction design network model to infer the interaction relationship and interaction path between the interaction elements;

[0111] An interaction prototype generation module 504, configured to generate an interaction prototype of the design file according to the interaction relationship and interaction path between the interaction elements.

[0112] In an optional manner, it further includes a custom adjustment module, and the custom adjustment module is further configured to:

[0113] Perform custom adjustment on the interaction relationship and interaction path between the interaction elements;

[0114] Wherein, the interaction relationship is modified through interaction logic rules, and the interaction logic rules include conditional judgment rules and loop rules;

[0115] The interaction path modifies the connection relationship of the selected interaction path through a path attribute panel, or creates a connection relationship between the interaction elements through gestures.

[0116] In an optional manner, the interaction element extraction module 501 is further configured to:

[0117] Extract the parent-child relationship between each layer from the design file, and construct a layer tree network structure, where the nodes of the layer tree are layers, and the edges of the layer tree are layer attributes, and the layer attributes include the name, type, position, size, style, visibility, constraint, and link of the layer;

[0118] Identify and extract the interaction elements in the layer structure according to layer attribute rules or a trained interaction element prediction model.

[0119] In an optional manner, the logical order determination module 502 is further configured to:

[0120] Infer the interaction trigger points and the interaction states of the interaction elements contained in the layer according to the layer naming rules and the layer name;

[0121] Determine the logical order of the interaction trigger points according to the interaction states of the interaction elements and the position and arrangement of the layer on the canvas page.

[0122] In an optional manner, the interaction relationship inference module 503 is further configured to:

[0123] Encode the interaction elements according to the layer attributes to obtain an interaction element encoding vector;

[0124] Assign a logical sequence coding combination to each interaction element, where the logical sequence coding combination includes an absolute position number and a relative position code; wherein, the absolute position number is used to represent the order in the interaction sequence; the relative position code is used to record the relative order relationship between interaction elements;

[0125] Input the interaction element coding vector and the logical sequence coding combination into the interaction design network model, and output multiple possible interaction paths and interaction relationship probabilities between each interaction element and other interaction elements;

[0126] Input the interaction paths and interaction relationship probabilities into the Beam Search algorithm to calculate the best interaction path for each interaction element.

[0127] In an alternative manner, the interaction design network model includes a layer attribute coding layer, a UI interface context coding layer, an interaction reasoning layer, and an interaction path decoding layer;

[0128] Among them, the layer attribute coding layer is used to code layer name, type, position, size, style, visibility, constraints, and link information to obtain a layer attribute coding vector;

[0129] The UI interface context coding layer is used to code interface type, screen size, theme color, and visual complexity, and the visual complexity includes the number of colors, edge density, and information entropy, to obtain a UI interface context coding vector;

[0130] The interaction reasoning layer is used to output an interaction relationship probability matrix according to the layer attribute coding vector, the UI interface context coding vector, and the logical sequence coding combination;

[0131] The interaction path decoding layer is used to generate multiple possible interaction paths for each interaction element according to the interaction relationship probability matrix.

[0132] In an alternative manner, the interaction prototype generation module 504 is further configured to:

[0133] Select interaction relationships with probabilities higher than a preset threshold according to the interaction relationship probabilities between each interaction element and other interaction elements;

[0134] Establish connections between interaction elements according to the selected interaction relationships, and bind corresponding types of interaction actions to the interaction elements according to the types of interaction relationships;

[0135] Generate and preview the interaction prototype of the design file according to the interaction actions.

[0136] In an alternative manner, the interaction prototype generation module 504 is further configured to:

[0137] Determine the role of each interaction element in the interaction prototype, where the role includes a source element that triggers the interaction and a target element that responds to the interaction;

[0138] According to a predefined interaction action mapping table, find an interaction action that matches the role of the interaction element and the type of interaction relationship;

[0139] According to the found interaction action and its parameter configuration items, automatically generate corresponding interaction event listening code, and bind the listening code to the interaction element to achieve an interaction effect.

[0140] In an alternative approach, it further includes an automatic detection module, and the automatic detection module is further configured to:

[0141] Conduct experience analysis and automatic detection on the generated interaction prototype. Among them, the indicators of experience analysis include interaction completion time, number of clicks, error rate, and user satisfaction score; automatic detection includes the loop state, reachable state of the interaction process, and consistency of operation logic.

[0142] The solution provided in the above embodiments of the present invention analyzes the layer structure of the selected design file in the canvas page, automatically identifies and extracts the interaction elements in the layer structure; determines the logical order of the interaction elements according to the layer names and arrangement order in the layer structure; inputs the interaction elements and their logical order into a pre-trained interaction design network model to infer the interaction relationship and interaction path between the interaction elements; generates an interaction prototype of the design file according to the interaction relationship and interaction path between the interaction elements. The present invention improves the generation speed of the interaction design prototype and ensures the consistency of the interaction prototype design.

[0143] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. Based on the above description, the structure required to construct such a system is obvious. In addition, the embodiments of the present invention are not directed to any specific programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the description of a specific language above is for disclosing the best implementation manner of the present invention.

[0144] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and technologies have not been shown in detail so as not to obscure the understanding of this specification.

[0145] Similarly, it should be understood that, in order to streamline the present invention and assist in understanding one or more of the various inventive aspects, in the foregoing description of the exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in less than all of the features of the single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the present invention.

[0146] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and disposed in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0147] In addition, those skilled in the art will be able to understand that, although some of the embodiments herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of the present invention and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0148] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0149] It should be noted that the above embodiments illustrate the present invention rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for automatically generating an interactive design prototype, characterized in that: include: Parsing the layer structure of the design file selected in the canvas page, and automatically identifying and extracting interactive elements in the layer structure; Determining the logical order of the interactive elements according to the layer names and arrangement order in the layer structure; Inputting the interactive elements and their logical sequence into a pre-trained interactive design network model to infer the interactive relationship and interactive path between the interactive elements; An interactive prototype of the design file is generated according to the interactive relationship and interactive path between the interactive elements.

2. The method for automatically generating an interactive design prototype according to claim 1, characterized in that: The method further comprises: Customizing and adjusting the interaction relationship and interaction path between the interactive elements; Wherein, the interaction relationship is modified by interaction logic rules, and the interaction logic rules include conditional judgment rules and loop rules; The interactive path modifies the connection relationship of the selected interactive path through the path property panel, or creates the connection relationship between the interactive elements through gestures.

3. The method for automatically generating an interactive design prototype according to claim 1, characterized in that: The step of parsing the layer structure of the design file selected in the canvas page and automatically identifying and extracting the interactive elements in the layer structure further comprises: Extracting the parent-child relationship between each layer from the design file, and constructing a layer tree network structure, wherein the nodes of the layer tree are layers, the edges of the layer tree are layer attributes, and the layer attributes include the name, type, position, size, style, visibility, constraints and links of the layer; The interactive elements in the layer structure are identified and extracted according to the layer attribute rules or the trained interactive element prediction model.

4. The method for automatically generating an interactive design prototype according to claim 1, characterized in that: Determining the logical order of the interactive elements according to the layer names and arrangement order in the layer structure further includes: According to the layer naming rules and layer names, infer the interaction trigger points and interaction status of the interactive elements contained in the layer; The logical order of the interaction trigger points is determined according to the interaction status of the interactive elements and the position and arrangement of the layers on the canvas page.

5. The method for automatically generating an interactive design prototype according to claim 1, characterized in that: The step of inputting the interactive elements and their logical sequence into a pre-trained interactive design network model to infer the interactive relationship and interactive path between the interactive elements further includes: Encode the interactive elements according to the layer attributes to obtain the interactive element encoding vector; Assign a logical sequence coding combination to each interactive element, wherein the logical sequence coding combination includes an absolute position number and a relative position code; wherein the absolute position number is used to indicate the order in the interactive sequence; and the relative position code is used to record the relative order relationship between the interactive elements; The interaction element coding vector and the logic sequence coding combination are input into the interaction design network model, and a plurality of possible interaction paths and interaction relationship probabilities between each interaction element and other interaction elements are output; The interaction paths and interaction relationship probabilities are input into the Beam Search algorithm to calculate the optimal interaction path for each interaction element.

6. The method for automatically generating an interactive design prototype according to claim 5, characterized in that: The interactive design network model includes a layer attribute encoding layer, a UI interface context encoding layer, an interactive reasoning layer and an interactive path decoding layer; The layer attribute coding layer is used to encode the layer name, type, position, size, style, visibility, constraint and link information to obtain a layer attribute coding vector; The UI interface context coding layer is used to encode the interface type, screen size, theme color and visual complexity, wherein the visual complexity includes the number of colors, edge density and information entropy, to obtain a UI interface context coding vector; The interactive reasoning layer is used to output an interactive relationship probability matrix according to the layer attribute coding vector, the UI interface context coding direction, and the logical sequence coding combination; The interaction path decoding layer is used to generate multiple possible interaction paths for each interaction element according to the interaction relationship probability matrix.

7. The method for automatically generating an interactive design prototype according to claim 1, characterized in that: The generating the interactive prototype of the design file according to the interactive relationship and the interactive path between the interactive elements further comprises: According to the probability of interaction between each interactive element and other interactive elements, an interaction relationship with a probability higher than a preset threshold is selected; According to the selected interaction relationship, a connection is established between the interaction elements, and according to the type of interaction relationship, a corresponding type of interaction action is bound to the interaction element; An interactive prototype of the design file is generated and previewed according to the interactive action.

8. The method for automatically generating an interactive design prototype according to claim 7, characterized in that: Binding the interactive element with the corresponding type of interactive action according to the type of the interactive relationship further includes: Determine the role of each interactive element in the interactive prototype, wherein the role includes a source element that triggers the interaction and a target element that responds to the interaction; According to a predefined interaction action mapping table, searching for an interaction action that matches the role and interaction relationship type of the interaction element; According to the interaction action and its parameter configuration items found, the corresponding interaction event monitoring code is automatically generated, and the monitoring code is bound to the interaction element to achieve the interaction effect.

9. The method for automatically generating an interactive design prototype according to claim 1, characterized in that: The method further comprises: The generated interactive prototype is subjected to experience analysis and automatic detection, wherein the indicators of experience analysis include interaction completion time, number of clicks, error rate and user satisfaction score; and automatic detection includes the cycle state, reachable state and consistency of operation logic of the interactive process.

10. An automatic generation device for interactive design prototype, characterized in that: include: An interactive element extraction module, used to parse the layer structure of the design file selected in the canvas page, and automatically identify and extract interactive elements in the layer structure; A logic order determination module, used to determine the logic order of the interactive elements according to the layer names and arrangement order in the layer structure; An interaction relationship inference module, used to input the interaction elements and their logical sequence into a pre-trained interaction design network model, and infer the interaction relationship and interaction path between the interaction elements; The interactive prototype generation module is used to generate the interactive prototype of the design file according to the interactive relationship and interactive path between the interactive elements.

Images