A deep learning-based configuration interface dynamic layout optimization method

Abstract

The application discloses a kind of based on deep learning's configuration interface dynamic layout optimization method, comprising the following steps: step one: original data is collected and standardized configuration interface;Step two: respectively by multimodal fusion module is carried out feature extraction and is carried out feature fusion;Step three: layout correlation diagram is constructed and node information propagation and edge relationship update are carried out;Step four: the layout element relationship in layout optimization feature set and layout space limit are extracted;Step five: configuration interface fusion feature is input to improved DeepAR model and is carried out time series modeling;Step six: interface layout optimization is carried out based on interactive behavior prediction result and space constraint model;Step seven: the position of layout element is adjusted according to the long-term behavior data of each user, generates interface layout image.The improved DeepAR model of the application optimizes configuration interface layout, realizes personalization, intelligentization and space constraint self-adaptive adjustment.

Description

Technical Field

[0001] This invention relates to the field of interface optimization technology, and in particular to a method for dynamic layout optimization of configuration interfaces based on deep learning. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, user interface (UI) design has gradually shifted from traditional static layouts to dynamic and intelligent layouts. Traditional UI design methods rely heavily on manual rules and templates to arrange the position and layout of interface elements, often depending on the designer's experience and intuition to optimize the interface. While these traditional methods can meet basic needs in some applications, they often have significant shortcomings in complex and dynamic interactive environments. First, traditional UI design methods often fail to fully consider the personalized needs of users; the position and size of layout elements are often based on general design principles, failing to accurately adapt to the usage habits of different users. Second, traditional methods have poor adaptability to different devices and display environments; the arrangement, size, and position of interface elements are difficult to dynamically adjust according to the user's real-time needs, resulting in low user interaction efficiency. To overcome the shortcomings of traditional methods, dynamic UI layout optimization methods based on artificial intelligence and deep learning have gradually attracted attention in recent years. These methods, by combining big data analysis, deep learning models, and other technologies, can automatically adjust the interface layout to a certain extent based on user behavior and interaction patterns, thereby achieving personalized and adaptive optimization. However, existing deep learning-based UI layout optimization methods still face some challenges. First, many existing methods focus only on a single data source or a single aspect of information, such as relying solely on user behavior data or device status data, ignoring the interrelationships between multiple data sources. Second, most existing methods neglect the physical space constraints and interaction constraints of the interface layout, resulting in layouts that, while meeting certain behavioral requirements, suffer from visually and spatially unreasonable designs, negatively impacting the user experience.

[0003] Furthermore, despite significant advancements in temporal modeling and behavior prediction using deep learning techniques, existing interface layout optimization methods still suffer from inaccurate predictions of user interaction behavior and inadequate spatial constraints. Most methods fail to adequately incorporate real-time user feedback and behavioral patterns, leading to prediction failures in certain complex interaction scenarios.

[0004] Therefore, how to provide a method for dynamic layout optimization of configuration interfaces based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a deep learning-based dynamic layout optimization method for configuration interfaces. This invention optimizes the configuration interface layout through an improved DeepAR model. By employing personalized adjustments and perturbation correction mechanisms, it improves the accuracy of user behavior prediction, ensuring that the layout meets both user needs and physical space constraints. The layout is automatically adjusted based on user behavior data, providing personalized interface configurations, enhancing user experience and interaction efficiency, and achieving intelligent, personalized, and adaptive optimization of the interface.

[0006] A method for dynamic layout optimization of configuration interfaces based on deep learning according to an embodiment of the present invention includes the following steps: Step 1: Collect raw configuration interface data from multiple heterogeneous data sources, and standardize the raw configuration interface data to obtain a configuration interface dataset with a unified structure. Step 2: Input the configuration interface dataset into the multimodal fusion module, extract features and fuse features to obtain the configuration interface fusion features; Step 3: Construct a layout association graph by treating each interface element in the configuration interface fusion feature as a node, and perform node information propagation and edge relationship update on the layout association graph to obtain the layout optimization feature set; Step 4: Extract the relationships between layout elements and layout space constraints from the layout optimization feature set, and establish a space constraint model; Step 5: Input the fused features of the configuration interface into the improved DeepAR model, and perform time-series modeling through the temporal feature encoder, behavior pattern recognition module, disturbance correction mechanism and dynamic prediction module to obtain the interaction behavior prediction results; Step Six: Optimize the interface layout based on the interaction behavior prediction results and spatial constraint model to obtain the optimized layout decision; Step 7: Based on the optimized layout decision, adjust the position of the layout elements according to each user's long-term behavior data to generate the interface layout image.

[0007] Optionally, step one specifically includes: The configuration interface raw data is collected from multiple heterogeneous data sources, including user interaction data, device status data, interface image data, and environmental data. The device status data is real-time status information during the interface layout process, including system load, network status, and resource consumption; the interface image data includes the arrangement, position, and size information of each element on the configuration interface; the environmental data is background information related to user interaction, including user behavior patterns and background tasks. The raw data of the configuration interface is standardized, and the standardization process includes: Standardize the timestamps of user interaction data and scale device status data proportionally until the numerical range is consistent with other data sources. Resize and standardize the colors of the interface image data; Gaussian filtering is applied to the background information in the environmental data to remove environmental noise. The standardized configuration interface raw data is cleaned and filtered to remove invalid data and outliers, and then organized into a unified format to form a configuration interface dataset with a unified structure.

[0008] Optionally, step two specifically includes: The standardized configuration interface dataset is input into the multimodal fusion module, which includes image data channels, behavior data channels, and device status data channels. The image data channel is used to process interface image data from a standardized configuration interface dataset. The image data includes the arrangement, position, and size information of each interface element in the configuration interface. After convolution and max pooling, the image feature vector is obtained. The behavioral data channel is used to process user interaction data from a standardized configuration interface dataset, including user click, swipe, drag and input interaction behaviors. It performs time-series processing, organizes the data according to the time sequence of each user interaction, and extracts features through a long short-term memory network to obtain behavioral feature vectors. The device status data channel is used to process device status data from a standardized configuration interface dataset. The device's operating status features are extracted through a fully connected neural network to obtain a device status feature vector. The image feature vector, behavior feature vector, and device status feature vector are weighted and fused according to preset weights to obtain the final configuration interface fusion feature.

[0009] Optionally, step three specifically includes: Each interface element is treated as a node in the layout association diagram. The interface elements include buttons, text boxes, menus, input boxes, and label interface components. In the layout relationship graph, nodes are connected by edges, which represent dependencies between interface elements. These dependencies include distance constraints, interaction frequency, layout constraints, and functional dependencies. The distance constraint represents the spatial distance limit between interface elements; the interaction frequency represents the interaction dependency between interface elements and determines the priority of elements in the layout. Layout constraints include the positional relationships and alignment rules between interface elements; the functional dependencies are the functional dependencies between elements in the interface. The layout association graph is processed by a graph neural network to propagate node information and update edge relationships. The graph neural network uses graph convolution operations. In each layer of graph convolution operations, nodes receive information from neighboring nodes and aggregate it with their own features. The aggregation is a weighted average of the information of neighboring nodes. The edge weights between nodes are updated according to the aggregation of node features during each layer of propagation. Through multi-layer graph convolution operations, the features of nodes are gradually fused with information from neighboring nodes until the feature information of each node can be exchanged and aggregated with the information of all neighboring nodes, thus obtaining the optimized feature representation of each node in the layout. The layout optimization feature set is obtained by aggregating the optimized feature representations of all nodes in the layout.

[0010] Optionally, step four specifically involves: Extract feature information of each layout element from the layout optimization feature set. The feature information includes the relative relationship between layout elements, interaction frequency information, positional relationship and spatial constraints. The spatial constraints are the minimum distance and maximum distance between each layout element. Based on the distance constraints between the extracted layout elements, spatial constraint model rules are established, which include: Each layout element is aligned according to its characteristic information; Each layout element is cropped according to a preset size threshold.

[0011] Optionally, the improved DeepAR model includes a temporal feature encoder, a behavior pattern recognition module, a perturbation correction mechanism, and a dynamic prediction module, specifically: The configuration interface fusion features are input to the timing feature encoder, which encodes the timing interaction data in the configuration interface fusion features and extracts the timing features of user behavior. The time-series interaction data includes each user's operation history on the interface, including click events, swipe events and input events, and is sorted according to time order. The time-series features of user behavior are extracted to generate a time-series feature vector. The time-series feature vector represents the user's interaction pattern within a certain period of time, including operation frequency, duration and interaction change trend information. The time-series feature vector is input into the behavior pattern recognition module, which identifies the user's long-term behavior patterns and short-term fluctuations based on the time-series feature vector, and identifies the user's interaction patterns in different time periods. The interaction patterns include the user's daily operation behavior patterns and task switching patterns. Detect short-term fluctuations in user behavior to obtain a set of behavioral pattern features, wherein the short-term fluctuations include sudden changes in interaction patterns or abnormal behavior; Based on the behavioral pattern feature set, a perturbation correction mechanism is used to correct the error in the user interaction behavior to obtain the corrected prediction feature set. The deviation is the behavioral data deviation caused by device problems or network latency. The corrected prediction feature set is input into the dynamic prediction module, and the user's interaction behavior over a future period is predicted using a spatial constraint model to obtain the interaction behavior prediction result, which includes: Predict the areas of the interface that users are likely to click on in the future; Predict the frequency with which users will interact with the interface in the future; Predict the order and flow of user actions on the interface within a given time period.

[0012] Optionally, step six specifically includes: The interaction behavior prediction results and the spatial constraint model are input into the layout optimization module. The interaction behavior prediction results describe the user's possible future behavior patterns, and the spatial constraint model represents the physical space constraints in the layout. The current layout state includes the actual position, size, and interaction state of the interface elements. By combining the interaction behavior prediction results and the spatial constraint model, the existing layout state is evaluated and potential layout problems are identified. Adjust the position, size, and order of UI elements based on the current layout and predicted user behavior; In each layout optimization decision, the basis for each optimization decision step is output in conjunction with a self-explanatory strategy; The layout optimization strategy is dynamically adjusted based on real-time user feedback to obtain an optimized layout decision. The real-time feedback includes operation time, accidental touches, and efficiency. The optimized layout decision includes adjusting the position, size, and order of each interface element.

[0013] Optionally, step seven specifically includes: Based on the optimization layout decision, the user's operating habits are analyzed to identify the user's preferences in the interface. The long-term behavioral data includes frequently used buttons, operation paths, and misoperation patterns. The operation path is the order of execution of tasks or operation flow of the user in the interface; the misoperation mode is the interface element that the user frequently touches or performs incorrect operations. Based on long-term user behavior data, the position, size, and order of interface elements are automatically adjusted to obtain a personalized layout feature set, which includes user-personalized layout adjustment information. Based on the optimized layout decision set and the personalized layout feature set, a variational autoencoder is used to generate an interface layout image, which includes the visual hierarchy, alignment, space occupation, and user operation habits of interface elements.

[0014] The beneficial effects of this invention are: The deep learning-based dynamic layout optimization method for configuration interfaces proposed in this invention achieves significant advantages over existing technologies in several aspects. Existing layout optimization methods typically rely on static design rules, failing to flexibly adjust dynamically according to real-time user needs, resulting in a poor user experience. This invention, however, achieves dynamic analysis and optimization of interface layout by fusing multi-source heterogeneous data and combining graph neural networks, temporal modeling, and spatial constraint models. Employing multiple data sources, including user behavior data, device status data, interface image data, and environmental data, this allows layout optimization to more comprehensively consider user habits and the relationships between interface elements, avoiding the limitations of existing methods that rely on only a single data source. Furthermore, by combining graph neural networks to propagate node information and update edge relationships in the layout association graph, the characteristics of each interface element more accurately reflect its spatial and functional dependencies with other elements, improving the rationality and visual effect of the layout.

[0015] By introducing an improved DeepAR model, the accuracy of user interaction behavior prediction is further enhanced. Existing methods often overlook biases in interaction behavior data, such as errors caused by device problems or network latency, affecting the effectiveness of layout optimization. The perturbation correction mechanism can identify and correct these biases, thereby ensuring that the generated layout can adapt to user needs to the greatest extent in any environment, improving the stability and reliability of prediction results. Combined with a spatial constraint model, it can ensure that the generated layout meets physical space requirements, avoiding overcrowding or unreasonable arrangement of interface elements, and improving the usability and aesthetics of the layout. This invention makes personalized adjustments to the layout, automatically optimizing the position, size, and order of interface elements based on long-term user behavior data, thereby achieving a personalized interface layout. Existing technologies often fail to fully consider the unique needs of each user, resulting in a lack of personalized adjustments to the interface layout, making it difficult to meet users' usage habits on different devices and in different environments. Through this invention, the system can provide a tailored layout scheme according to each user's interaction pattern, greatly improving user experience and interaction efficiency. Finally, the interface layout image generated based on optimized layout decisions and personalized adjustments can ensure a balance between the visual and functional aspects of the interface, meeting the personalized needs of different users, while enhancing the adaptability and flexibility of the interface. The implementation of this method not only makes the interface layout more intelligent and personalized, but also greatly improves user operating efficiency and user satisfaction. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0017] Figure 1 This is an overall flowchart of a deep learning-based dynamic layout optimization method for configuration interfaces proposed in this invention. Figure 2 This is a schematic diagram illustrating the steps of generating the layout optimization feature set in a deep learning-based dynamic layout optimization method for configuration interfaces proposed in this invention. Figure 3 This is a flowchart of the improved DeepAR model processing procedure for a configuration interface dynamic layout optimization method based on deep learning proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figure 1-3 A deep learning-based method for dynamic layout optimization of configuration interfaces includes the following steps: Step 1: Collect raw configuration interface data from multiple heterogeneous data sources, and standardize the raw configuration interface data to obtain a configuration interface dataset with a unified structure. Step 2: Input the configuration interface dataset into the multimodal fusion module, extract features and fuse features to obtain the configuration interface fusion features; Step 3: Construct a layout association graph by treating each interface element in the configuration interface fusion feature as a node, and perform node information propagation and edge relationship update on the layout association graph to obtain the layout optimization feature set; Step 4: Extract the relationships between layout elements and layout space constraints from the layout optimization feature set, and establish a space constraint model; Step 5: Input the fused features of the configuration interface into the improved DeepAR model, and perform time-series modeling through the temporal feature encoder, behavior pattern recognition module, disturbance correction mechanism and dynamic prediction module to obtain the interaction behavior prediction results; Step Six: Optimize the interface layout based on the interaction behavior prediction results and spatial constraint model to obtain the optimized layout decision; Step 7: Based on the optimized layout decision, adjust the position of the layout elements according to each user's long-term behavior data to generate the interface layout image.

[0020] In this embodiment, step one specifically includes: Raw configuration interface data is collected from multiple heterogeneous data sources. This raw configuration interface data includes user interaction data, device status data, interface image data, and environmental data. User interaction data refers to various behavioral data generated when users interact with interface elements, such as clicks, swipes, drags, and input. This data reflects users' operating habits, preferences, and frequency on the interface, and is key to optimizing interface layout and improving user experience. Device status data refers to the real-time system performance and resource usage during interface layout, including system load, network status, and resource consumption. Device status data is crucial for understanding current hardware conditions and optimizing resource allocation, ensuring effective layout optimization across different devices and environments. Interface image data includes the arrangement, position, and size information of various elements on the configuration interface. This data helps in visual analysis and optimization of the interface layout, ensuring that the interface design meets users' visual needs. Environmental data is background information related to user interaction, including user behavior patterns and background tasks. This information helps analyze changes in user needs in different contexts and allows for corresponding adjustments to the interface layout.

[0021] The raw data from the configuration interface is standardized to ensure format consistency across different data sources, facilitating subsequent processing and analysis. The purpose of standardization is to eliminate differences and noise in the raw data, enabling effective fusion and computation of various data types. Specifically, standardization includes the following steps:

[0022] Standardizing the timestamps of user interaction data ensures consistency across all interaction records on the timeline, preventing data misalignment caused by different timestamps and guaranteeing data temporality and consistency. This step ensures that various types of interaction data can be compared and modeled within the same time frame in time series analysis.

[0023] The device status data is scaled proportionally until its numerical range aligns with other data sources. Since the numerical range of device status data can differ significantly from other data sources (such as behavioral and image data), directly fusing it with other data can lead to computational biases. Scaling adjusts all data to the same range, ensuring that all data types have equal weight in subsequent processing.

[0024] Resizing and color normalizing the interface image data ensures that its size and color are consistent with other input data. Interface image data often exhibits significant size variations and color variations, which can lead to inconsistencies during subsequent model processing. Resizing and color normalization eliminate these inconsistencies, allowing the image data to be seamlessly integrated into other data sources.

[0025] To remove background noise from environmental data, Gaussian filtering is applied. Environmental data may contain external interference or irrelevant information; if left unprocessed, this can affect the accuracy of subsequent analysis. Gaussian filtering is an effective noise reduction technique that smooths the data, removes unnecessary noise, retains valuable information, and further improves data quality.

[0026] The standardized configuration interface raw data undergoes cleaning and filtering to remove invalid data and outliers. The purpose of cleaning and filtering is to ensure the accuracy and completeness of the dataset used, avoiding the impact of outliers or incomplete data on subsequent model training and analysis. The cleaned data will be organized into a unified format, forming a configuration interface dataset with a consistent structure, ensuring the efficiency and accuracy of subsequent data processing.

[0027] Through this series of standardized processing steps, the final generated configuration interface dataset will have a unified data structure and format, providing a reliable foundation for subsequent layout optimization, feature extraction, and model training.

[0028] In this embodiment, step two specifically includes: The standardized configuration interface dataset is input into a multimodal fusion module, which includes image data channels, behavior data channels, and device status data channels. Each channel processes different types of data, aiming to extract and fuse multi-dimensional features to generate a comprehensive feature representation of the configuration interface.

[0029] The image data channel processes interface image data from a standardized configuration interface dataset. This image data includes information on the arrangement, position, and size of various interface elements. This information is crucial for understanding the interface's layout structure and the relationships between elements. The image data channel extracts spatial features of interface elements through convolution operations, such as the relative positions and layout density of elements. Simultaneously, max pooling is used to reduce feature dimensionality, making the network more efficient during learning. After these operations, the image channel outputs image feature vectors, which represent the abstract features of the image elements in the interface and serve as the basis for subsequent feature fusion.

[0030] The behavioral data channel processes user interaction data from a standardized configuration interface dataset, including user clicks, swipes, drags, and inputs. This user interaction data has time-series characteristics, reflecting the user's operation sequence, habits, and frequency within the interface. This channel performs time-series processing on the interaction data, sorting the data according to the chronological order of each user interaction to ensure that subsequent modeling can capture the temporal patterns of the interaction behavior. Next, the behavioral data channel uses a Long Short-Term Memory (LSTM) network for feature extraction. LSTM effectively learns long-term dependencies and short-term changes in time-series data, outputting a behavioral feature vector containing key information such as the user's interaction patterns and behavioral preferences.

[0031] The device status data channel processes device status data from a standardized configuration interface dataset. This data includes real-time device status information such as system load, network status, and resource consumption. This information helps understand the hardware and software conditions under which the interface is currently operating, influencing the optimization and adjustment of the interface layout. The device status data channel processes the device status data using a fully connected neural network (FCNN) to extract the device's operating status features, resulting in a device status feature vector. This feature vector characterizes the current operating status of the device and how these conditions affect interface layout optimization.

[0032] The image feature vector, behavior feature vector, and device status feature vector are weighted and fused according to preset weights. The purpose of weighted fusion is to assign different importance to features extracted from different data channels, thus integrating the feature information from each channel. The fused feature representation is more comprehensive, encompassing interface visual information, user interaction information, and device status information, thereby providing strong data support for subsequent interface layout optimization. Finally, the obtained "configuration interface fusion feature" serves as input to the optimization model, driving more precise and personalized layout adjustments.

[0033] In this embodiment, step three specifically includes: Each UI element is treated as a node in the layout relationship graph. UI elements include UI components such as buttons, text boxes, menus, input boxes, and labels. Each node represents an element in the UI, and nodes are connected by edges, representing the dependencies between UI elements. Establishing these dependencies reflects the interactions and constraints between UI elements, which is crucial for optimizing the layout's rationality. These dependencies include the following types:

[0034] Distance constraints: Spatial distance limitations between interface elements. Some interface elements may need to maintain a certain minimum distance to avoid overcrowding or visual clutter. For example, the distance between two buttons should be sufficient so that users can clearly identify and operate them.

[0035] Interaction frequency: This indicates the interaction dependencies between UI elements. Some elements are used more frequently and may need to be placed in easily accessible locations to improve user convenience. Interaction frequency can be determined by analyzing user behavior data, thereby determining the priority of elements in the layout.

[0036] Layout constraints: These include the positional relationships and alignment rules between interface elements. Some interface elements may require alignment according to certain rules, such as the vertical alignment of text boxes and buttons, or consistent spacing between multiple input boxes. These layout constraints ensure a neat and aesthetically pleasing interface while improving user efficiency.

[0037] Functional dependencies refer to the functional relationships between elements in a user interface. For example, the relationship between an input field and a submit button: after the user completes their input, they click the button to submit. Functional dependencies help determine the logical order of interface elements and the flow of operations.

[0038] Graph Neural Networks (GNNs) are used to propagate node information and update edge relationships in the layout association graph. GNNs effectively capture the relationships between nodes and update node features through graph convolution operations. In each layer of graph convolution operations, each node receives information from its neighboring nodes and aggregates it with its own features; this aggregation is a weighted average of the information from its neighbors. The edge weights between nodes are updated based on the aggregation of node features during each layer of propagation to reflect the strength of the relationships and mutual influence between nodes. After each information propagation, the node features become richer, better reflecting the actual situation in the interface layout.

[0039] Through multi-layer graph convolution operations, node features gradually fuse information from more neighboring nodes until the feature information of each node can be exchanged and aggregated with the information of all neighboring nodes. In this way, the interrelationships and dependencies between interface elements can be more fully expressed through multiple propagations of the graph neural network, thereby obtaining an optimized feature representation of each node in the layout.

[0040] Finally, by aggregating the optimized feature representations of all nodes in the layout, a "layout optimization feature set" is obtained. This feature set integrates the spatial layout information, interaction dependencies, functional relationships, and visual constraints of interface elements, providing comprehensive feature support for subsequent layout optimization. In this way, graph neural networks can fully consider the mutual influence of various elements in the layout, generating optimized layout features that meet user needs and interface design principles.

[0041] In this embodiment, step four specifically includes: Feature information for each layout element is extracted from the layout optimization feature set. This feature information includes the relative relationships between layout elements, interaction frequency information, positional relationships, and spatial constraints. The feature information of each layout element reflects its function in the interface, the frequency of user interaction, and the spatial and functional dependencies between elements. This feature information helps provide necessary data support for optimizing the interface layout, thereby improving the user experience while ensuring layout rationality.

[0042] The relative relationships between layout elements: This reflects the spatial relationships and layout arrangement of interface elements. For example, is a button adjacent to a text box, or is it necessary to maintain a certain amount of space in the layout to avoid the interface becoming too crowded? Relative relationships help optimize the arrangement of interface elements, ensuring a neat interface that conforms to user habits.

[0043] Interaction frequency information: This indicates how often users interact with each interface element, helping to identify the most frequently used functional elements. These high-interaction-frequency elements should typically be placed in more easily accessible locations, while low-interaction-frequency elements can be arranged in less conspicuous locations to avoid taking up too much space.

[0044] Positional relationships: This refers to the order and position of UI elements relative to other elements. For example, some UI elements may need to maintain a fixed position on the screen, while other elements can be dynamically adjusted as needed. Positional relationships ensure that the layout meets functional requirements while avoiding a cluttered interface.

[0045] Spatial constraints describe the spatial limitations between interface elements, specifically including the minimum and maximum distances between each layout element. Minimum distances ensure that interface elements do not overlap or become too crowded, while maximum distances prevent excessive space between elements, thus wasting screen space. Spatial constraints are fundamental to achieving a reasonable layout and a good user experience.

[0046] Based on the distance constraints extracted between layout elements, spatial constraint model rules are established. This spatial constraint model effectively controls the distance relationships between interface elements during layout optimization, preventing the generation of layouts that do not meet physical space requirements. Specific rules include:

[0047] Each layout element is aligned according to its characteristic information: by analyzing the characteristic information of each interface element, the elements in the layout are ensured to be aligned according to certain rules. For example, text boxes and buttons may need to be aligned vertically, or a group of buttons may need to be aligned horizontally. Alignment rules ensure that the relative positions of interface elements are neat and conform to design principles.

[0048] Each layout element is cropped according to a preset size threshold: the size of interface elements is crucial in interface design, especially within limited screen space. To prevent some elements from taking up too much space or being too small and affecting operability, each layout element is cropped according to a preset size threshold. Size cropping ensures that all elements are within a reasonable range, not only conforming to visual aesthetics but also improving the efficiency of user interaction.

[0049] Through the above steps, the extracted layout element features will help establish a spatial constraint model and be applied in the subsequent layout optimization process, ensuring that the layout design not only conforms to physical constraints but also meets the user's interaction needs.

[0050] In this embodiment, the improved DeepAR model includes a temporal feature encoder, a behavior pattern recognition module, a perturbation correction mechanism, and a dynamic prediction module, specifically: The configuration interface fusion features are input to the timing feature encoder, which encodes the timing interaction data in the configuration interface fusion features and extracts the timing features of user behavior. The time-series interaction data includes each user's operation history on the interface, including click events, swipe events and input events, and is sorted according to time order. The time-series features of user behavior are extracted to generate a time-series feature vector. The time-series feature vector represents the user's interaction pattern within a certain period of time, including operation frequency, duration and interaction change trend information. The time-series feature vector is input into the behavior pattern recognition module, which identifies the user's long-term behavior patterns and short-term fluctuations based on the time-series feature vector, and identifies the user's interaction patterns in different time periods. The interaction patterns include the user's daily operation behavior patterns and task switching patterns. Detect short-term fluctuations in user behavior to obtain a set of behavioral pattern features, wherein the short-term fluctuations include sudden changes in interaction patterns or abnormal behavior; Based on the behavioral pattern feature set, a perturbation correction mechanism is used to correct the error in the user interaction behavior to obtain the corrected prediction feature set. The deviation is the behavioral data deviation caused by device problems or network latency. The corrected prediction feature set is input into the dynamic prediction module, and the user's interaction behavior over a future period is predicted using a spatial constraint model to obtain the interaction behavior prediction result, which includes: Predict the areas of the interface that users are likely to click on in the future; Predict the frequency with which users will interact with the interface in the future; Predict the order and flow of user actions on the interface within a given time period.

[0051] This step enhances the accuracy and adaptability of configuration interface layout optimization through an improved DeepAR model. The improved model, combining a temporal feature encoder, a behavior pattern recognition module, a perturbation correction mechanism, and a dynamic prediction module, accurately captures long-term and short-term user interaction patterns, effectively predicting future user interactions. By correcting for deviations in user behavior, this method overcomes data errors caused by device issues or network latency, further improving prediction accuracy. The dynamic prediction module, combined with a spatial constraint model, ensures that the prediction results not only meet user interaction needs but also adapt to the spatial limitations of different interfaces. Furthermore, the model can identify user interaction patterns, including click areas, frequencies, and operation sequences, providing more refined user behavior data for subsequent layout optimization, thereby enhancing interface personalization and flexibility and improving user experience.

[0052] In this embodiment, step six specifically includes: The interaction behavior prediction results and the spatial constraint model are input into the layout optimization module. The interaction behavior prediction results describe the user's possible future behavior patterns, and the spatial constraint model represents the physical space constraints in the layout. The interaction behavior prediction results are the interactive behaviors that the user may perform in the future, including clicks, swipes, and input, predicted by a deep learning model. The spatial constraint model considers the relative position, size, spacing, and alignment rules of interface elements, ensuring that the layout generation follows the actual physical space constraints and avoids overlapping interface elements or occupying too much screen space.

[0053] The current layout state includes the actual position, size, and interaction state of interface elements. It refers to the actual position, size, arrangement order, and interaction state of all interface elements within a given interface, such as whether currently displayed elements are interactive or have been interacted with by the user. By combining interaction behavior prediction results and spatial constraint models, the existing layout state is evaluated to identify potential layout problems. For example, some interface elements may be too crowded, leading to inconvenient interaction, or some elements may be poorly positioned, causing frequent user errors. The evaluation process can uncover these potential problems, providing a basis for subsequent layout adjustments.

[0054] Based on the current layout and predicted user behavior, the position, size, and order of interface elements are adjusted. The layout optimization module uses optimization algorithms to rationally adjust the interface elements according to the evaluation results. By adjusting the position, size, and arrangement of interface elements, the layout not only improves user efficiency but also enhances visual appeal. For example, frequently used function buttons can be moved to easily accessible locations, while less frequently used functions can be placed in more concealed positions to avoid a cluttered interface.

[0055] Each layout optimization decision is accompanied by a self-explanatory strategy, providing the rationale behind each step. This self-explanatory strategy ensures transparency in the optimization process, making every layout adjustment decision understandable to users and developers. For example, if the order or position of a button changes, the system can clearly explain why this decision was made—perhaps based on predicted user behavior data, where the button's position on the interface is crucial for improving interaction efficiency. This self-explanatory nature helps users understand the logic behind the layout optimization, increasing user trust.

[0056] The layout optimization strategy is dynamically adjusted based on real-time user feedback to arrive at an optimized layout decision. This feedback includes information such as operation time, accidental touches, and efficiency. For example, if the system detects frequent accidental touches in a certain area, it may adjust the size, spacing, or position of interface elements based on this feedback. The validity of user feedback directly influences the adjustment of the layout optimization strategy, enabling the optimization process to better adapt to user needs and preferences, and gradually improve the user experience.

[0057] The optimized layout decisions include adjusting the position, size, and order of each interface element. The final optimized layout decision generates a set of explicit adjustment instructions, specifying the new position, size, and arrangement order of each interface element. These adjustments can reflect user interaction habits and feedback in real time, and each optimization results in an interface layout that better meets user needs, further improving interface adaptability and user satisfaction.

[0058] In this embodiment, step seven specifically includes: Based on optimized layout decisions, long-term behavioral data analysis of each user reveals their operational habits and preferences within the interface. This long-term behavioral data includes key information such as frequently used buttons, operation paths, and error operation patterns. This data reflects user habits and preferences during interface usage, helping the system provide a more personalized layout for each user. Frequently used buttons refer to buttons or functional areas that users click or operate on frequently; these buttons typically need to be placed in easily accessible locations. Operation paths refer to the order in which users execute tasks or the flow of operations within the interface, usually reflecting their common workflows or functional usage habits. The system can optimize the interface based on this information to reduce user operation steps and time. Error operation patterns refer to interface elements that users frequently accidentally touch or misoperate during operation. These elements may be due to factors such as being too small or inappropriately positioned, leading to frequent misoperations. The system will make targeted adjustments based on this information to prevent unnecessary operational errors caused by unreasonable interface design.

[0059] Based on long-term user behavior data, the system can automatically adjust the position, size, and order of interface elements to meet users' personalized needs. Through analysis of this data, the system can not only identify user behavioral preferences but also predict future user operational needs, thereby dynamically adjusting the interface layout to better suit user interaction habits and requirements. This process generates a "personalized layout feature set" based on personalized user operation data. This feature set contains personalized layout adjustment information, such as the position, size, and arrangement of frequently used functional elements. The personalized layout feature set helps to directly integrate user operating habits into the layout design, improving interaction efficiency and user satisfaction.

[0060] Based on the optimized layout decision set and the personalized layout feature set, the system further uses a variational autoencoder to generate an interface layout image. A variational autoencoder is a generative model that maps input data to a latent space through an encoder, and then generates an image through a decoder. In this invention, the variational autoencoder utilizes information from the "optimized layout decision set" and the "personalized layout feature set" to generate an interface layout image that meets user needs. The generated interface layout image must not only conform to visual hierarchy and alignment specifications, but also ensure reasonable space occupancy between interface elements, so that the entire interface not only conforms to aesthetic design principles but also maximizes the user's interaction needs.

[0061] The generated interface layout image incorporates multi-dimensional features such as the visual hierarchy of interface elements, alignment, space occupation, and user operating habits. Visual hierarchy ensures that important interactive elements are highlighted, facilitating quick user identification and operation; alignment ensures neat arrangement of interface elements, enhancing the overall aesthetics; space occupation considers screen size and user operating habits, allocating space rationally to avoid overly crowded or wasteful layouts; and personalized layout images generated based on user operating habits make the interface layout more aligned with user preferences, greatly improving user experience and interaction efficiency.

[0062] Example 1: To verify the feasibility of this invention in practice, it was applied to a modern industrial control system. In this system, operators need to monitor and control multiple devices and processes on a complex interface. Traditional control interface designs are typically fixed and do not dynamically adjust their layout based on operator behavior. This fixed interface design often leads to operator fatigue after long hours of work, as operators need to constantly search for and click on various controls to complete tasks. Especially in multi-tasking environments, an unreasonable layout of interface elements can cause frequent misoperations, thus affecting production efficiency and operator experience. A deep learning-based dynamic layout optimization method for configuration interfaces can effectively address the shortcomings of traditional interface designs. Assuming an industrial control system with multiple monitoring devices and complex control items, operators need to view the status of different devices in real time and perform control. To improve operational efficiency and accuracy, raw configuration interface data from multiple heterogeneous data sources was collected and input into a deep learning model for dynamic layout optimization.

[0063] The data acquisition module collected interaction data between the operator and the interface, including clicks, swipes, drags, and other interactive behaviors; device status data (such as system load, network status, and resource consumption); and interface image data and environmental data (such as operator behavior patterns and background tasks). This data accurately reflects the operator's interaction habits and common operation paths on the interface. For example, analysis of the interaction data revealed that certain device monitoring buttons were frequently clicked, while control buttons on other devices were ignored. Device status data indicated that when the system load was high, the response time of some interface elements slowed down, resulting in a poor user experience. Image data provided information on the position, arrangement, and size of interface elements, helping to understand the visual effect of the interface layout and the relative positions of elements. This data underwent standardization. Standardized processing of the interaction data timestamps ensured synchronization between different data sources. Device status data was scaled proportionally to match the numerical range of behavioral and image data, thus avoiding processing errors caused by numerical differences. Interface image data underwent size adjustment and color standardization, enabling better integration with other data sources. Meanwhile, the environmental data was processed using Gaussian filtering to remove noise and ensure data accuracy.

[0064] After standardization, the data is input into a multimodal fusion module for feature extraction from image data, behavioral data, and device status data. Through the image data channel, spatial features such as the arrangement, position, and size of interface elements are extracted; through the behavioral data channel, user interaction features, such as frequently clicked locations and operation paths, are extracted; and through the device status data channel, device operating status features, such as load and resource consumption, are extracted. These features are then weighted and fused to obtain the fused features of the configuration interface. A layout association graph is constructed using a graph neural network, with each interface element as a node and edges representing dependencies between elements, such as distance constraints, interaction frequency, layout constraints, and functional dependencies. Graph convolution operations propagate node information and update edge relationships, ultimately yielding an optimized feature representation for each interface element. These optimized feature representations include spatial layout information, interaction requirements, and functional relationships of interface elements, helping to guide subsequent layout optimization decisions.

[0065] By extracting the layout optimization feature set, a spatial constraint model was further established, considering the minimum and maximum distances between interface elements to ensure that the layout of interface elements meets physical space requirements and avoids overcrowding or wasted space. Based on this, layout optimization decisions were made by combining real-time user feedback and predicted interaction behavior. By monitoring operator interaction behavior, misoperation instances, and operational efficiency in real time, the position, size, and order of interface elements were continuously adjusted to ensure that the layout always conforms to operator usage habits. To further optimize layout decisions, long-term operator behavior data was analyzed to identify features such as frequently used buttons, operation paths, and misoperation patterns. It was found that monitoring buttons on certain devices were frequently clicked, while control buttons on other devices were rarely used. Through personalized layout adjustments based on long-term behavior data, the positions of frequently used buttons were automatically adjusted to be closer to the operator's work area to improve interaction efficiency. Based on the optimized layout decisions and personalized layout feature set, a variational autoencoder (VAE) was used to generate the final interface layout image. The generated interface layout image considered the visual hierarchy, alignment, space occupation, and operator operating habits of interface elements. The final layout image is not only visually neat and beautiful, but also fully considers the user's personalized needs, greatly improving the operator's work efficiency and user experience.

[0066] Table 1 Comparison of Interaction Efficiency Before and After Optimization

[0067] As shown in Table 1, before optimization, operators spent a long time on the interface, had a high frequency of accidental touches, and low interaction efficiency. After optimization using the method of this invention, operator operation time decreased by 30%, accidental touch frequency decreased by 80%, and interaction efficiency improved by 20%. These results demonstrate that the deep learning-based dynamic layout optimization method for configuration interfaces can significantly improve user interaction efficiency, reduce misoperations, and enhance the overall user experience.

[0068] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for dynamic layout optimization of configuration interfaces based on deep learning, characterized in that, Includes the following steps: Step 1: Collect raw configuration interface data from multiple heterogeneous data sources, and standardize the raw configuration interface data to obtain a configuration interface dataset with a unified structure. Step 2: Input the configuration interface dataset into the multimodal fusion module, extract features and fuse features to obtain the configuration interface fusion features; Step 3: Construct a layout association graph by treating each interface element in the configuration interface fusion feature as a node, and perform node information propagation and edge relationship update on the layout association graph to obtain the layout optimization feature set; Step 4: Extract the relationships between layout elements and layout space constraints from the layout optimization feature set, and establish a space constraint model; Step 5: Input the fused features of the configuration interface into the improved DeepAR model, and perform time-series modeling through the temporal feature encoder, behavior pattern recognition module, disturbance correction mechanism and dynamic prediction module to obtain the interaction behavior prediction results; The improved DeepAR model includes a temporal feature encoder, a behavior pattern recognition module, a perturbation correction mechanism, and a dynamic prediction module, specifically: The configuration interface fusion features are input to the timing feature encoder, which encodes the timing interaction data in the configuration interface fusion features and extracts the timing features of user behavior. The time-series interaction data includes each user's operation history on the interface, including click events and swipe events, and is sorted according to time order. The time-series features of user behavior are extracted to generate a time-series feature vector. The time-series feature vector represents the user's interaction pattern within a certain period of time, including operation frequency, duration and interaction change trend information. The time-series feature vector is input into the behavior pattern recognition module, which identifies the user's long-term behavior patterns and short-term fluctuations based on the time-series feature vector, and identifies the user's interaction patterns in different time periods. The interaction patterns include the user's daily operation behavior patterns and task switching patterns. Detect short-term fluctuations in user behavior to obtain a set of behavioral pattern features, wherein the short-term fluctuations include sudden changes in interaction patterns or abnormal behavior; Based on the behavioral pattern feature set, a perturbation correction mechanism is used to correct the error in the user interaction behavior to obtain the corrected prediction feature set. The deviation is the behavioral data deviation caused by device problems or network latency. The corrected prediction feature set is input into the dynamic prediction module, and the user's interaction behavior over a future period is predicted using a spatial constraint model to obtain the interaction behavior prediction result, which includes: Predict the areas of the interface that users are likely to click on in the future; Predict the frequency with which users will interact with the interface in the future; Predict the order and flow of user actions on the interface within a given time period; Step Six: Optimize the interface layout based on the interaction behavior prediction results and spatial constraint model to obtain the optimized layout decision; Step 7: Based on the optimized layout decision, adjust the position of the layout elements according to each user's long-term behavior data to generate the interface layout image.

2. The method for dynamic layout optimization of configuration interfaces based on deep learning according to claim 1, characterized in that, Step one specifically involves: The configuration interface raw data is collected from multiple heterogeneous data sources, including user interaction data, device status data, interface image data, and environmental data. The device status data is real-time status information during the interface layout process, including system load, network status, and resource consumption; the interface image data includes the arrangement, position, and size information of each element on the configuration interface; the environmental data is background information related to user interaction, including user behavior patterns and background tasks. The raw data of the configuration interface is standardized, and the standardization process includes: Standardize the timestamps of user interaction data and scale device status data proportionally until the numerical range is consistent with other data sources. Resize and standardize the colors of the interface image data; Gaussian filtering is applied to the background information in the environmental data to remove environmental noise. The standardized configuration interface raw data is cleaned and filtered to remove invalid data and outliers, and then organized into a unified format to form a configuration interface dataset with a unified structure.

3. The method for dynamic layout optimization of configuration interfaces based on deep learning according to claim 1, characterized in that, Step two specifically involves: The standardized configuration interface dataset is input into the multimodal fusion module, which includes image data channels, behavior data channels, and device status data channels. The image data channel is used to process interface image data from a standardized configuration interface dataset. The image data includes the arrangement, position, and size information of each interface element in the configuration interface. After convolution and max pooling, the image feature vector is obtained. The behavioral data channel is used to process user interaction data from a standardized configuration interface dataset, including user clicks, swipes, and drags. It performs time-series processing, organizes the data according to the time sequence of each user interaction, and extracts features through a long short-term memory network to obtain behavioral feature vectors. The device status data channel is used to process device status data from a standardized configuration interface dataset. The device's operating status features are extracted through a fully connected neural network to obtain a device status feature vector. The image feature vector, behavior feature vector, and device status feature vector are weighted and fused according to preset weights to obtain the final configuration interface fusion feature.

4. The method for dynamic layout optimization of configuration interfaces based on deep learning according to claim 1, characterized in that, Step three specifically involves: Each interface element is treated as a node in the layout relationship diagram, and the interface elements include buttons, text boxes, menus, and labels; In the layout relationship graph, nodes are connected by edges, which represent dependencies between interface elements. These dependencies include distance constraints, interaction frequency, layout constraints, and functional dependencies. The distance constraint represents the spatial distance limit between interface elements; the interaction frequency represents the interaction dependency between interface elements and determines the priority of elements in the layout. Layout constraints include the positional relationships and alignment rules between interface elements; the functional dependencies are the functional dependencies between elements in the interface. The layout association graph is processed by a graph neural network to propagate node information and update edge relationships. The graph neural network uses graph convolution operations. In each layer of graph convolution operations, nodes receive information from neighboring nodes and aggregate it with their own features. The aggregation is a weighted average of the information of neighboring nodes. The edge weights between nodes are updated according to the aggregation of node features during each layer of propagation. Through multi-layer graph convolution operations, the features of nodes are gradually fused with information from neighboring nodes until the feature information of each node can be exchanged and aggregated with the information of all neighboring nodes, thus obtaining the optimized feature representation of each node in the layout. The layout optimization feature set is obtained by aggregating the optimized feature representations of all nodes in the layout.

5. The method for dynamic layout optimization of configuration interfaces based on deep learning according to claim 1, characterized in that, Step four specifically involves: The feature information of each layout element is extracted from the layout optimization feature set. The feature information includes the interaction frequency information between layout elements, the positional relationship and the spatial constraints. The spatial constraints are the minimum distance and the maximum distance between each layout element. Based on the distance constraints between the extracted layout elements, spatial constraint model rules are established, which include: Each layout element is aligned according to its characteristic information; Each layout element is cropped according to a preset size threshold.

6. The method for dynamic layout optimization of configuration interfaces based on deep learning according to claim 1, characterized in that, Step six specifically involves: The interaction behavior prediction results and the spatial constraint model are input into the layout optimization module. The interaction behavior prediction results describe the user's possible future behavior patterns, and the spatial constraint model represents the physical space constraints in the layout. The current layout state includes the actual position, size, and interaction state of the interface elements. By combining the interaction behavior prediction results and the spatial constraint model, the existing layout state is evaluated and potential layout problems are identified. Adjust the position, size, and order of UI elements based on the current layout and predicted user behavior; Each time a layout optimization decision is made, the basis for each optimization decision step is output; The layout optimization strategy is dynamically adjusted based on real-time user feedback to obtain an optimized layout decision. The real-time feedback includes operation time, accidental touches, and efficiency. The optimized layout decision includes adjusting the position, size, and order of each interface element.

7. The method for dynamic layout optimization of configuration interfaces based on deep learning according to claim 1, characterized in that, Step seven specifically involves: Based on the optimization layout decision, the user's operating habits are analyzed to identify the user's preferences in the interface. The long-term behavioral data includes frequently used buttons, operation paths, and misoperation patterns. The operation path is the order of execution of tasks or operation flow of the user in the interface; the misoperation mode is the interface element that the user frequently touches or performs incorrect operations. Based on long-term user behavior data, the position, size, and order of interface elements are automatically adjusted to obtain a personalized layout feature set, which includes user-personalized layout adjustment information. Based on the optimized layout decision set and the personalized layout feature set, a variational autoencoder is used to generate an interface layout image, which includes the visual hierarchy, alignment, space occupation, and user operation habits of interface elements.

Images