An adaptive interface adjustment method and adjustment system for human-computer interaction
Through the adaptive interface adjustment method, the layout and behavior prediction modules are used to optimize interface layout and element classification, which solves the problem that traditional interface design cannot adapt to user needs, realizes dynamic adjustment and continuous optimization of the interface, and improves the user experience and interface self-interface Adaptability.
Patent Information
- Application Number
- CN202411131348.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-17
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2044-08-17
AI Technical Summary
Traditional human-computer interaction interface design cannot be flexibly adjusted according to user specific needs, usage habits or changes in the device environment, which limits the improvement of user experience and the full use of system functions.
Adaptive interface adjustment method is adopted, and the pre-trained layout prediction model and behavior prediction module are combined with the classification module based on rule matching to realize accurate prediction of interface layout, fine classification and flexible adjustment of elements, and interface optimization is carried out according to user behavior.
It significantly improves the adaptability and user experience of the human-computer interactive interface, ensuring the optimal usability and ease of use of the interface in different devices and environments.
Smart Images
Figure CN118656076B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to an adaptive interface adjustment method and an adjustment system applied to human-computer interaction. Background Art
[0002] With the rapid development of information technology, the design and experience quality of the human-computer interaction interface (HCI) as a medium for information exchange between users and computer systems is directly related to user satisfaction and the overall effectiveness of the system. Traditional human-computer interaction interface design often adopts a static layout and cannot be flexibly adjusted according to the specific needs of users, usage habits or changes in the device environment, which greatly limits the improvement of user experience and the full play of system functions.
[0003] Therefore, it is necessary to provide an adaptive interface adjustment method and adjustment system for human-computer interaction to solve the above technical problems. Summary of the invention
[0004] In order to solve the above technical problems, the present invention provides an adaptive interface adjustment method and adjustment system for human-computer interaction, which realizes accurate prediction of interface layout, fine classification of elements, flexible adjustment strategy and interface optimization based on user behavior, thereby significantly improving the adaptability and user experience of the human-computer interaction interface.
[0005] The present invention provides an adaptive interface adjustment method for human-computer interaction, the adjustment method comprising the following steps:
[0006] Based on the device attributes of the device to which the target interface belongs as input, a pre-trained layout prediction model is used to perform layout prediction and output the optimal layout of the device to which the target interface belongs;
[0007] Use the classification module based on rule matching to classify the elements in the human-computer interaction interface and generate classification labels for the elements;
[0008] According to the classification labels of the elements, the elements in the human-computer interaction interface are adjusted using an adjustment strategy pre-matched with the classification labels, and the adjusted elements are deployed in the target interface according to the optimal layout to obtain an initial interface;
[0009] Taking the initial interface as input, performing behavior prediction through a pre-trained behavior prediction module and outputting the user behavior of the device to which the target interface belongs;
[0010] Based on the user behavior, the elements in the initial interface are optimized to complete the interface adaptive adjustment.
[0011] Preferably, the method of taking the device attributes of the device to which the target interface belongs as input, performing layout prediction through a pre-trained layout prediction model and outputting the optimal layout of the device to which the target interface belongs includes:
[0012] Reading device attributes of the device to which the target interface belongs, and converting the device attributes into a feature vector, wherein the device attributes include device type and screen size;
[0013] The feature vector of the device attribute is used as the input of the layout prediction model, and the prediction function is called to perform inference calculation to output the optimal layout of the device to which the target interface belongs.
[0014] Preferably, the training process of the layout prediction model includes:
[0015] Collecting and preprocessing an interface layout instance data set, wherein the interface layout instance includes marked sample device attributes and sample layouts;
[0016] The deep learning-based layout prediction model is trained using the training set in the preprocessed interface layout instance dataset, and the model parameters are iteratively adjusted to minimize the prediction error.
[0017] Based on the evaluation indicators, the performance is evaluated using the validation set in the preprocessed interface layout instance dataset, and the model parameters are adjusted according to the evaluation results, finally obtaining a trained layout prediction model.
[0018] Preferably, the method of using a classification module based on rule matching to classify elements in the human-computer interaction interface and generate classification labels for the elements includes:
[0019] Using an OCR recognition method to identify elements of the human-computer interaction interface, and extracting features of the identified elements, wherein the extracted features include visual features, geometric features, and text features;
[0020] Comparing the extracted features with the rules in the rule base of the classification module for similarity, and selecting the rule with the highest similarity score, wherein the rule base contains feature descriptions and classification criteria for various element types;
[0021] According to the selected rules, a classification label is assigned to each element in the human-computer interaction interface, wherein the classification label includes buttons, texts, pictures and input boxes.
[0022] Preferably, the adjusting the elements in the human-computer interaction interface according to the classification labels of the elements using an adjustment strategy pre-matched with the classification labels, and deploying the adjusted elements to the target interface according to the optimal layout to obtain the initial interface includes:
[0023] Use the predefined adjustment strategy to match the classification labels of the elements to obtain the adjustment rules for each element;
[0024] Modify the attributes of the element using corresponding adjustment rules, wherein the attributes of the element include position, size and color;
[0025] The elements with modified attributes are readjusted according to the optimal layout, and the adjusted elements are optimized through the layout management algorithm to obtain the initial interface.
[0026] Preferably, the method of taking the initial interface as input, performing behavior prediction through a pre-trained behavior prediction module and outputting the user behavior of the device to which the target interface belongs includes:
[0027] The feature extraction method is used to extract the features of the initial interface to obtain the interface features;
[0028] The interface features are used as input, behavior prediction is performed using a pre-trained behavior prediction model, and user behavior of the device to which the target interface belongs is output, wherein the behavior prediction model is trained based on historical interfaces and historical user behavior data.
[0029] Preferably, the user behavior includes click frequency, sliding direction, dwell time and operation path.
[0030] Preferably, the optimizing process of the elements in the initial interface based on the user behavior to complete the interface adaptive adjustment includes:
[0031] Adjusting the weights of elements in the initial interface according to user behavior, wherein the weight adjustment includes increasing button size, changing element position, and adjusting font size;
[0032] Performing personalized adjustments to the elements in the initial interface according to user behavior, wherein the personalized adjustments include adjusting color themes, font styles, and making function entrances more obvious;
[0033] The initial interface is adjusted in layout according to user behavior, wherein the layout adjustment includes adjusting element spacing, rearranging element order, and optimizing navigation structure.
[0034] The present invention also provides an adaptive interface adjustment system for human-computer interaction, which is used to execute the adaptive interface adjustment method for human-computer interaction. The adjustment system includes:
[0035] A device adaptation module, which is used to perform layout prediction based on the device attributes of the target interface as input through a pre-trained layout prediction model and output the optimal layout of the device to which the target interface belongs;
[0036] An element classification module is used to classify elements in the human-computer interaction interface and generate classification labels for the elements using a classification module based on rule matching;
[0037] An element adjustment module, used to adjust the elements in the human-computer interaction interface according to the classification labels of the elements and using an adjustment strategy pre-matched with the classification labels, and deploy the adjusted elements to the target interface according to the optimal layout to obtain an initial interface;
[0038] A behavior prediction module, used to take the initial interface as input, perform behavior prediction through a pre-trained behavior prediction module, and output the user behavior of the device to which the target interface belongs;
[0039] The interface optimization module is used to optimize the elements in the initial interface based on the user behavior to complete the interface adaptive adjustment.
[0040] Compared with the related art, the adaptive interface adjustment method and adjustment system for human-computer interaction provided by the present invention have the following beneficial effects:
[0041] The present invention utilizes a pre-trained layout prediction model to predict the layout according to the device attributes of the device to which the target interface belongs, and outputs the optimal layout that best suits the device; secondly, a classification module based on rule matching is introduced to perform detailed classification processing on the elements in the human-computer interaction interface, and generate corresponding classification labels; then, according to the classification labels of the elements, the interface elements are adjusted using an adjustment strategy pre-matched with the classification labels, and the adjusted elements are deployed to the target interface according to the optimal layout to obtain an initial interface; finally, the initial interface is used as input, user behavior prediction is performed through a pre-trained behavior prediction module, and the elements in the initial interface are optimized based on the prediction results, which makes full use of user behavior data, realizes dynamic adjustment and continuous optimization of the interface, and further improves the adaptability and user experience of the interface. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 A flowchart of an adaptive interface adjustment method for human-computer interaction provided by the present invention;
[0043] Figure 2 A module structure diagram of an adaptive interface adjustment system for human-computer interaction provided by the present invention. DETAILED DESCRIPTION
[0044] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.
[0045] It should also be noted that, for ease of description, only the parts related to the present invention, but not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as being processed sequentially, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the various operations can be rearranged. The processing can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The processing can correspond to methods, functions, procedures, subroutines, subprograms, etc.
[0046] Embodiment 1
[0047] The present invention provides an adaptive interface adjustment method for human-computer interaction, referring to Figure 1 As shown, the adjustment method includes the following steps:
[0048] S1: Based on the device attributes of the device to which the target interface belongs as input, a pre-trained layout prediction model is used to perform layout prediction and output the optimal layout of the device to which the target interface belongs.
[0049] In this embodiment, the device attribute information of the device to which the target interface belongs is first collected, including the device type and screen size. This information is passed as input to a pre-trained layout prediction model, which uses a convolutional neural network (CNN) in deep learning to identify and understand the visual features and spatial constraints of different devices. The model predicts the layout solution that best suits the target device by analyzing the association between device attributes in historical data sets and the optimal layout, so as to generate customized layouts for different devices, ensuring that UI elements are clearly visible and easy to interact with on any screen, thereby improving cross-platform compatibility and user experience.
[0050] S2: Use the rule-matching-based classification module to classify the elements in the human-computer interaction interface and generate classification labels for the elements.
[0051] In this embodiment, a rule matching algorithm is used to classify each element in the human-computer interaction interface. Each element has its own specific characteristics. The rule matching module can identify these characteristics and mark them as corresponding categories. The generated classification labels provide a basis for the next step of personalized adjustment.
[0052] The classification module helps to understand the function and importance of interface elements, making the adjustment strategy more precise and enabling targeted optimization based on the category of the elements, such as adjusting the size of a button to fit a small screen or changing the position of a text box to avoid occlusion.
[0053] S3: According to the classification labels of the elements, the elements in the human-computer interaction interface are adjusted using an adjustment strategy pre-matched with the classification labels, and the adjusted elements are deployed in the target interface according to the optimal layout to obtain an initial interface.
[0054] In this embodiment, a set of predefined adjustment strategies are applied according to the classification labels generated in the previous step. For example, for buttons, if the device screen is small, its size needs to be reduced or its position needs to be changed; for text, the font size or line spacing needs to be adjusted. The adjusted elements are deployed according to the predicted optimal layout to form an initial interface, which realizes the preliminary personalized adjustment of interface elements and ensures that all elements still maintain good usability and readability under the new layout, while taking into account the differences in the characteristics of different devices.
[0055] S4: Taking the initial interface as input, performing behavior prediction through a pre-trained behavior prediction module and outputting the user behavior of the device to which the target interface belongs.
[0056] In this embodiment, by analyzing the user's interaction data under similar devices and interface configurations, it is predicted how the user interacts with the current interface. By predicting user behavior, it is possible to know in advance which interface elements may be used frequently, so that these elements can be optimized first in the next step, thereby improving user satisfaction and interface response speed.
[0057] S5: Based on the user behavior, the elements in the initial interface are optimized to complete the interface adaptive adjustment.
[0058] In this embodiment, based on the results of user behavior prediction, the interface elements will be adjusted again to achieve a better user interaction experience. The final adjusted interface will be more in line with the user's actual needs and usage habits, reduce the user's learning cost, improve operational efficiency, and overall improve user satisfaction and interface usability.
[0059] Specifically, step S1 includes the following steps:
[0060] S101: Read device attributes of the device to which the target interface belongs, and convert the device attributes into a feature vector, wherein the device attributes include device type and screen size.
[0061] In this embodiment, the attribute information of the device to which the target interface belongs is first read through an API or other means, focusing on the device type (such as smartphone, tablet, desktop) and screen size (width and height). These raw information are then converted into numerical feature vectors to facilitate the understanding and processing of the machine learning model. For example, the device type can be encoded as a one-hot vector, and the screen size is directly used as a numerical feature. Such a feature vector can more effectively express the characteristics of the device so that the model can make layout predictions.
[0062] S102: Using the feature vector of the device attribute as the input of the layout prediction model, calling the prediction function to perform inference calculation, and outputting the optimal layout of the device to which the target interface belongs.
[0063] In this embodiment, after having the feature vector of the device attributes, it is then input into the pre-trained layout prediction model, which is a deep learning model of a convolutional neural network. It has been trained with a large amount of historical data and can understand the relationship between different device attributes and the optimal layout. The model internally performs matrix multiplication, activation function application, and pooling operations to finally output a solution representing the optimal layout.
[0064] Specifically, in step S1, the training process of the layout prediction model includes:
[0065] An interface layout instance data set is collected and preprocessed, wherein the interface layout instance includes marked sample device attributes and sample layouts.
[0066] In this example, we first need to build a large dataset that contains interface layout instances on various devices. Each instance contains two parts of information: one is the device attributes, including device type (such as smartphones, tablets, desktop monitors) and screen size (resolution, aspect ratio, etc.); the other is the corresponding interface layout, that is, the UI layout solution that performs best on the device. This dataset should cover as many device types and interface layout styles as possible to ensure the generalization ability of the model.
[0067] Preprocessing includes cleaning data, removing outliers, standardizing numerical features (such as screen size), and encoding categorical features (such as device type). Preprocessing ensures data consistency and efficiency of model training.
[0068] The deep learning-based layout prediction model is trained using the training set in the preprocessed interface layout instance dataset, and the model parameters are iteratively adjusted to minimize the prediction error.
[0069] In this embodiment, during the model training phase, the preprocessed training set data is used to train the layout prediction model. The goal of the model is to minimize the error between the predicted layout and the actual layout. A loss function is usually used to measure this gap.
[0070] During the training process, the model parameters are updated through the back-propagation algorithm. This process uses the gradient descent method to gradually optimize the model parameters so that the model can predict the layout more accurately.
[0071] Based on the evaluation indicators, the performance is evaluated using the validation set in the preprocessed interface layout instance dataset, and the model parameters are adjusted according to the evaluation results, finally obtaining a trained layout prediction model.
[0072] In this embodiment, during model training, in addition to the training set, an independent validation set is also required to evaluate the performance of the model. The validation set is used to test the performance of the model on unseen data to prevent overfitting. Evaluation indicators include accuracy, precision, recall, F1 score, and area under the AUC-ROC curve.
[0073] If the model performs poorly on the validation set, you need to adjust hyperparameters (such as learning rate, batch size, and number of hidden layer units) to improve the model. Through repeated iterations, the ultimate goal is to obtain a model that performs well on the validation set, which marks the success of the model training process.
[0074] After training is completed, the model will be used to predict the optimal interface layout for unknown devices, achieving true adaptive interface adjustment.
[0075] Specifically, step S2 includes the following steps:
[0076] S201: Using an OCR recognition method to recognize elements of the human-computer interaction interface, and extracting features of the recognized elements, wherein the extracted features include visual features, geometric features, and text features.
[0077] In this embodiment, optical character recognition (OCR) technology is first used to parse and identify all elements on the interface. OCR technology can recognize text from images and is combined with computer vision technology to be able to recognize non-text elements such as icons, buttons, etc.
[0078] For each identified element, a variety of features are extracted, including:
[0079] Visual features: color distribution, texture pattern, edge contour, etc.
[0080] Geometric characteristics: position, size, shape, etc. of elements.
[0081] Text features: If the element is text or contains text, the text content, font, size, etc. are extracted.
[0082] S202: Compare the extracted features with the rules in the rule base of the classification module for similarity, and select the rule with the highest similarity score, wherein the rule base contains feature descriptions and classification standards of various element types.
[0083] In this embodiment, a rule library is maintained, which contains feature descriptions and classification standards of various interface elements. For example, buttons have specific color distribution, position tendency and text features (such as "OK" and "Cancel"). The features extracted from step S201 are compared with each rule in the rule library to calculate the similarity score.
[0084] Specifically, the similarity calculation is based on the Euclidean distance. The smaller the distance, the higher the similarity score. Therefore, the rule with the highest score will be selected as the basis for classification.
[0085] S203: assigning a classification label to each element in the human-computer interaction interface according to the selected rule, wherein the classification label includes a button, text, picture and input box.
[0086] In this embodiment, once the rule that best matches the extracted feature is determined, the corresponding interface element will be labeled with a classification label. The specific classification labels include:
[0087] Button: A clickable area that usually contains text or an icon to indicate a function.
[0088] Text: A plain text area where information is displayed.
[0089] Image: A static image element.
[0090] Input box: An area where users can enter text or data.
[0091] Specifically, step S3 includes the following steps:
[0092] S301: Use a predefined adjustment strategy to match the classification labels of the elements to obtain an adjustment rule for each element.
[0093] In this embodiment, predefined adjustment strategies are applied according to the classification labels previously assigned to each element. These strategies are formulated based on design principles and user experience criteria, and are intended to optimize the interface layout on different devices and screen sizes.
[0094] For example, for buttons, the adjustment strategy includes maintaining a sufficient click area to avoid overlapping with other elements; for text, the font size needs to be adjusted to adapt to different screen resolutions; for pictures, they need to be scaled or cropped to fit the screen space.
[0095] S302: modifying the attributes of the element using corresponding adjustment rules, wherein the attributes of the element include position, size and color.
[0096] In this embodiment, according to the adjustment rule determined in step S301, the attributes of the element are modified. These attribute modifications include:
[0097] Position: Move elements to better fit the grid or alignment guidelines.
[0098] Sizing: Resize elements to fit the screen size while maintaining readability and usability.
[0099] Color: Adjust the color of elements to improve contrast or follow brand guidelines.
[0100] S303: Re-adjusting the elements whose attributes have been modified according to the optimal layout, and optimizing the adjusted elements through a layout management algorithm to obtain an initial interface.
[0101] In this embodiment, after the property adjustment of all elements is completed, these elements are re-arranged on the interface to create an overall coordinated and beautiful layout. Specifically, conventional layout management algorithms are used to layout the elements to ensure that the elements can be correctly arranged on different devices and screen sizes.
[0102] The layout management algorithm will take into account the spacing and alignment between elements and the available space on the screen to generate an optimized layout solution. Ultimately, this optimized layout will be presented to the user as a new initial interface, providing a more personalized and adaptable user experience.
[0103] Specifically, step S4 includes the following steps:
[0104] S401: extracting features from the initial interface using a feature extraction method to obtain interface features.
[0105] In this embodiment, a feature extraction method based on deep learning is used to analyze the initial interface. Feature extraction can extract key information from the interface, such as the location of buttons, the distribution of text, the characteristics of images, etc., and express them in the form of mathematical vectors. These feature vectors contain the structure and visual characteristics of the interface, which can be understood and processed by the machine learning model.
[0106] Feature extraction includes:
[0107] Image processing: Identify and extract graphic elements in the interface.
[0108] Text Analysis: Analyze text content, formatting, and layout.
[0109] Layout understanding: Analyze the interface layout, such as the relative position and hierarchical relationship between elements.
[0110] S402: Taking the interface features as input, using a pre-trained behavior prediction model to perform behavior prediction, and outputting user behavior of the device to which the target interface belongs, wherein the behavior prediction model is trained based on historical interfaces and historical user behavior data.
[0111] In this embodiment, once features are extracted from the initial interface, these features are passed as input to a pre-trained behavior prediction model. The model is trained on a large number of historical interfaces and corresponding historical user behavior data sets, and can learn and infer user behavior patterns under a specific interface design.
[0112] The behavior prediction model is a deep neural network that is trained to predict multiple user behavior metrics, including:
[0113] Click frequency: The number and frequency of user clicks on interface elements.
[0114] Slide direction: the direction in which the user slides on the interface.
[0115] Dwell time: The length of time a user spends on an element or page.
[0116] Action path: The sequence of actions a user takes on an interface to complete a specific task.
[0117] The behavior prediction model will output a set of probability distributions, indicating the various behaviors that users may take and their likelihood. These prediction results will be used for the next step of interface optimization decisions to improve user experience and interface interaction efficiency.
[0118] Specifically, step S5 includes the following steps:
[0119] S501: Adjusting the weights of elements in the initial interface according to user behavior, wherein the weight adjustment includes increasing button size, changing element position, and adjusting font size.
[0120] In this embodiment, regarding increasing the button size: if data analysis shows that certain buttons are frequently clicked or have a high false-touch rate, their sizes can be increased to make them easier to touch or click, especially for users with larger fingers or small screens on mobile devices.
[0121] For changing element position: Frequently used features or controls can be moved to a more accessible location, such as at the bottom of the screen or near where the thumb naturally rests, for easy one-handed operation.
[0122] For adjusting font size: If users often zoom in on text or report that the text is too small to read, you can automatically adjust the font size to improve readability.
[0123] S502: Personalizing the elements in the initial interface according to user behavior, wherein the personalized adjustment includes adjusting the color theme, font style, and making the function entrance obvious.
[0124] In this embodiment, regarding adjusting the color theme: the user may have a preference for a certain color scheme, or may need a high contrast mode to improve visual clarity. Depending on the user's behavior or settings, the color theme of the interface can be changed.
[0125] Regarding font styles: Different font styles affect reading speed and comfort. Users may prefer a certain style or require a specific font to improve readability.
[0126] Make the function entrance obvious: important functions or frequently used functions should be more prominent, which can be achieved by enlarging the icon, using eye-catching colors or placing it in a conspicuous position.
[0127] S503: Adjusting the layout of the initial interface according to the user behavior, wherein the layout adjustment includes adjusting element spacing, rearranging element order, and optimizing navigation structure.
[0128] In this example, for adjusting element spacing: ensuring that there is appropriate spacing between elements to avoid crowding helps users better understand and navigate the interface.
[0129] For Rearrange Elements: Reorder elements on the interface based on frequency of use and importance to make the most commonly used features easier to access.
[0130] For optimizing navigation structure: simplify menus and navigation paths, reduce the number of clicks required for users to complete tasks, and improve efficiency.
[0131] The working principle of an adaptive interface adjustment method for human-computer interaction provided by the present invention is as follows: using a pre-trained layout prediction model, layout prediction is performed according to the device attributes of the device to which the target interface belongs, and the optimal layout that best suits the device is output; secondly, a classification module based on rule matching is introduced to perform detailed classification processing on the elements in the human-computer interaction interface, and generate corresponding classification labels; then, according to the classification labels of the elements, the interface elements are adjusted using an adjustment strategy pre-matched with the classification labels, and the adjusted elements are deployed to the target interface according to the optimal layout to obtain an initial interface; finally, the initial interface is used as input, user behavior prediction is performed through a pre-trained behavior prediction module, and the elements in the initial interface are optimized based on the prediction results, thereby making full use of user behavior data, realizing dynamic adjustment and continuous optimization of the interface, and further improving the adaptability of the interface and user experience.
[0132] Embodiment 2
[0133] The present invention also provides an adaptive interface adjustment system for human-computer interaction, which is used to execute the adaptive interface adjustment method for human-computer interaction. Figure 2 As shown, the adjustment system includes:
[0134] The device adaptation module 100 is used to perform layout prediction based on the device attributes of the target interface as input through a pre-trained layout prediction model and output the optimal layout of the device to which the target interface belongs.
[0135] The element classification module 200 is used to classify the elements in the human-computer interaction interface using a classification module based on rule matching and generate classification labels for the elements.
[0136] The element adjustment module 300 is used to adjust the elements in the human-computer interaction interface according to the classification labels of the elements using an adjustment strategy pre-matched with the classification labels, and deploy the adjusted elements to the target interface according to the optimal layout to obtain an initial interface.
[0137] The behavior prediction module 400 is used to take the initial interface as input, perform behavior prediction through a pre-trained behavior prediction module, and output the user behavior of the device to which the target interface belongs.
[0138] The interface optimization module 500 is used to optimize the elements in the initial interface based on the user behavior to complete the interface adaptive adjustment.
[0139] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0140] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically-erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0141] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
Claims
1. An adaptive interface adjustment method for human-computer interaction, characterized in that: The adjustment method includes the following steps: Based on the device attributes of the device to which the target interface belongs as input, a pre-trained layout prediction model is used to perform layout prediction and output the optimal layout of the device to which the target interface belongs; Use the classification module based on rule matching to classify the elements in the human-computer interaction interface and generate classification labels for the elements; Specifically include: Using an OCR recognition method to identify elements of the human-computer interaction interface, and extracting features of the identified elements, wherein the extracted features include visual features, geometric features, and text features; Comparing the extracted features with the rules in the rule base of the classification module for similarity, and selecting the rule with the highest similarity score, wherein the rule base contains feature descriptions and classification criteria for various element types; According to the selected rules, assigning a classification label to each element in the human-computer interaction interface, wherein the classification label includes buttons, texts, pictures and input boxes; According to the classification labels of the elements, the elements in the human-computer interaction interface are adjusted using an adjustment strategy pre-matched with the classification labels, and the adjusted elements are deployed in the target interface according to the optimal layout to obtain an initial interface; Specifically include: Use the predefined adjustment strategy to match the classification labels of the elements to obtain the adjustment rules for each element; Modify the attributes of the element using corresponding adjustment rules, wherein the attributes of the element include position, size and color; Re-adjust the elements with modified attributes according to the optimal layout, and optimize the adjusted elements through the layout management algorithm to obtain the initial interface; Taking the initial interface as input, performing behavior prediction through a pre-trained behavior prediction module and outputting the user behavior of the device to which the target interface belongs; Specifically include: Using feature extraction method to extract features from the initial interface, and obtaining interface features; Taking the interface features as input, using a pre-trained behavior prediction model to perform behavior prediction, and outputting the user behavior of the device to which the target interface belongs, wherein the behavior prediction model is trained based on historical interfaces and historical user behavior data; Based on the user behavior, the elements in the initial interface are optimized to complete the interface adaptive adjustment; Specifically include: Adjusting the weights of elements in the initial interface according to user behavior, wherein the weight adjustment includes increasing button size, changing element position, and adjusting font size; Performing personalized adjustments to the elements in the initial interface according to user behavior, wherein the personalized adjustments include adjusting color themes, font styles, and making function entrances more obvious; The initial interface is adjusted in layout according to user behavior, wherein the layout adjustment includes adjusting element spacing, rearranging element order, and optimizing navigation structure.
2. The method for adjusting an adaptive interface for human-computer interaction according to claim 1, characterized in that: The method of taking the device attribute of the device to which the target interface belongs as input, performing layout prediction through a pre-trained layout prediction model and outputting the optimal layout of the device to which the target interface belongs includes: Reading device attributes of the device to which the target interface belongs, and converting the device attributes into a feature vector, wherein the device attributes include device type and screen size; The feature vector of the device attribute is used as the input of the layout prediction model, and the prediction function is called to perform inference calculation to output the optimal layout of the device to which the target interface belongs.
3. The method for adjusting an adaptive interface for human-computer interaction according to claim 2, characterized in that: The training process of the layout prediction model includes: Collecting and preprocessing an interface layout instance data set, wherein the interface layout instance includes marked sample device attributes and sample layouts; The deep learning-based layout prediction model is trained using the training set in the preprocessed interface layout instance dataset, and the model parameters are iteratively adjusted to minimize the prediction error. Based on the evaluation indicators, the performance is evaluated using the validation set in the preprocessed interface layout instance dataset, and the model parameters are adjusted according to the evaluation results, finally obtaining a trained layout prediction model.
4. The method for adjusting an adaptive interface for human-computer interaction according to claim 3, characterized in that: The user behavior includes click frequency, sliding direction, dwell time and operation path.
5. An adaptive interface adjustment system for human-computer interaction, used to execute an adaptive interface adjustment method for human-computer interaction as claimed in any one of claims 1 to 4, characterized in that: The adjustment system comprises: A device adaptation module, which is used to perform layout prediction based on the device attributes of the target interface as input through a pre-trained layout prediction model and output the optimal layout of the device to which the target interface belongs; An element classification module is used to classify elements in the human-computer interaction interface and generate classification labels for the elements using a classification module based on rule matching; An element adjustment module, used to adjust the elements in the human-computer interaction interface according to the classification labels of the elements and using an adjustment strategy pre-matched with the classification labels, and deploy the adjusted elements to the target interface according to the optimal layout to obtain an initial interface; A behavior prediction module, used to take the initial interface as input, perform behavior prediction through a pre-trained behavior prediction module, and output the user behavior of the device to which the target interface belongs; The interface optimization module is used to optimize the elements in the initial interface based on the user behavior to complete the interface adaptive adjustment.