Adaptive User Interface Recommendation Method and System Based on Foot Interaction

Abstract

The present invention belongs to the technical field of interface recommendation, and provides an adaptive user interface recommendation method and system based on foot interaction. By using the foot length difference of users, the sizes of elements in the interaction interface are designed; the theme style and element shape in the interaction interface are recommended by using the deep learning field perception decomposition machine method; at the same time, an interface library with different layouts is constructed by using interface design rules, and the element layout in the interaction interface is recommended by using the reinforcement learning multi-armed bandit UCB1 algorithm. Through the personalized design of elements and layouts in the interaction interface, the possible misoperations that may occur during user movement are reduced or avoided, the accuracy of the user's natural interaction with the interface using the foot is improved, the efficiency of human-computer interaction is increased, and the user's needs are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of interface recommendation, and particularly relates to an adaptive user interface recommendation method and system based on foot interaction. Background Art

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] A human-computer interaction system consists of three parts: humans, machines, and the environment. Among them, the interaction interface is the carrier for information transmission and exchange between humans and computers. An appropriate interaction interface can help users quickly remember available functions in the interaction system and improve user productivity. However, when the interaction interface is not well designed, it may cause incorrect operations, thereby reducing productivity and accidentally tampering with data, etc. Usually, interface designers first follow general design principles, and then personalize design elements (such as buttons, input boxes) according to the characteristics of the target audience. To expand the user group and adapt to the usage habits of different people, many applications also provide functions such as user-defined theme colors, font sizes, etc. However, allowing users to actively modify design elements will make users spend more time learning operations and adapting to the interface, and the achieved effects often have many limitations.

[0004] In addition, in the field of human-computer interaction, although people have always been very interested in hand interaction based on touchscreens, the exploration of foot-based interaction has also begun. Foot-based interaction is often combined with other interaction methods to provide an alternative input method to make up for the deficiencies of other interaction methods. For example, by combining foot input with eye tracking, the "Midas Touch" problem is solved, and it is found that the performance in most tasks is as good as that of a mouse. However, at present, the research on foot-based interaction interfaces is relatively less, and there is a problem of poor interaction accuracy when foot interaction is used as the main interaction input. Summary of the Invention

[0005] To solve at least one of the above technical problems in the background art, the present invention provides an adaptive user interface recommendation method and system based on foot interaction, which proposes an interface recommendation method based on foot interaction for foot-based interaction interfaces, fully considers the interaction accuracy of feet and individual differences, generates a personalized interaction interface that is beneficial to improving foot interaction accuracy, and does not require users to actively modify design elements, reducing the burden on users for operation learning and interface adaptation.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] The first aspect of the present invention provides an adaptive user interface recommendation method based on foot interaction, including the following steps:

[0008] Obtain user image data;

[0009] Based on the user image data, determine whether the user is in the initial position area of the interaction interface. If so, confirm the user's identity. If it is a new user, recommend an interaction interface suitable for the current user. Otherwise, directly read the interaction interface preferred by the current user;

[0010] Among them, the specific steps of recommending an interaction interface suitable for the current user for the new user include:

[0011] Extract the new user's foot length feature and personal information feature based on the user image data;

[0012] Based on the relationship between the new user's foot length feature and the size of the elements in the interaction interface, obtain the size of the interaction interface elements suitable for the current user. Based on the new user's personal information feature and the trained FFM recommendation model, obtain the shape of the interaction interface elements suitable for the current user and the interface theme information;

[0013] At the same time, use the interface design rules to construct an interface library with different layouts, and recommend the element layout in the interface for the current user through the reinforcement learning multi-armed bandit UCB1 algorithm;

[0014] Fuse the obtained interaction interface element size, element shape, element layout, and interface theme information to obtain the recommended interaction interface.

[0015] The second aspect of the present invention provides an adaptive user interface recommendation system based on foot interaction, including:

[0016] A data acquisition module for obtaining user image data;

[0017] A user interface recommendation module for determining whether the user is in the initial position area of the interaction interface based on the user image data. If so, confirm the user's identity. If it is a new user, recommend an interaction interface suitable for the current user. Otherwise, directly read the interaction interface preferred by the current user;

[0018] Among them, the specific steps of recommending an interaction interface suitable for the current user for the new user include:

[0019] Extract the new user's foot length feature and personal information feature based on the user image data;

[0020] Based on the relationship between the new user's foot length feature and the size of the elements in the interaction interface, obtain the size of the interaction interface elements suitable for the current user. Based on the new user's personal information feature and the trained FFM recommendation model, obtain the shape of the interaction interface elements suitable for the current user and the interface theme information;

[0021] Meanwhile, an interface library with different layouts is constructed using interface design rules, and the element layout in the interface is recommended to the current user through the reinforcement learning multi-armed bandit UCB1 algorithm;

[0022] The obtained sizes of interactive interface elements, element shapes, layouts between elements, and interface theme information are fused to obtain the recommended interactive interface.

[0023] The third aspect of the present invention provides a computer-readable storage medium.

[0024] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in the above-mentioned foot-based interactive adaptive user interface recommendation method are implemented.

[0025] The fourth aspect of the present invention provides a computer device.

[0026] A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the steps in the above-mentioned foot-based interactive adaptive user interface recommendation method are implemented.

[0027] Compared with the prior art, the beneficial effects of the present invention are:

[0028] 1. Compared with common interface recommendation methods, the present invention proposes a foot-based interactive interface recommendation method for foot-based interactive interfaces, and fully considers the interaction accuracy of feet and individual differences, generating a personalized interactive interface that is conducive to improving the interaction accuracy of feet.

[0029] 2. The present invention selects the natural interaction method of feet, which has the advantages of convenience and easy operation, does not require additional learning and training for users, reduces the learning burden of users, and users can interact with the interactive interface by walking freely, creating a stronger and more natural experience that users can feel. At the same time, fully considering the individual characteristic differences of users, it mobilizes the enthusiasm of users to participate, and at the same time avoids the health hazards caused by sitting still for a long time, and promotes the development of cognitive, social and motor skills.

[0030] 3. The present invention introduces a clustering algorithm and an FFM algorithm for user characteristics, avoiding the cold start problem caused by the lack of historical usage data of new users in user interface recommendation, that is, based on user usage records, so as to recommend user interface elements to users in a more refined manner.

[0031] 4. When the present invention recommends the layout of interactive interface elements to users, it not only ensures the recommendation of the element layout interface with fewer misoperation times, but also provides more possibilities for users to explore other un-recommended interfaces, ensuring the diversity of the recommended interface and the freshness of the interaction experience.

[0032] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 is a flowchart of an adaptive recommendation method based on foot interaction according to a first embodiment of the present invention;

[0035] Figure 2(a) - Figure 2(b) This is an interface layout distribution diagram of the first embodiment of the present invention;

[0036] Figure 3 This is a coordinate diagram of interface elements in the first embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of limiting the visible area of the overall interface of the first embodiment of the present invention;

[0038] Figure 5 is a diagram showing the relationship between the projection interface and the top-down angle of a human body in a top-down situation according to the first embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of variables related to interface recommendation in the first embodiment of the present invention;

[0040] Figure 7(a) - Figure 7(c) This is a schematic diagram of the corresponding relationship between different leg lengths and elements in the first embodiment of the present invention;

[0041] Figure 8(a) - Figure 8(c) This is a schematic diagram of the sizes of interface elements corresponding to different user categories in the first embodiment of the present invention;

[0042] Figure 9 This is a schematic diagram of the structure of a model training data set in Embodiment 1 of the present invention;

[0043] Figure 10(a) - Figure 10(d) This is a schematic diagram of different interface theme styles in the first embodiment of the present invention;

[0044] Figure 11(a) - Figure 11(d) is a schematic diagram of the shapes of different elements in the interface of the first embodiment of the present invention;

[0045] Figure 12(a) - Figure 12(b) This is a diagram illustrating the detailed structure of model training data in Embodiment 1 of the present invention;

[0046] Figure 13(a) - Figure 13(d) This is a diagram showing the operation effect of the adaptive recommendation system based on foot interaction according to the second embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0048] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0049] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0050] Embodiment 1

[0051] As Figure 1 shown, this embodiment provides an adaptive user interface recommendation method based on foot interaction, including the following steps:

[0052] S1: Obtain user image data;

[0053] S2: Based on the user image data, determine whether the user is in the initial position area of the interaction interface. If so, confirm the user's identity. If it is a new user, recommend an interaction interface adapted to the current user. Otherwise, directly read the interaction interface preferred by the current user;

[0054] Among them, the specific steps of recommending an interaction interface adapted to the current user for the new user include:

[0055] Extract the foot length feature and personal information feature of the new user based on the user image data;

[0056] Based on the relationship between the foot length feature of the new user and the size of the elements in the interaction interface, obtain the size of the interaction interface elements adapted to the current user. Based on the personal information feature of the new user and the trained FFM recommendation model, obtain the shape of the interaction interface elements and the interface theme information adapted to the current user;

[0057] At the same time, use the interface design rules to construct an interface library with different layouts, and recommend the element layout in the interface for the current user through the reinforcement learning multi-armed bandit UCB1 algorithm;

[0058] Fuse the obtained interaction interface element size, element shape, element layout, and interface theme information to obtain the recommended interaction interface.

[0059] S3: Present the recommended interactive interface to the user through projection, and determine whether the user interacts with the elements in the interface. If it is detected that the user interacts with the elements in the interface through the feet, give corresponding interactive feedback, update the interface benefits with different interface element positions in the interface library, and update the data of the corresponding user in the database; otherwise, the interactive interface remains unchanged.

[0060] The above recommendation method specifically includes the following steps:

[0061] Step 1: Use the user's color image data and depth image data captured by the RGB-D camera, and use Baidu's face recognition algorithm to identify the user's identity, compare it with the data in the face database, and obtain the user's identity information. If it is a new user, register the user, and store the recognized user's height, gender, age, and information such as the location and major filled in by the new user in the database as the user's personal characteristic information, and assign a unique identity identification number to the new user; if it is an old user, query the user's personal information and interface usage records from the data.

[0062] Step 2: Call different interactive interface recommendation strategies according to whether the user is a new user. If it is a new user and there is no historical usage data, recommend the element size, shape, layout, and interface theme style in the interactive interface according to the obtained user personal characteristics, B_SH_FFM model, and UCB1 algorithm, etc. If it is an old user, directly recommend the appropriate interface element size, interface element shape, and interface theme style using the user's historical records stored in the database, and at the same time, use the UCB1 algorithm to recommend the interface element position in real time according to the user's current interaction behavior.

[0063] Step 3: Present the obtained recommended interface on the ground through projection for the user to perform interactive operations;

[0064] Step 4: Determine whether the user interacts with the interactive interface and trigger different interactive operations. Use the position data of the user's foot joints captured by the RGB-D camera to perform collision detection with the elements on the interactive interface. If the user interacts with the target element, give feedback from two aspects: vision and hearing.

[0065] Among them, the specific implementation method of Step 2 is:

[0066] If the user is a new user, the corresponding interactive interface recommendation method is:

[0067] 1) Calculate the user's foot length according to the user's height, H foot = H height *0.142; According to the user's foot length characteristics, determine that the user belongs to M footWhich class in the class, that is, calculate the Euclidean distance between the user's foot length and the cluster center of the M foot cluster centers of the foot length clusters, and divide the current user's foot length into a certain cluster M it (i = 1, 2, ..., M foot ), and recommend the size L of the interface element corresponding to the cluster i ×L i to this user;

[0068] 2) According to the user characteristics, judge which class in the M feature class the user belongs to, and find a similar user class M for the current user if (i = 1, 2, ..., M feature ). Use the B_SH_FFM model to recommend the top ten interface layouts (theme style, interface element shape) sets of similar users (users in the M if class), and recommend them to the current new user to solve the cold start problem of new users;

[0069] 3) Integrate the recommended interface element size, interface element shape, theme style, and the interface element layout given by the UCB1 algorithm, and present the entire interactive interface k to the user;

[0070] 4) The RGB-D camera captures the current user's interaction behavior, calculates the number of misoperations of the current user on the interface k, and updates the benefit value of the current user on the interface k Use the UCB1 algorithm to update the upper confidence limit value of each interface in the interface library in real time, select the interface with the highest upper confidence limit value and recommend it to the current user; at the same time, record the number of times n j (j = 1, 2, 3…N u ) that the current user uses each interface, the total number of times t that the current user uses all interfaces, the total number of misoperations a j of the current user on each interface, the average number of misoperations of the current user on each interface and the benefit value of each interface into the database, and continuously recommend different interface element positions to the user to minimize the number of misoperations of the user.

[0071] The specific process of using the UCB1 algorithm to recommend the interface element layout for the user in step 4) is as follows:

[0072] Step 4-1. Initialization: a1, a2, …a j ; u1, u2……u j ; where j = 1, 2, 3…N u , is the average number of misoperations for each interface, aj is the total number of misoperations for each interface during the initialization phase is the initial average revenue value for each interface; u j is the normalized result;

[0073] Step 4-2. Update the total number of misoperations: Calculate the number of misoperations generated by the current user i using the current interface k during this interaction Update the total number of misoperations a of the current user i on the current interface k k , (i is the user number of the current user, where is the total number of misoperations of the current user i on the current interface k excluding this misoperation count );

[0074] Step 4-3. Update the average number of misoperations: Update the average number of misoperations of interface k (n k is the total number of times the current user i uses interface k. Since the initial usage count of each interface is 1 during the initialization phase, n k ≥2);

[0075] Step 4-4. Update the average revenue value: Update the average revenue value of interface k (Q is a constant, and in this embodiment, Q = 3);

[0076] Step 4-5. Normalize the revenue value using the normalization exponential function (softmax function) The normalized value is u k : (N u is the total number of interfaces)

[0077] Step 4-6. Update the upper confidence bound of each interface: (After the current user i completes the interaction with the current interface k, this formula will be updated once). t is the total number of times the current user uses all interfaces (since the initialization count of each interface is 1, so t >= N u ); Select the largest UCB1 max , and max is the interface number corresponding to UCB1 max ; Recommend the position of the interface element with the interface number max to the current user, and update k = max;

[0078] Step 4-7. If the interaction operation between the user and the interface ends, store k in the database for recommendation when the user uses the system next time; otherwise, repeat steps Set4-2 to Step4-7.

[0079] In addition, as can be seen from the above Step4-6, the magnitude of the maximum value is the upper limit of the possible revenue value for selecting each interface. As n j increases, it decreases. Therefore, when interface k is frequently selected, the usage times n w (w = 1, 2…(k - 1), (k + 1)……N u ) of the other unselected interfaces remain unchanged, while the total usage times t of all interfaces used by the user increases. Therefore, the UCB1 values of the other interfaces will increase. In this way, diverse interfaces can be provided for the user; moreover, considering the use of the logarithm lnt, as time goes by, the interfaces with lower revenue value estimates or those that have been selected multiple times are selected less frequently. Therefore, when the present invention recommends interfaces with different element layouts, it not only ensures the recommendation of the element layout interfaces with fewer misoperation times, but also provides more possibilities for the user to explore other un-recommended interfaces, ensuring the diversity of the recommended interfaces.

[0080] If the user is an old user, the corresponding interactive interface recommendation method is:

[0081] 1) Extract the user's historical record information from the database, including the classification to which the user's foot length belongs, the user's ratings for the theme style and element shape of each interface, the number of missteps and revenue information of the user on each interface;

[0082] 2) Directly use the above information to recommend appropriate-sized interface elements, the top ten theme styles and element shapes with the user's ratings, and the interface element layout for this user;

[0083] 3) Integrate the above information and present the entire interactive interface k to the user;

[0084] 4) The RGB-D camera captures the current user's interaction behavior, calculates the number of misoperations that occur to the current user on interface k, and updates the revenue value of the current user on interface k Use the UCB1 algorithm to update the upper confidence limit value of each interface in the interface library in real time, and select the interface with the highest upper confidence limit value to recommend to the current user; at the same time, record the number of times n j (j = 1, 2, 3…N u ) that the current user uses each interface, the total number of times t that the current user uses all interfaces, the total number of misoperations a of the current user on each interface j , and the average number of misoperations of the current user on each interface And the revenue value of each interface The user can store the information in the database and continuously recommend different interface element positions to minimize the number of wrong operations by the user. Here, the recommendation method based on the UCB1 algorithm is the same as the UCB1 recommendation method for new users, and will not be repeated here.

[0085] It should be noted that in S2, in the interface library with different layouts constructed by using interface design rules, the design rules used can be selected according to actual needs. In this embodiment, the design rules used include Gestalt psychology and Nielsen F visual model interface design rules to construct an interface library with different layouts.

[0086] Design principle 1: The interface design follows the principle of similarity in visual design (Gestalt psychology). One of the important concepts is related elements. Design elements that look similar in some way from a visual perspective (same color, shape, size or distance between elements) are considered the same group, while elements that look different are considered different groups. Therefore, in terms of user interface design, objects with similar visual features are likely to be related, or at least in the direction that they should be related. Following this principle, the interface is divided into three main areas, as shown in Figure 2 (a): description area, waiting area, and initial position area. Moreover, the same area uses a consistent pattern, while different areas use patterns with large differences.

[0087] This embodiment mainly recommends the description area and the area to be selected.

[0088] Taking the question answering system based on ground projection as an example, as shown in Figure 2(b), the description area is the question display area S1, the area to be selected is the candidate answer display area S2, and the initial position area is the start button area S3. Among them, the question display area and the candidate answer display area are the main areas of the system interface design, and the candidate answer display area is the most important area of interface change.

[0089] Design Principle 2: Nielsen's F visual model points out that in the absence of subtitles and bullet points, users tend to concentrate words at the beginning of the line and at the top of the page. This scanning behavior will produce an eye movement pattern similar to the capital letter F. In view of this principle, when designing the interface, place the description area and the area to be selected above the initial position area, and place the initial position area at the bottom of the interface to ensure that the description area and the area to be selected are concentrated at the top of the page, which is in line with the user eye movement pattern of the F-type principle.

[0090] Figure 2(b) shows the module division of the user interface of the answering system based on floor projection. Before each round of answering questions, the user needs to stand in area S3 and step on the start button to trigger the display of questions and answers. During the answering interaction process, the user mainly moves in modules S1 and S2. Therefore, finding the best design solutions for S1 and S2 is crucial for improving the accuracy of the user's foot operations when answering questions in the answering system. The interface parameters are set according to the F-type rule as follows: Each element in areas S1, S2, and S3 will have a coordinate on the screen, Figure 3 Show the coordinate distribution of the interface elements.

[0091] Taking candidate answer A in S1 as an example: Its coordinate position on the screen is (X1, Y1), and the coordinate position of the question in area S2 is (X T , Y T ), and the coordinate position of the start button in area S3 is (X k , Y k ). The existing constraints are:

[0092] Y k > Y i , Y k > Y t , (i = 1, 2, 3, 4)

[0093] The above are the constraints in the Y direction, making S3 below S1 and S2.

[0094] Design principle 3: Proximity is one of the most important grouping principles and can override other competing irrelevant visual cues, such as similarity in color or shape. Modules can overlap, but the elements within the same module cannot be separated by the elements of other modules. The description area and the initial position area should be on the same side of the area to be selected; and it is necessary to ensure that the elements in the same area have the same size, shape, style, and color. Ensure that the elements in the same area are visually a whole.

[0095] This embodiment still takes the answering system based on floor projection as an example. The question display area and the start button display area are on the same side of the area containing the four answer display areas; and it is necessary to ensure that the sizes, shapes, styles, and colors of the four answers included in the answer display area are the same. Ensure that the four answers are visually a whole.

[0096] Define as follows:

[0097] The square of the distance between answer i and answer j: D ij = (X i - X j ) 2 + (Y i - Y j ) 2 ;

[0098] The square of the distance between Answer i and the question: D iT =(X i - X T ) 2 +(Y i - Y T ) 2 ;

[0099] The square of the distance between Answer i and the start button: D iK =(X i - X K ) 2 +(Y i - Y K ) 2 ;

[0100] According to Design Principle 3, the following constraints are imposed on the interface design:

[0101] 1) Constraint 1:

[0102] D iT > min(D ij ) and D ik > min(D ij )

[0103] This constraint ensures that the answer area is not the closest to the question area and the start button area, avoiding incorrect operations.

[0104] Constraint 2:

[0105] max(X i ) < X T or min(X i ) > X T ; max(Y i ) < Y T or min(Y i ) > Y T

[0106] This constraint ensures that each candidate answer within the answer area is on one side of the question in the X direction and on one side of the question in the Y direction.

[0107] In addition, considering the interface layout margins and the safe area for user interaction, an overall interface area limit is also imposed. If the total layout size is W*H and the visible interface size is w*h, then w < W, h < H.

[0108] Taking the answer system based on ground projection as an example, as Figure 4 shown, the overall layout is 1920*1080, W = 1920, H = 1080, and the visible interface is 1750*1020, w = 1750, h = 1020.

[0109] Considering that the interface display and interaction are in the form of ground projection, users will use the system from a top-down perspective. Therefore, it is also necessary to consider the relationship between the projected interface and the user's top-down viewing angle.

[0110] Taking the ground-projected answering system as an example, the relationship is obtained by the experimental method.

[0111] The specific experimental method is as follows: Invite 10 testers. The testers stand in the start button area in turn to view the question display area and the answer area, measure the closest distance from the question display area and the answer display area that each tester can accept to the start button area. Beyond this distance, the user will feel uncomfortable. Record this distance, and take the average value V of all the measured top-down distances.

[0112] Therefore, to avoid discomfort for users from the top-down viewing angle when using the system, the distance from the initial position area where the user is located to the area to be selected and the description area should be greater than V. As Figure 5 shown, in this embodiment, V = 3013 mm. To enable users to view the question display area and the answer display area at a comfortable angle in the start button area, the distance from the start button area to the question display area and the answer display area should be greater than 3013 mm.

[0113] In the design variable selection of the embodiment, based on the research and judgment of users, it is considered that the four variables of the interface element size of the area to be selected and the description area, the shape of the interface elements in the area to be selected, the overall interface background style, and the position of the interface elements in the area to be selected and the description area are very important design variables affecting the interface layout. Therefore, they are given priority as the objects to be considered. As Figure 6 shown.

[0114] Based on the above design principles, combined with the selected design variables, randomly generate N u different user interfaces as the interface alternative library to support the algorithm for recommendation.

[0115] Taking the ground-projected answering system as an example, construct an interface library with N u = 90. Among them, the interface element sizes of the answer display area and the question display area are divided into three types, and each type of element size corresponds to thirty different positions. Therefore, N u = 3 × 30.

[0116] In S2, obtaining the size of the interface elements suitable for the current user based on the relationship between the new user's foot length characteristics and the elements in the interaction interface specifically includes:

[0117] Establish the relationship between the user's foot length characteristics and the element size in the interaction interface, and classify the user's foot length characteristics by the clustering method so that users with different foot lengths correspond to different-sized elements in the interface;

[0118] In this embodiment, a questionnaire is used to collect the height, foot length, and stride data of users (the number of users is N p ). The data is analyzed to obtain the proportional data of the user's height and foot length. The K-means clustering algorithm is used to cluster the foot length data of users into M foot categories. The average value of the foot length in each user category is measured, and the size value of the interface elements in the area to be selected is calculated according to the size of the interface display area.

[0119] Taking the answer system based on ground projection as an example, the height, foot length, and stride data of 22 users are collected, that is, N p = 22. By analyzing the data, the proportional formula of the user's height and foot length is obtained: H foot = H height *0.142,; (H foot is the user's foot length, H height is the user's height). And according to the foot length data of all users the k-means clustering algorithm is used to cluster users. The specific method is:

[0120] (1) Initialization, randomly select M clustering centers, denoted as O cen (cen = 1, 2,..., M foot ). In this embodiment, M foot = 3;

[0121] (2) Calculate the Euclidean distance between the foot length data of all users and each clustering center and divide the users into the category with the smallest distance from the clustering center;

[0122] (3) Recalculate the clustering centers of the 3 categories (that is, move each clustering center to the center position of the cluster it belongs to), and this position is the position of the updated clustering center, update O cen ;

[0123] (4) If the distance between the new clustering center and the old clustering center is less than a set threshold ε t , the algorithm terminates. In this embodiment, ε t = 0.1;

[0124] (5) If the distance between the new clustering center and the old clustering center exceeds the given value ε t , steps (2)-(4) need to be iterated.

[0125] In addition, the relationship between the foot length and the size of the elements in the area to be selected also needs to be considered, such as Figure 7(a) - Figure 7(c)As shown in the figure, in Fig. 7(a), the foot length is much greater than the answer length. In Fig. 7(b), the foot length is almost the same as the answer length. In Fig. 7(c), the answer length is much greater than the foot length. An overly long foot length will cover the elements in the area to be selected, and users cannot determine whether the interaction is successful. However, overly large interface elements will result in a waste of interface space. Therefore, the most appropriate relationship between the foot length and the interface elements should be as shown in Fig. 7(b). The appropriate foot length corresponds to the appropriate size of the interface elements, which will neither affect the user's interaction experience nor cause excessive waste of the design space.

[0126] In this embodiment, by measuring the size ratio information of the ground projection area and the screen area, M foot = 3 types of clustering center values O cen (cen = 1, 2, 3) are converted into interface element size data L1*L1, L2*L2, L3*L3. The clustering centers are O1 = 2200mm, O2 = 2450mm, O3 = 2700mm respectively, and the corresponding sizes of the elements in the area to be selected are 135*135, 150*150, 165*165, as Figure 8(a) - Figure 8(c) shown.

[0127] In S2, in this embodiment, the construction process of the B_SH_FFM model is as follows: Based on the field-aware factorization machine (FFM) model, the concept of domain (Field) is introduced. In this embodiment, the domain can be regarded as grouping features.

[0128] As Figure 9 shown, it is divided into six domains (Fields). Among them, user features are divided into four domains (Fields), namely location of family, major, age, and sex; interface features are divided into two domains (Fields), namely theme style and shape of elements.

[0129] In this embodiment, the method of recruiting subjects to conduct on-site experiences and then filling out questionnaires is used to collect and analyze user feature data and evaluation data of users on the interactive interface (here, the interface attributes mainly examine the interface theme style and the shape of interface elements) (a 5-point Likert rating scale is used, where 1 means very dislike and 5 means very like).

[0130] The user feature and interface feature data are used as the input data of the model, and the evaluation data of users on each interface are used as the output data to train the B_SH_FFM model.

[0131] Taking the ground projection-based answering system as an example, the above user characteristics and interface characteristics can be split into several independent characteristics. The gender (sex) characteristic has two values, male and female. After one-hot encoding the values, the gender characteristic will be split into two independent characteristics x male , x female . Obviously, these two characteristics have a common property: they both belong to gender. So these two characteristics can be grouped under the same field, that is, they have the same field number, and these two independent characteristics have their own characteristic numbers.

[0132] Similarly, the location of the family is divided into two independent characteristics x shandong , x ohterLocation , the major is divided into three independent characteristics x software , x media , x otherMajor ; age is numerical and does not need to be converted by one-hot encoding. The theme style is divided into four independent characteristics x b1 , x b2 , x b3 , x b4 ; the element shape has four independent characteristics x s1 , x s2 , x s3 , x s4 ; between the characteristics of different fields, there are often obvious differences. As Figure 10(a) - Figure 10(d) shows, there are 4 types of theme styles. As Figure 11(a) - Figure 11(d) shows, there are 4 types of domain interface element shapes, and the corresponding radian values are 0 degrees, 15 degrees, 40 degrees, and 70 degrees. Therefore, the set S ui of interface element design schemes contains a total of 16 types.

[0133] The scoring data of the user for each interface layout, that is, the label data of the B_SH_FFM model is:

[0134]

[0135] Among them, w0 is the constant term, w i is the coefficient of the independent characteristic x i , is the dot product value of the two, x i , x j represent different independent characteristics, represents the hidden vector of the independent characteristic x i for the independent characteristic x j , represents the independent characteristic x jFor the independent feature x i The latent vector. For each independent feature x j , for each domain (Field) to which other independent features belong, a latent vector will be learned f i Denoted as x i The domain (Field) number corresponding to the feature. If the number of independent features is n feature , and the number of domains is n field , then the number of latent vectors is n feature ×n field (j = 1, 2... n feature , i = 1, 2... n field ).

[0136] As Figure 9 shown, the input data set of the model is composed of and concatenated together, and score is the label data of the B_SH_FFM model The model training data set is composed of and its corresponding scoring set . Among them, the three columns of numbers in each User Feature represent the domain (Field) number Field number, the feature number Featurenumber, and the feature value value, as Figure 12(a) - Figure 12(b) shown.

[0137] In S2, an interface library with different layouts is constructed using the interface design rules. Before recommending the element layout in the interface for the current user through the reinforcement learning multi-armed bandit UCB1 algorithm, the number of incorrect operations, the revenue value, and the number of usage times of each interface in the interface library are initialized to provide the initial data source for the multi-armed bandit UCB1 algorithm;

[0138] The specific process includes:

[0139] Invite N p subjects. Each subject interacts with N u interfaces in the constructed interface library once, and count and calculate:

[0140] The number of incorrect operations generated by each subject on each interface (i represents the subject number, i = 1, 2, 3... N p , j represents the interface number, j = 1, 2, 3... N u );

[0141] The average number of incorrect operations of each interface

[0142] Total number of accidental clicks on each interface The number of times each interface is used C = 1;

[0143] Revenue value of each interface Among them, Q is a constant. Let the number of interactive buttons in the area to be selected be N button , and the number of buttons that the user can select in one interaction in the area to be selected is N choose , then Q = N button -N choose . As shown in Figure 2(b), the area to be selected is the answer display area, and there are four answers, that is, the number of interactive buttons is 4, N button = 4, and there is only one correct answer, that is, the number of buttons that the user can select in one interaction in the answer display area is 1, N choose = 1, then the constant Q = 4 - 1, and the value of Q is 3.

[0144] For the revenue value perform normalization, and the normalized value is u j ,

[0145] To solve the cold start problem of new users lacking historical interface usage data, when extracting the foot length feature and personal information feature of new users based on user image data, cluster users according to user characteristics (such as place of residence, major, age, gender), and divide them into M feature categories. When a new user arrives, cluster the new user to find a user group with similar user characteristics for the new user, and select an interface from the set of interface layouts liked by similar users to recommend to the new user, so as to solve the cold start problem faced by new users.

[0146] Specifically, it includes the following steps:

[0147] (1) Initialization, randomly select M feature cluster centers, denoted as CFea f (f = 1, 2,..., M feature ). In this embodiment, M feature = 3;

[0148] (2) Calculate the distance between all user feature data and each cluster center, and divide the users into the category with the smallest distance from the cluster center;

[0149] (3) Recalculate the cluster centers of M feature categories (that is, move each cluster center to the center position of the cluster it belongs to), and this position is the position of the updated cluster center, and update CFea f(f = 1, 2, ..., M feature );

[0150] (4) If the distance between the new clustering center and the old clustering center is less than a set threshold ε t , the algorithm terminates. In this embodiment, ε t = 0.1;

[0151] (5) If the distance between the new clustering center and the old clustering center exceeds the given value ε t , steps (2)-(5) need to be iterated.

[0152] In S3, the maintenance and update of the database specifically include using the user characteristics and interface attribute information data (interface theme style and interface element shape) of the current user as the input data of the B_SH_FFM model. The output data is the score of the current user for each interface layout. Sort the score data from high to low, select the top ten interfaces and store them in the database, and update the preference data of the current user for the interactive interface.

[0153] In a specific embodiment, after each user uses the system, the user numbers and user characteristics of these users will be saved in the database.

[0154] In this embodiment, all user numbers and user characteristics are extracted during the database update phase, and users are distinguished according to the user numbers. For user P, the characteristics of user P are processed into a vector The interface attribute (background style and element shape) characteristics are processed into a vector The input data set of the model consists of and stitched together in the same way as in the model training phase, which will not be elaborated here; Set a timer. After the end of each day, at 00:00, the input data will be sent into the model. The output of the model is the score of each user for each interface. For user P, sort the score of user P for each interface, select the top ten interfaces with the highest scores of user P, and update the interface set stored by user P in the database.

[0155] Embodiment 2

[0156] This embodiment provides an adaptive user interface recommendation system based on foot interaction, including:

[0157] A data acquisition module for acquiring user image data;

[0158] A user interface recommendation module for judging whether the user is in the initial position area of the interactive interface based on the user image data. If so, confirm the identity of the user. If it is a new user, recommend an interactive interface suitable for the current user. Otherwise, directly read the interactive interface preferred by the current user;

[0159] Among them, the specific steps of recommending an interactive interface suitable for the current user to the new user include:

[0160] Extract the foot length feature and personal information feature of the new user based on the user image data;

[0161] Based on the relationship between the foot length feature of the new user and the size of the elements in the interactive interface, obtain the size of the interactive interface elements suitable for the current user. Based on the personal information feature of the new user and the trained FFM recommendation model, obtain the shape of the interactive interface elements and the interface theme information suitable for the current user;

[0162] At the same time, use the interface design rules to construct an interface library with different layouts, and recommend the element layout in the interface for the current user through the reinforcement learning multi-armed bandit UCB1 algorithm;

[0163] Fuse the obtained interactive interface element size, element shape, element layout, and interface theme information to obtain the recommended interactive interface.

[0164] Such as Figure 13(a) - Figure 13(d) is the operation effect diagram of the foot-based adaptive recommendation system.

[0165] Embodiment III

[0166] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps in the above-mentioned foot-based adaptive user interface recommendation method.

[0167] Embodiment IV

[0168] This embodiment provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in the above-mentioned foot-based adaptive user interface recommendation method.

[0169] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program code.

[0170] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0171] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0173] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0174] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An adaptive user interface recommendation method based on foot interaction, characterized in that It includes the following steps: Obtain user image data; Based on the user image data, determine whether the user is in the initial position area of the interaction interface. If so, confirm the user's identity. If it is a new user, recommend an interaction interface suitable for the current user. Otherwise, directly read the interaction interface preferred by the current user; Among them, the specific steps of recommending an interaction interface suitable for the current user for the new user include: Extract the foot length feature and personal information feature of the new user based on the user image data; Based on the relationship between the foot length feature of the new user and the size of the elements in the interaction interface, obtain the size of the interaction interface elements suitable for the current user. Based on the personal information feature of the new user and the trained FFM recommendation model, obtain the shape of the interaction interface elements suitable for the current user and the interface theme information; At the same time, use the interface design rules to construct an interface library with different layouts, and recommend the element layout in the interface for the current user through the reinforcement learning multi-armed bandit UCB1 algorithm; Fuse the obtained interaction interface element size, element shape, element layout, and interface theme information to obtain the recommended interaction interface.

2. The self-adaptive user interface recommendation method based on foot interaction according to claim 1, wherein After obtaining the recommended interaction interface, present it to the user through projection, and judge whether the user interacts with the elements in the interface. If it is detected that the user interacts with the elements in the interface through the feet, give corresponding interaction feedback, update the interface benefits with different interface element positions in the interface library, and update the data of the corresponding user in the database; otherwise, the interaction interface remains unchanged.

3. The self-adaptive user interface recommendation method based on foot interaction according to claim 2, wherein Calculate the number of incorrect operations performed by the current user on interface k, and update the revenue value of the current user on interface k Use the UCB1 algorithm to update the upper confidence bound value of each interface in the interface library in real time, and select the interface with the highest upper confidence bound value to recommend to the current user; at the same time, store the number of times the current user uses each interface, the total number of times the current user uses all interfaces, the total number of incorrect operations of the current user on each interface, the average number of incorrect operations of the current user on each interface, and the revenue value of each interface in the database, and continuously recommend different interface element positions to the user 4. The self-adaptive user interface recommendation method based on foot interaction according to claim 2, wherein The update of the data of the corresponding user in the database includes combining the user characteristics of the current user, the shape of the interaction interface elements of the current user, and the interface theme information data, obtaining the score of the current user for each interface layout through the B_SH_FFM model, sorting the score data from high to low, screening the top ten interfaces and storing them in the database, and updating the preference data of the current user for the interaction interface.

5. The method for adaptively recommending a user interface based on foot interaction according to claim 1, wherein, The interface design rules adopted include the Gestalt psychology and the Nielsen F visual model interface design rules. The interface is divided into regions by using Gestalt psychology, including a description area, a to-be-selected area, and an initial position area. Recommendations are made for the description area and the to-be-selected area. The description area and the to-be-selected area are placed above the initial position area by using the Nielsen F visual model, and the initial position area is placed at the bottom layer of the interface.

6. The self-adaptive user interface recommendation method based on foot interaction according to claim 5, characterized in that The specific steps of obtaining the size of the interaction interface elements suitable for the current user based on the relationship between the foot length feature of the new user and the size of the elements in the interaction interface include: Collect the height, foot length, and stride data of the user, parse the data, obtain the proportional data of the user's height and foot length, and use the K-means clustering algorithm to cluster the user's foot length data into M foot categories. Measure the average value of the foot length under each user category, and calculate the size value of the interface elements in the area to be selected according to the size of the interface display area.

7. The method for adaptively recommending a user interface based on foot interaction according to claim 1, wherein The construction process of the FFM recommendation model is as follows: Based on the field-aware factorization machine FFM model, introduce the concept of domain and group the features.

8. An adaptive user interface recommendation system based on foot interaction, characterized in that, It includes: A data acquisition module for obtaining user image data; A user interface recommendation module for determining whether the user is in the initial position area of the interaction interface based on the user image data. If so, confirm the user's identity. If it is a new user, recommend an interaction interface suitable for the current user. Otherwise, directly read the interaction interface preferred by the current user; Among them, the specific steps of recommending an interaction interface suitable for the current user for the new user include: Extract the foot length feature and personal information feature of the new user based on the user image data; Based on the relationship between the foot length characteristics of the new user and the sizes of the elements in the interaction interface, the sizes of the interaction interface elements suitable for the current user are obtained. Based on the personal information characteristics of the new user and the trained FFM recommendation model, the shapes of the interaction interface elements suitable for the current user and the interface theme information are obtained; Meanwhile, an interface library with different layouts is constructed using interface design rules, and the element layout in the interface is recommended for the current user through the reinforcement learning multi-armed bandit UCB1 algorithm; The obtained sizes of the interaction interface elements, element shapes, element layouts, and interface theme information are fused to obtain the recommended interaction interface.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the steps in the foot interaction-based self-adaptive user interface recommendation method described in any one of claims 1-7 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the steps in the foot interaction-based self-adaptive user interface recommendation method described in any one of claims 1-7 are implemented.

Images