A software interface management method and system

Abstract

This invention provides a software interface management method and system. The method includes determining multiple similar users and the similarity between each user and the user based on user information and user operation records in the mobile phone system; determining the user's attention to each module in the target software using a video processing model based on screen recordings of the user's operations in the target software; constructing a knowledge graph; and processing the knowledge graph using a graph cyclic network to determine the adjusted target software interface. This method can intelligently adjust the software interface and improve the user experience.

Description

Technical Field

[0001] The present invention relates to the field of software technology, and in particular to a software interface management method and system. Background Art

[0002] Software applications (APPs) have become an indispensable part of people's daily lives, covering many fields from social communication, online shopping, health management to education and learning. However, with the continuous enrichment of software functions, user interface (UI) design has become increasingly complex. How to provide users with a personalized, efficient and easy-to-operate interface has become one of the important challenges of software development. The current software interface management methods on the market mainly rely on manual adjustments by users. Users need to manually adjust the interface settings, which is often cumbersome and time-consuming, especially for those users who are not familiar with software operations, which increases the difficulty of use.

[0003] Therefore, how to intelligently adjust the software interface and improve user experience is a problem that needs to be solved at present. Summary of the invention

[0004] The main technical problem solved by the present invention is how to intelligently adjust the software interface to improve the user experience.

[0005] According to a first aspect, the present invention provides a software interface management method, comprising: obtaining user information and user operation record information in a mobile phone system; determining multiple similar users and the similarity between the user and each similar user based on the user information and the user operation record information in the mobile phone system; obtaining multiple display modules set by each similar user in the target software interface; obtaining the user's operation screen recording in the target software; using a video processing model to determine the user's attention to each module in the target software based on the user's operation screen recording in the target software; constructing a knowledge graph, the knowledge graph comprising multiple nodes and multiple edges between the multiple nodes, the multiple nodes comprising multiple similar user nodes and a user node, each similar user node respectively establishing an edge with the user node, the node feature of each similar user node comprising the multiple display modules set by each similar user in the target software interface, the node feature of the user node comprising the user's attention to each module in the target software, and the feature of each similar user node respectively establishing an edge with the user node comprising the similarity between the user and each similar user; using a graph recurrent network to process the knowledge graph to determine the adjusted target software interface.

[0006] Furthermore, the video processing model is a long-term and short-term neural network model.

[0007] Furthermore, the user information includes basic information and behavior preference information of the user.

[0008] Furthermore, the input of the graph recurrent network is the knowledge graph, and the output of the graph recurrent network is the adjusted target software interface.

[0009] Furthermore, the determining of multiple similar users and the similarity between the user and each similar user based on the user information and the user's operation record information in the mobile phone system includes: inputting the user information and the user's operation record information in the mobile phone system into a similarity determination model to obtain multiple similar users and the similarity between the user and each similar user, and the similarity determination model is a deep neural network model.

[0010] According to a second aspect, the present invention provides a software interface management system, comprising:

[0011] The first acquisition module is used to acquire user information and user operation record information in the mobile phone system;

[0012] A similarity determination module, used to determine a plurality of similar users and the similarity between the user and each similar user based on the user information and the operation record information of the user in the mobile phone system;

[0013] A second acquisition module is used to acquire multiple display modules set by each similar user in the target software interface;

[0014] The third acquisition module is used to obtain the user's operation screen recording in the target software;

[0015] A video processing module, used to determine the user's attention to each module in the target software using a video processing model based on the screen recording of the user's operation in the target software;

[0016] A construction module, for constructing a knowledge graph, wherein the knowledge graph includes a plurality of nodes and a plurality of edges between the plurality of nodes, wherein the plurality of nodes include a plurality of similar user nodes and a user node, wherein each similar user node respectively establishes an edge with the user node, wherein the node feature of each similar user node includes a plurality of display modules set by each similar user in the target software interface, wherein the node feature of the user node includes the user's attention to each module in the target software, and wherein the feature of each similar user node respectively establishing an edge with the user node includes the similarity between the user and each similar user;

[0017] A graph recurrence module is used to process the knowledge graph using a graph recurrence network to determine an adjusted target software interface.

[0018] Furthermore, the video processing model is a long-term and short-term neural network model.

[0019] Furthermore, the user information includes basic information and behavior preference information of the user.

[0020] Furthermore, the input of the graph recurrent network is the knowledge graph, and the output of the graph recurrent network is the adjusted target software interface.

[0021] Furthermore, the similarity determination module is also used to: input the user information and the user's operation record information in the mobile phone system into a similarity determination model to obtain multiple similar users and the similarity between the user and each similar user, and the similarity determination model is a deep neural network model.

[0022] The present invention provides a software interface management method and system, the method comprising obtaining user information and user operation record information in a mobile phone system; determining multiple similar users and the similarity between the user and each similar user based on the user information and the user operation record information in the mobile phone system; obtaining multiple display modules set by each similar user in a target software interface; obtaining a screen recording of the user's operation in the target software; using a video processing model to determine the user's attention to each module in the target software based on the screen recording of the user's operation in the target software; constructing a knowledge graph, the knowledge graph comprising multiple nodes and multiple edges between the multiple nodes, the multiple nodes comprising multiple similar user nodes and a user node, each similar user node respectively establishing an edge with a user node, the node feature of each similar user node comprising multiple display modules set by each similar user in the target software interface, the node feature of the user node comprising the user's attention to each module in the target software, and the feature of each similar user node respectively establishing an edge with the user node comprising the similarity between the user and each similar user; using a graph recurrent network to process the knowledge graph to determine an adjusted target software interface, the method can intelligently adjust the software interface and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a software interface management method provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of a software interface management system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In an embodiment of the present invention, there is provided Figure 1 A software interface management method is shown, the software interface management method comprising steps S1 to S7:

[0026] Step S1, obtaining user information and user operation record information in the mobile phone system.

[0027] The user information includes the user's basic information and behavior preference information. Basic information includes the user's registered name, real name (if provided), age, gender, occupation, location, etc. For example, user Zhang San, male, 25 years old, IT engineer, lives in Shanghai. Behavior preference information includes the user's habits and preferences in the process of using mobile applications, such as commonly used application categories, browsing content, shopping preferences, usage time patterns, etc. For example, Zhang San often browses technology news, likes to use fitness applications at night, and prefers to buy electronic products.

[0028] The user's operation record information in the mobile phone system records all the user's activity tracks on the mobile phone operating system, including but not limited to installed applications, the time, frequency, and duration of application use, opened web pages, search keywords, setting preferences, location access records, etc. Mobile phone operating systems usually have built-in logging functions that can track application usage, system setting changes, location service requests, etc. Developers can integrate into applications through SDK (Software Development Kit) and use these interfaces to obtain user activity data within the application.

[0029] Step S2: determining a plurality of similar users and the similarity between the user and each similar user based on the user information and the user's operation record information in the mobile phone system.

[0030] Similar users refer to other user groups that have high similarity in user information (such as age, interests, usage habits, etc.) and operation records (such as commonly used applications, active time periods, operation preferences). These users may show similar needs and preferences. Similarity is a quantitative indicator used to measure the degree of similarity between two users in information and behavior.

[0031] In some embodiments, machine learning or statistical methods may be used to calculate the similarity between users, such as cosine similarity, Pearson correlation coefficient, K-means clustering, etc.

[0032] In some embodiments, the user information and the user's operation record information in the mobile phone system can be input into a similarity determination model to obtain multiple similar users and the similarity between the user and each similar user, and the similarity determination model is a deep neural network model.

[0033] Step S3, obtaining multiple display modules set by each similar user in the target software interface.

[0034] The target software interface refers to the part of a specific software application that users interact with directly, including various visual components such as function buttons, information display windows, and navigation menus. Display modules are independent units in the software interface designed for specific functions or information display, and they can be arranged and configured according to user needs or preferences.

[0035] Step S4, obtaining a screen recording of the user's operations in the target software.

[0036] Screen recording refers to a video file that records all user interactions in a software application, including visual records of screen touch actions (such as clicking, sliding, and long pressing), page browsing, and use of functional modules.

[0037] Step S5, based on the screen recording of the user's operation in the target software, use the video processing model to determine the user's attention to each module in the target software.

[0038] The operation screen recording directly records the entire process of the user's interaction with the software interface, including clicks, slides, stays and other actions, which are direct reflections of the user's attention distribution. For example, if the user stays in a certain module for a long time, it means that the module has aroused the user's interest or need.

[0039] The video processing model is a long-term neural network model. The long-term neural network model includes a long-term neural network (LSTM, Long Short-Term Memory). The long-term neural network model can process sequence data, capture sequence information, and output results based on the correlation between the previous and next data in the sequence. The long-term neural network model can comprehensively consider the characteristics of the correlation between the user's operation recording sequence data in the target software, and output the user's attention to each module in the target software.

[0040] In some embodiments, the video processing model includes a screen content recognition layer, a user interaction detection layer, an attention heat map generation layer, and a module attention evaluation layer. The screen content recognition layer, the user interaction detection layer, the attention heat map generation layer, and the module attention evaluation layer all include long-term and short-term neural networks.

[0041] The input of the screen content recognition layer is the screen recording video of the user's operation in the target software, and the output of the screen content recognition layer is the various visual modules of the software interface and their position information in each frame.

[0042] The input of the user interaction detection layer is the visual modules of the software interface and their position information in each frame, and the screen recording video of the user's operation in the target software. The output of the user interaction detection layer is the record of the interaction behavior between the user and each module. The record of the interaction behavior between the user and each module includes clicking, sliding, double-clicking, pinching, etc.

[0043] The input of the attention heatmap generation layer is the record of the interaction between the user and each module, and the output of the attention heatmap generation layer is a heatmap of the user's visual attention distribution on the software interface. The main task of the attention heatmap generation layer is to create a visual representation - the "attention heatmap" - based on the records of the interaction between the user and each module of the software interface. This heatmap shows the distribution of the user's attention on different modules during the use of the software. The darker the module color, the more attention it attracts.

[0044] The input of the module attention evaluation layer is the heat map of the user's visual attention distribution on the software interface, and the output of the module attention evaluation layer is the user's attention to each module in the target software. Each layer focuses on solving specific problems, such as the screen content recognition layer specializes in video content analysis, while the user interaction detection layer focuses on identifying user interaction behaviors. This specialized design helps improve the efficiency and accuracy of each step.

[0045] Step S6, constructing a knowledge graph, wherein the knowledge graph includes multiple nodes and multiple edges between the multiple nodes, the multiple nodes include multiple similar user nodes and one user node, each similar user node establishes an edge with the user node respectively, the node characteristics of each similar user node include multiple display modules set by each similar user in the target software interface, the node characteristics of the user node include the user's attention to each module in the target software, and the characteristics of each similar user node establishing an edge with the user node include the similarity between the user and each similar user.

[0046] The knowledge graph includes multiple nodes and multiple edges between the multiple nodes. The knowledge graph is a data structure consisting of two parts: nodes and edges. In some embodiments, a graph database (such as Neo4j) or a graph algorithm library (such as NetworkX) can be used to store and manage the constructed knowledge graph.

[0047] By building a knowledge graph, user behavior data (such as operation records and attention) can be combined with the behavior data of similar users to form rich contextual information. This data enrichment process helps to more deeply understand the user's needs and preferences for the software interface, providing a solid foundation for personalized interface adjustment.

[0048] Step S7, using a graph recurrent network to process the knowledge graph to determine an adjusted target software interface.

[0049] Graph Recurrent Network (GRN) is designed for processing graph structured data. Graph recurrent network can capture the complex dependencies between nodes in the graph and learn the overall characteristics of the graph by passing information between nodes. Graph recurrent network is used to analyze user preference patterns in knowledge graphs to determine the adjusted target software interface.

[0050] By using graph recurrent networks, we can learn each user's unique preferences and behavioral characteristics from the knowledge graph, and thus personalize the target user interface. The graph recurrent network can predict the target user's possible optimal interface layout based on the similarity between the user and similar users, as well as other users' preferences for module settings. This personalized prediction not only takes into account the attention of a single user, but also integrates the commonalities of similar groups, improving the accuracy and practicality of interface adjustments.

[0051] Based on the same inventive concept, Figure 2 A schematic diagram of a software interface management system provided by an embodiment of the present invention, the software interface management system includes:

[0052] The first acquisition module 21 is used to acquire user information and user operation record information in the mobile phone system;

[0053] A similarity determination module 22, configured to determine a plurality of similar users and a similarity between the user and each similar user based on the user information and the user's operation record information in the mobile phone system;

[0054] A second acquisition module 23 is used to acquire multiple display modules set by each similar user in the target software interface;

[0055] The third acquisition module 24 is used to obtain the screen recording of the user's operation in the target software;

[0056] A video processing module 25, configured to determine the user's attention to each module in the target software using a video processing model based on the screen recording of the user's operation in the target software;

[0057] A construction module 26 is used to construct a knowledge graph, wherein the knowledge graph includes a plurality of nodes and a plurality of edges between the plurality of nodes, wherein the plurality of nodes include a plurality of similar user nodes and a user node, wherein each similar user node respectively establishes an edge with the user node, wherein the node feature of each similar user node includes a plurality of display modules set by each similar user in the target software interface, wherein the node feature of the user node includes the user's attention to each module in the target software, and wherein the feature of each similar user node respectively establishing an edge with the user node includes the similarity between the user and each similar user;

[0058] The graph loop module 27 is used to use a graph loop network to process the knowledge graph and determine an adjusted target software interface.

Claims

1. A software interface management method, characterized in that: include: Obtain user information and user operation records in the mobile phone system; Determine a plurality of similar users and the similarity between the user and each similar user based on the user information and the operation record information of the user in the mobile phone system; Acquire multiple display modules set by each similar user in the target software interface; Obtain screen recordings of users’ operations in the target software; Determine the user's attention to each module in the target software using a video processing model based on the user's operation screen recording in the target software; Constructing a knowledge graph, wherein the knowledge graph includes a plurality of nodes and a plurality of edges between the plurality of nodes, wherein the plurality of nodes include a plurality of similar user nodes and a user node, wherein each similar user node respectively establishes an edge with the user node, wherein the node feature of each similar user node includes a plurality of display modules set by each similar user in the target software interface, wherein the node feature of the user node includes the user's attention to each module in the target software, and wherein the feature of each similar user node respectively establishing an edge with the user node includes the similarity between the user and each similar user; The knowledge graph is processed using a graph recurrent network to determine an adjusted target software interface.

2. The software interface management method according to claim 1, characterized in that: The video processing model is a long-term and short-term neural network model.

3. The software interface management method according to claim 1, characterized in that: The user information includes basic information and behavior preference information of the user.

4. The software interface management method according to claim 1, characterized in that: The input of the graph recurrent network is the knowledge graph, and the output of the graph recurrent network is the adjusted target software interface.

5. The software interface management method according to claim 1, characterized in that: The determining of multiple similar users and the similarity between the user and each similar user based on the user information and the user's operation record information in the mobile phone system includes: inputting the user information and the user's operation record information in the mobile phone system into a similarity determination model to obtain multiple similar users and the similarity between the user and each similar user, wherein the similarity determination model is a deep neural network model.

6. A software interface management system, characterized in that: include: The first acquisition module is used to acquire user information and user operation record information in the mobile phone system; A similarity determination module, used to determine a plurality of similar users and the similarity between the user and each similar user based on the user information and the operation record information of the user in the mobile phone system; A second acquisition module is used to acquire multiple display modules set by each similar user in the target software interface; The third acquisition module is used to obtain the user's operation screen recording in the target software; A video processing module, used to determine the user's attention to each module in the target software using a video processing model based on the screen recording of the user's operation in the target software; A construction module, for constructing a knowledge graph, wherein the knowledge graph includes a plurality of nodes and a plurality of edges between the plurality of nodes, wherein the plurality of nodes include a plurality of similar user nodes and a user node, wherein each similar user node respectively establishes an edge with the user node, wherein the node feature of each similar user node includes a plurality of display modules set by each similar user in the target software interface, wherein the node feature of the user node includes the user's attention to each module in the target software, and wherein the feature of each similar user node respectively establishing an edge with the user node includes the similarity between the user and each similar user; A graph recurrence module is used to process the knowledge graph using a graph recurrence network to determine an adjusted target software interface.

7. The software interface management system according to claim 6, characterized in that: The video processing model is a long-term and short-term neural network model.

8. The software interface management system according to claim 6, characterized in that: The user information includes basic information and behavior preference information of the user.

9. The software interface management system according to claim 6, characterized in that: The input of the graph recurrent network is the knowledge graph, and the output of the graph recurrent network is the adjusted target software interface.

10. The software interface management system according to claim 6, characterized in that: The similarity determination module is also used to: input the user information and the user's operation record information in the mobile phone system into a similarity determination model to obtain multiple similar users and the similarity between the user and each similar user, and the similarity determination model is a deep neural network model.

Images