Systems, methods, and computer program products for generating customized user interfaces to accomplish tasks

Abstract

Disclosed herein are system, method, and computer program product embodiments for generating a customized user interface (UI) to complete a task. One embodiment operates by obtaining contextual information associated with a user and an application on a user device operated by the user, wherein the application includes a plurality of UI elements. Then, based on the contextual information and a predictive model, it is determined that the user is attempting to complete a first task within the application. The embodiment further operates by obtaining a minimum set of UI elements required for the first task. Furthermore, the embodiment operates by transmitting a first customized UI including the minimum set of UI elements for the first task to the user device for display to the user.

Description

Background Art

[0001] It is not uncommon for software applications to have one or more workflows to perform one or more tasks. During each workflow, a user may be presented with a large number of user interface (UI) elements (e.g., GUI widgets) distributed across multiple screens. To complete a task, the user may manipulate only some UI elements and not others. Furthermore, different tasks may require the user to manipulate different UI elements.

[0002] Because the user must determine which UI elements need to be manipulated, a large number of UI elements may increase the time required for the user to complete a task. In addition, a large number of UI elements may cause errors due to the user manipulating UI elements that are not required to complete the task. In addition, because UI elements that do not need to be manipulated by the user to complete the task are still transmitted to the user's computing device, a large number of UI elements may cause excessive network bandwidth consumption. Regardless, workflows and UI elements remain a common way for users to complete tasks in software applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The accompanying drawings are incorporated herein and form a part of the specification.

[0004] Figure 1 A block diagram of a system according to one or more embodiments is shown.

[0005] Figure 2 A flowchart for determining a minimum set of UI elements required to complete a task is shown in accordance with one or more embodiments.

[0006] Figure 3 An example of determining a minimum set of UI elements required to complete a task is shown in accordance with one or more embodiments.

[0007] Figure 4 A flow diagram is shown for generating a custom UI for completing a prediction task according to one or more embodiments.

[0008] Figure 5 An example of a custom UI for completing a task and a preview image of another custom UI for completing a different task are shown in accordance with one or more embodiments.

[0009] Figure 6 is an example computer system for implementing various embodiments.

[0010] In the drawings, like reference numbers generally refer to the same or similar elements. Additionally, generally, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. DETAILED DESCRIPTION

[0011] This document provides a system, method, and computer program product embodiment for utilizing machine learning to determine the minimum set of UI elements required to complete an application's task. Specifically, a data structure is used to record which UI elements a user is manipulating, and the data structure is then grouped into clusters using a machine learning model. Each cluster can correspond to a task. The minimum set of UI elements required to complete a task is determined by performing a join operation on the data structures in the cluster corresponding to the task. The clusters can be sorted, and the distance between pairs of clusters representing the similarity between two tasks can also be determined. A custom UI for a task can be generated using the minimum set of UI elements instead of other UI elements, and transmitted to the user's computing device to complete the task. As discussed throughout the disclosure, the disclosed embodiments reduce task completion time, reduce task errors, and reduce bandwidth consumption on existing systems and processes.

[0012] Also provided herein are system, method, and computer program product embodiments that determine and prioritize tasks a user is attempting to complete based on contextual information. A custom UI can be generated for each identified task. Each custom UI has the minimum number of UI elements required to complete the corresponding task. These custom UIs can be transmitted to a user device operated by the user. One or more images of the custom UIs can also be generated, rather than the custom UIs themselves, and transmitted to the user device as previews of the custom UIs.

[0013] Figure 1 A system 100 is shown according to one or more embodiments. Figure 1 As shown, system 100 has multiple components, including, for example, an event listener 102, a machine learning model 104, a UI generator 106, a cluster management engine 108, a prediction model 180, and a repository 160. Each of these components (102, 104, 106, 108, 160, 180) can be executed on the same computing device (e.g., a server, a mainframe, a desktop personal computer (PC), a laptop, a tablet PC, a smartphone, etc.), or on different computing devices connected via a network with wired and / or wireless segments. These components (102, 104, 106, 108, 120, 180) can form at least a portion of an application (e.g., an email application, a customer relationship management application, a social media application, an image processing application, a word processing application, an encryption application, a network (web) browser, etc.). In addition, a portion of the application can reside and / or execute on a user's computing device (e.g., user device 1 122A, user device X 122X). The user device 122 may correspond to a desktop PC, a laptop computer, a tablet PC, a smartphone, or the like.

[0014] The application is associated with multiple UI elements 168 (e.g., UI element 1 168A, UI element Z 168Z). Each UI element 168 can correspond to a text box, a set of radio buttons, a drop-down list, a slider, or any GUI widget that can be manipulated by the user. The UI elements 168 can be transmitted to the user device 122 and then displayed across multiple screens. In order to complete different tasks, the user can be required to manipulate different subsets of the UI elements 168. Example tasks include updating records, configuring options on outgoing emails, configuring user permissions, creating or editing steps in a workflow, updating a product catalog, creating new records, configuring database fields, creating or editing components within the application interface builder, etc. Therefore, manipulating the UI elements 168 can update fields in a table, configure settings on outgoing emails, update database fields or schemas, execute processes on another machine (also known as a server), schedule processes to be executed later, etc.

[0015] In one or more embodiments, the event listener 102 is configured to track which UI elements 168 the user is manipulating while completing a task within the application. The event listener 102 may record the identifiers of the manipulated UI elements in a data structure 162 (e.g., a vector, a linked list, an array, etc.). Additionally or alternatively, the manipulated UI elements themselves may be detected to update the data structure 162. For example, data structure 1 162A may include the identifiers of UI elements manipulated while the user is completing a task. Another data structure (not shown) may include the identifiers of UI elements manipulated while different users are completing the same task. Data structure N 162N may include the identifiers of UI elements manipulated while the same user or different users are completing different tasks. The identifiers of the UI elements in the data structure may be sorted based on the order in which the UI elements appear in the workflow. For example, in the case of a multi-screen workflow, the identifier of the UI element that appears on the first screen will be placed before the identifier of the UI element that appears on the last screen. Similarly, the identifier of the UI element that appears at the top of the screen will be placed before the identifier of the UI element that appears at the bottom of the same screen. Additionally or alternatively, the order in which the manipulated UI elements appear in the workflow can be stored in data structure 162 separately from the identification of the manipulated UI elements. Additionally or alternatively, the order in which the user manipulates the UI elements can be stored in the data structure. If the user manipulates the UI element multiple times to complete a task, the identification of the UI element can appear more than once in data structure 162. In one or more embodiments, the identification of the screen on which the manipulated UI element appears (e.g., screen 3 in a 5-screen workflow) is also stored in the data structure.

[0016] In one or more embodiments, the event listener 102 stores a timestamp 164 associated with each data structure 162. The timestamp 164 reflects the time (e.g., year, month, day, hour, minute, second, etc.) when the task associated with the data structure 162 was completed. For example, timestamp 1 164A reflects the time when the task associated with data structure 1 162A was completed. Similarly, timestamp N 164N reflects the time when the task associated with data structure N 162N was completed.

[0017] In one or more embodiments, machine learning model 104 is configured to group data structures 162 into clusters based on the UI elements identified in each data structure 162. Machine learning model 104 can utilize any type of clustering algorithm (e.g., centroid-based clustering, density-based clustering, hierarchical clustering, distribution-based clustering, etc.) to group data structures 162. Each cluster can correspond to one of the tasks.

[0018] In one or more embodiments, the cluster management engine 108 is configured to calculate age statistics for the cluster. For example, the age statistics for the cluster can correspond to the average age of the data structures in the cluster, as determined from the timestamps 164. As another example, the age statistics for the cluster can correspond to the age of the youngest data structure in the cluster or the standard deviation of the ages of the data structures in the cluster, as determined from the timestamps 164.

[0019] In one or more embodiments, the cluster management engine 108 is configured to calculate the distance between pairs of clusters. For example, the distance between two clusters can be the distance from the center of one cluster to the center of the other cluster. A smaller distance indicates that the two tasks corresponding to the two clusters are similar. Similarly, a larger distance indicates that the two tasks corresponding to the two clusters are unrelated. These distances (not shown) can also be stored in the repository 160. The closest cluster (i.e., the cluster with the shortest distance) can be called a nearest neighbor.

[0020] In one or more embodiments, the prediction model 180 is configured to determine one or more tasks in an application that a user is attempting to complete. The prediction model 180 can implement machine learning to determine one or more tasks in an application that a user is attempting to complete. The prediction model 180 can determine one or more tasks based on contextual information associated with the application and the user. For example, the contextual information can include the time, day, week, quarter, or start / end of the quarter when the user accesses the application. As another example, the contextual information can include objects that the user has recently interacted with (e.g., tables, variables, files, etc.), objects that have been created by the user and have changed since the user last used the application, and objects with timestamps close to the current time / date. As another example, the contextual information can include the most recent search queries requested by the user. As another example, the contextual information can include the device the user is using (desktop PC, smartphone, tablet, etc.), the available bandwidth of their network connection (e.g., 3G cellular connection or gigabit fiber connection), or their absolute geographic location or their relative geographic location relative to the computing device transmitting the UI. As yet another example, the contextual information can include other users with similar job positions or titles to the user's, other users the user follows (e.g., on social media), etc.

[0021] In one or more embodiments, the prediction model 180 is configured to sort one or more determined tasks. According to the prediction model 180, the task with the highest ranking corresponds to the task that the user is most likely to attempt to complete. The remaining determined tasks have a lower ranking (i.e., according to the prediction model 180, the user is unlikely to attempt to complete the remaining tasks). As described above, each task can correspond to a cluster. The sorting of tasks can be based on the cardinality of the corresponding cluster (i.e., the number of data structures 162 in the cluster) relative to the cardinality of other clusters (corresponding to other tasks). For example, the more data structures in the cluster, the higher the sorting of the corresponding task. Additionally or alternatively, the sorting of tasks can depend on the age statistics of the cluster of tasks relative to the age statistics of other clusters (corresponding to other tasks). Tasks corresponding to clusters with young data structures can be sorted higher than tasks corresponding to clusters with older data structures.

[0022] As described above, each task can correspond to a cluster, and the distance between clusters can be calculated. In one or more embodiments, at least some rankings can also be based on these distances. For example, assume that cluster A, cluster B, and cluster C correspond to task A, task B, and task C, respectively. In addition, assume that the distance between cluster A and cluster C is shorter than the distance between cluster A and cluster B. If it is determined based on contextual information that the user is most likely trying to complete task A, task A will have the highest ranking. Since cluster C is the nearest neighbor of cluster A corresponding to task A, task C will be assigned the next highest ranking. Since cluster B is farther away from cluster A than cluster C, task B will be assigned a ranking lower than task C. Although only three tasks / clusters are mentioned, this distance-based ranking scheme can be used for any number of tasks / clusters.

[0023] Ranks 166 associated with tasks may also be stored in repository 160. For example, rank 1 166A and rank Q 166Q correspond to the ranks of task 1 and task Q (not shown), respectively.

[0024] In one or more embodiments, the UI generator 106 is configured to determine the minimum set of UI elements required for a task based on the cluster corresponding to the task. Specifically, the UI generator can perform a join operation on all UI elements identified in all data structures of the cluster. The result of the join operation is the minimum set of UI elements required for the task corresponding to the cluster. For example, minimum UI element set 1 169A and minimum UI element set Q 169Q correspond to the minimum set of UI elements for task 1 and the minimum set of UI elements for task Q, respectively. The join operation can preserve the ordering of UI element identifiers (discussed below). Therefore, the UI elements in the minimum set are ordered as the UI elements in the original multi-screen workflow (i.e., the UI elements on the first screen are before the UI elements on the last screen, the UI elements at the top of the screen are before the UI elements at the bottom of the screen, and so on).

[0025] Those skilled in the art who benefit from the disclosure will understand that during a join operation, two data structures may have conflicting ordering information for UI elements. In such an embodiment, the newer data structure, determined according to the timestamp, controls the ordering. For example, if there are two UI elements from two different data structures in the join that have the same visual / interaction order, then the UI element from the newer data structure will be "above" or "before" the UI element from the older data structure. All subsequent UI element orders can be adjusted accordingly.

[0026] In one or more embodiments, the UI generator 106 is configured to generate a custom UI for a task (e.g., the task with the highest ranking determined by the prediction model 180) based on the minimum set of UI elements 169 required to complete the task. The custom UI includes the minimum set of UI elements for the task, but all other UI elements 168 associated with the application can be excluded. Unlike a multi-screen workflow, a custom UI can be a single screen. The minimum set of UI elements can be positioned in the custom UI based on the order of the UI elements in the minimum set. As described above, this is consistent with the order in which the UI elements appear in the original multi-screen workflow. In one or more embodiments, since the custom UI only includes the minimum set of UI elements (and not all UI elements 168 associated with the application), the custom UI can be suitable for a single screen. The UI generator 106 can transmit the custom UI to a user device operated by the user so that the user can complete the task.

[0027] As described above, the identity of the screen where the manipulated UI element appears (e.g., screen 3 in a 5-screen workflow) can also be stored in the data structure. In one or more embodiments, where the custom UI does not fit on a single screen, the screen identities recorded in the data structure can be used as an additional heuristic to merge the minimal element set into a minimal screen set. The minimal screen set still has fewer screens than the original workflow.

[0028] In one or more embodiments, the UI generator 106 is configured to generate an image of a custom UI for a task. The image can have a low resolution to reduce the storage space required to store the image and / or reduce the network bandwidth required to transmit the image (e.g., to the user device 122). However, the resolution must be high enough to allow for a gestalt perception of the task (e.g., the resolution may need to be high enough so that a viewer (e.g., a user) can recognize one or more UI elements present in the custom UI).

[0029] In one or more embodiments, the UI generator 106 is configured to generate options for display with the custom UI. These options can correspond to images of other custom UIs. These options can be selected by the user. For example, an option selected by the user may trigger a corresponding image of another custom UI to be displayed (as described below).

[0030] Figure 2 A flow chart is shown according to one or more embodiments. Figure 2 The flowchart depicts the process for determining the minimum set of UI elements required to complete a task. Figure 1 The components (102, 104, 106, 108) discussed perform Figure 2In one or more embodiments, one or more steps may be omitted, repeated, and / or replaced with Figure 2 The order shown is different from the order of execution Figure 2 Therefore, the scope of the present invention should not be considered limited to Figure 2 The specific arrangement of the steps shown in .

[0031] In step 205, the UI elements associated with the application are transmitted to the user device and displayed across multiple screens as part of a multi-screen workflow. Each UI element can correspond to a text box, a set of radio buttons, a drop-down list, a slider, or any GUI widget that can be manipulated by the user. In order to complete different tasks in the application, the user operating the user device may be required to manipulate different subsets of UI elements. However, depending on the task the user is completing in the application, the user will not touch certain UI elements. Therefore, different subsets of UI elements are manipulated by the user to complete different tasks.

[0032] In step 210, a data structure is obtained. Each data structure can be a vector, an array, a linked list, etc. Each data structure identifies a subset of UI elements that are manipulated by the user to complete the application task. When the user is completing the task, the user's actions can be monitored (e.g., tracked) to determine which UI elements are being manipulated and which UI elements are not touched by the user. In one or more embodiments, for privacy reasons, only the user's manipulation of the UI elements (e.g., not keystrokes) will be monitored. The identification of the modified UI elements can be stored in the data structure. Different data structures can correspond to different completed tasks and / or different users. The identification of the UI elements in the data structure can be sorted based on the order in which the UI elements appear in the multi-screen workflow (i.e., the identification of the UI element on the first screen precedes the identification of the UI element on the last screen, the identification of the UI element at the top of the screen precedes the identification of the UI element at the bottom of the screen, etc.). A timestamp can be associated with each data structure. The timestamp can reflect the time when the task corresponding to the data structure is completed. A timestamp can also be generated and stored for each interaction with the UI element.

[0033] Those skilled in the art who have the benefit of this disclosure will understand that it is possible for different users to manipulate slightly different subsets of UI elements to complete the same task. Therefore, multiple data structures corresponding to the same task can identify slightly different subsets of UI elements. These variations in the manipulated UI elements are due to certain UI elements being optional for completing the task.

[0034] In step 215, the data structures are grouped into clusters using a machine learning model. For example, the data structures may be grouped using centroid-based clustering, density-based clustering, hierarchical clustering, distribution-based clustering, and the like. Clustering effectively identifies which data structures correspond to the same task based on the manipulated UI elements identified in the data structures. As described above, since manipulation of some UI elements is optional, data structures corresponding to the same task may identify slightly different subsets of UI elements.

[0035] In step 220, a cluster of data structures is selected. The cluster is selected from clusters that have not yet been selected. For example, the cluster can be randomly selected. As another example, the cluster with the largest cardinality (i.e., the largest number of data structures) can be selected. As yet another example, the cluster with the smallest cardinality can be selected. As described above, each cluster corresponds to a task.

[0036] In step 225, the minimum set of UI elements required to complete the task corresponding to the selected cluster is determined. In one or more embodiments, determining the minimum set of UI elements includes performing a join operation on a subset of the UI elements identified in all data structures in the selected cluster. The result of this join operation is the minimum set of UI elements required to complete the task corresponding to the selected cluster. The join operation can preserve the ordering of the UI element identifiers (discussed below). Thus, the UI elements in the minimum set are ordered as the UI elements in the original multi-screen workflow (i.e., UI elements on the first screen are before UI elements on the last screen, UI elements at the top of the screen are before UI elements at the bottom of the screen, and so on).

[0037] In step 230, it is determined whether one or more unselected clusters still exist. When it is determined that at least one unselected cluster still exists, the process returns to step 220. When it is determined that all clusters have been selected and therefore the minimum set of UI elements required to complete all tasks corresponding to the clusters has been determined, the process proceeds to step 235.

[0038] In step 235, the distance between the cluster pairs is determined. For example, the distance between two clusters can be the distance from the center of one cluster to the center of the other cluster. A smaller distance indicates that the two tasks corresponding to the two clusters are similar. Similarly, a larger distance indicates that the two tasks corresponding to the two clusters are unrelated. Step 235 may be optional.

[0039] Still referring to step 235, age statistics for the cluster are determined. For example, as determined from the timestamps, the age statistics for the cluster can correspond to the average age of the data structures in the cluster. As another example, as determined from the timestamps, the age statistics for the cluster can correspond to the age of the youngest data structure in the cluster or the standard deviation of the ages of the data structures in the cluster.

[0040] After step 235, a custom UI can be generated to complete the task in the application. As determined in step 225, the custom UI includes the minimum UI elements required to complete the task, but does not include other UI elements associated with the application. The minimum set of UI elements can be positioned in the custom UI based on the order of the UI elements in the minimum set. As described above, this ordering is consistent with the order in which the UI elements appear in the original multi-screen workflow. The custom UI can be transmitted to the user device for display, allowing the user operating the user device to manipulate the UI elements and complete the task.

[0041] Those skilled in the art who have the benefit of this detailed description will appreciate that because the custom UI has only the minimum set of UI elements required to complete the task, rather than all UI elements, less network bandwidth is required to transmit the custom UI to the user's device. Furthermore, because the custom UI has only the minimum set of UI elements, the user wastes less time determining which UI elements need to be manipulated, and thus can complete the task more quickly. Furthermore, because the custom UI has only the minimum set of UI elements, fewer errors may occur when completing the task because the user does not manipulate UI elements that should not be touched to complete the task.

[0042] Figure 3 An example according to one or more embodiments is shown. Specifically, Figure 3 Shown Figure 2 An example of the processing described in . Figure 3 As shown, there are multiple data structures: data structure 1 302A, data structure 2 302B, data structure 3 302C, data structure 4 304A, and data structure 5 304B. Each of these data structures corresponds to a task. In addition, each of these data structures includes the identification of the UI elements manipulated to complete the task. For example, data structure 2 302B includes the identification of UI elements A, B, and C. UI elements A, B, and C are manipulated by a user to complete a task in an application. As another example, data structure 5 304B includes the identification of UI elements B, C, and X 304B. UI elements A, B, and C are manipulated by the same or different users to complete a task in an application. If data structure 2 302B and data structure 5 304B correspond to the same user, then these two data structures may be from two different sessions and therefore have different timestamps. If these two data structures correspond to two different users, then these two data structures may have the same timestamp (i.e., the two users completed their respective tasks at the same time) or different timestamps.

[0043] The data structures are grouped into cluster A 300A or cluster B 300B using a machine learning model. As shown, data structure 1 302A, data structure 2 302B, and data structure 3 302C belong to cluster A 300A, while data structure 4 304A and data structure 5 304B belong to cluster B 300B. Data structures belonging to the same cluster are considered to correspond to the same task.

[0044] Assume that cluster A 300A corresponds to task A 398 in the application. To determine the minimum set of UI elements required for task A 398, a join operation is performed on all data structures 302 in cluster A 300A. The result of the join operation, and thus the minimum set of UI elements required to complete task A 398, is UI elements A, B, C, D, and E (e.g., Figure 3 To complete Task A 398, a custom UI having UI elements A, B, C, D, and E may be generated and transmitted to a user device. A user operating the user device may manipulate UI elements A, B, C, D, and E to complete Task A 398.

[0045] Assume that cluster B 300B corresponds to task B 399 in the application. To determine the minimum set of UI elements required for task B 399, a join operation is performed on all data structures 304 in cluster B 300B. The result of the join operation, and thus the minimum set of UI elements required to complete task B 399, is UI elements B, C, X, and Y (e.g., Figure 3 To complete task B 399 , a custom UI having UI elements B, C, X, and Y may be generated and transmitted to a user device. A user operating the user device may manipulate UI elements B, C, X, and Y to complete task B 399 .

[0046] Figure 4 A flow chart is shown according to one or more embodiments. Figure 4 The flowchart depicts the process for generating a custom UI to complete the prediction task. Figure 1 The components (102, 104, 106, 108, 180) discussed perform Figure 4 One or more steps in Figure 2 Execute after the steps in Figure 4 In one or more embodiments, one or more steps may be omitted, repeated, and / or replaced with Figure 4 The order shown is different from the order of execution Figure 4 Therefore, the scope of the present invention should not be considered limited to Figure 4 The specific arrangement of the steps shown in .

[0047] In step 405, contextual information associated with the user operating the application on the user device is obtained. For example, the contextual information may include the time, day, week, quarter, quarter start / end when the user accessed the application. As another example, the contextual information may include objects that the user recently interacted with (e.g., tables, variables, files, etc.), objects created by the user that have changed since the user last used the application, objects with timestamps close to the current time / date. As another example, the contextual information may include recent search queries requested by the user. As yet another example, the contextual information may include other users with employment positions or titles similar to the user's position / title, other users that the user follows (e.g., on social media), etc.

[0048] In step 410, one or more tasks that the user is attempting to complete are determined based on the contextual information. These tasks are essentially predictions performed by a predictive model based on the contextual information. The tasks may also be ranked. According to the predictive model, the tasks with the highest ranking correspond to the tasks that the user is most likely to attempt to complete. The remaining identified tasks have lower rankings (i.e., according to the predictive model, the user is less likely to attempt to complete the remaining tasks).

[0049] As described above, each task can correspond to a cluster. Also as described above, the cardinality of each cluster, the age statistics of each cluster, and the distances between clusters can be calculated. The ranking of tasks can be based on the cardinality of the corresponding cluster, the age statistics of the corresponding cluster, and / or the distances between the corresponding cluster and other clusters (corresponding to other tasks).

[0050] In step 415, the minimum set of UI elements required to complete one or more tasks may be obtained. For example, the minimum set of UI elements required to complete the highest-ranked task may be obtained, the minimum set of UI elements required to complete the second-highest-ranked task may be obtained, and so on. Figure 2 Shown is a minimal set of UI elements that handle predetermined tasks.

[0051] In step 420, a custom UI is generated for the highest-ranked task. The custom UI includes the minimum set of UI elements required to complete the task, but does not include other UI elements of the application. The minimum set of UI elements can be positioned in the custom UI based on the order of the UI elements in the minimum set (as described above). The custom UI can be transmitted to the user device.

[0052] In step 425, a custom UI for the second highest ranked task is generated. The custom UI includes the minimum set of UI elements required to complete the second highest ranked task, but does not include other UI elements of the application. An image of the custom UI can be generated. The image can have a low resolution to reduce the space required to store the image and / or reduce the network bandwidth required to transmit the image. However, the image can have a sufficiently high resolution so that a viewer (e.g., a user) can recognize the UI elements present in the custom UI. The image of the custom UI can be transmitted to the user device for display to the user.

[0053] Step 425 may be repeated for the third highest ranked task, the fourth highest ranked task, and so on, until some cutoff point.

[0054] In one or more embodiments, when the custom UI associated with the highest-ranked task is displayed to the user on the user device, one or more options may also be displayed to the user on the user device. Each option may correspond to a task other than the highest-ranked task. For example, a first option may correspond to the second-highest-ranked task. A second option may correspond to the third-highest-ranked task. If the user selects the second option, an image of the custom UI for the third-highest-ranked task may be displayed. The image may be displayed together with (e.g., on top of) the custom UI associated with the highest-ranked task or separately from it.

[0055] Those skilled in the art who have benefited from this detailed description will understand that the highest ranked task may not be the task that the user is trying to complete. In other words, the prediction of the predictive model is incorrect. By presenting these options, the user can effectively preview other custom UIs and select the correct custom UI (i.e., the custom UI corresponding to the task that the user is actually trying to complete). The custom UI corresponding to the correct task selected by the user can then be requested (e.g., from a UI generator), generated, and / or displayed to the user in order to complete the task. The user's selection can also be used as feedback to update the predictive model.

[0056] Those skilled in the art having the benefit of this detailed description will also appreciate that by initially transmitting an image rather than the actual custom UI, less network bandwidth is required (as opposed to transmitting the custom UI for all tasks).

[0057] In one or more embodiments, a prompt may be displayed to the user on the user device requesting confirmation that the highest-ranked task is indeed the task the user is attempting to complete. The prompt may be displayed to all users. Alternatively, the prompt may be displayed only to expert users. The prompt may be optional. Responses to the prompt, whether positive or negative, may serve as feedback to improve the predictive model.

[0058] Figure 5 An example of a custom UI for completing a task and an image of another custom UI for completing a different task are shown in accordance with one or more embodiments. Figure 5 In the example, it is assumed that task A has the highest ranking. In other words, it is predicted that the user operating the user device is trying to complete task A. Figure 5 As shown, the custom UI for task A includes a minimum set of UI elements 510 required to complete task A. The custom UI for task A may have been generated by the UI generator 106 and transmitted to the user device (see above). Figure 4 discuss).

[0059] Figure 5 Two options 520 are also shown. Options 520 are displayed at the top of the custom UI for Task A. One option asks the user if they are trying to complete Task Q. Task Q may have the second highest ranking. The other option asks the user if they are trying to complete Task Z. Task Z may have the third highest ranking. By selecting the first option, an image 525 of the custom UI for Task Q is displayed. This image 525 may be generated by the UI generator 106 and transmitted to the user device (see above). Figure 4 discuss).

[0060] If the user is not actually trying to complete Task A (i.e., the prediction model is incorrect), the user can preview other customized UIs (e.g., for Task Q and Task Z) and select a customized UI for the task the user is actually trying to complete. The user can manipulate UI elements on the customized UI to complete the task the user is actually trying to complete.

[0061] For example, one or more computer systems, e.g. Figure 6 The computer system 600 shown can be used to implement various embodiments. For example, the computer system 600 can be used to implement Figure 2 and Figure 4 The processing shown. For example, the computer system 600 can be configured to determine the minimum set of UI elements required to complete the task. The computer system 600 can also be configured to generate a customized UI for completing the prediction task. The computer system 600 can be any computer capable of performing the functions described herein.

[0062] Computer system 600 may be any known computer capable of performing the functions described herein.

[0063] Computer system 600 includes one or more processors (also called central processing units or CPUs), such as processor 604. Processor 604 is connected to a communication infrastructure or bus 606.

[0064] One or more processors 604 may each be a graphics processing unit (GPU). In an embodiment, a GPU is a processor that is a specialized electronic circuit designed to process mathematically intensive applications. A GPU may have a parallel architecture that is efficient for parallel processing of large blocks of mathematically intensive data, such as computer graphics applications, images, videos, and the like. One or more GPUs may be dedicated to performing the machine learning tasks discussed above (e.g., clustering, prediction, etc.).

[0065] The computer system 600 also includes user input / output devices 603 , such as a monitor, keyboard, pointing device, etc., that communicate with the communication infrastructure 606 through a user input / output interface 602 .

[0066] The computer system 600 also includes a main memory or primary storage 608, such as random access memory (RAM). The main memory 608 may include one or more levels of cache. The main memory 608 stores control logic (ie, computer software) and / or data therein.

[0067] The computer system 600 may also include one or more secondary storage devices or secondary memory 610. For example, the secondary memory 610 may include a hard drive 612 and / or a removable storage device or drive 614. The removable storage drive 614 may be a floppy disk drive, a tape drive, an optical drive, an optical storage device, a tape backup device, and / or any other storage device / drive.

[0068] Removable storage drive 614 can interact with removable storage unit 618. Removable storage unit 618 includes a computer-usable or computer-readable storage device having computer software (control logic) and / or data stored thereon. Removable storage unit 618 can be a floppy disk, magnetic tape, optical disk, DVD, optical storage disk, and / or any other computer data storage device. Removable storage drive 614 reads from and / or writes to removable storage unit 618 in a well-known manner.

[0069] According to an exemplary embodiment, the secondary memory 610 may include other devices, tools, or other means for allowing the computer system 600 to access computer programs and / or other instructions and / or data. Such devices, tools, or other means may include, for example, a removable storage unit 622 and an interface 620. Examples of the removable storage unit 622 and the interface 620 may include a program cartridge and a cartridge interface (such as those found in video game devices), a removable memory chip (such as an EPROM or PROM) and an associated socket, a memory stick and a USB port, a memory card and an associated memory card slot, and / or any other removable storage unit and an associated interface.

[0070] The computer system 600 may also include a communication or network interface 624. The communication interface 624 enables the computer system 600 to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referenced by reference numeral 628). For example, the communication interface 624 can allow the computer system 600 to communicate with a remote device 628 via a communication path 626, which can be wired and / or wireless and can include any combination of a LAN, a WAN, the Internet, etc. Control logic and / or data can be transmitted to and from the computer system 600 via the communication path 626.

[0071] In an embodiment, a tangible, non-transitory device or article of manufacture comprising a tangible, non-transitory computer-usable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system 600, main memory 608, secondary memory 610, and removable storage units 618 and 622, as well as tangible articles of manufacture embodying any combination of the foregoing. When executed by one or more data processing devices (e.g., computer system 600), such control logic causes such data processing devices to operate as described herein.

[0072] Based on the teachings contained in the disclosure, it is obvious to those skilled in the art how to use Figure 6 It will be apparent that data processing devices, computer systems, and / or computer architectures other than those shown can be used to make and use the embodiments of the disclosure. Specifically, the embodiments can be implemented using software, hardware, and / or operating systems other than those described herein.

[0073] It should be understood that the detailed description section, and not any other section, is intended to be used to interpret the claims. The other sections may set forth one or more, but not all, exemplary embodiments as contemplated by the inventors, and are therefore not intended to limit the disclosure or the appended claims in any way.

[0074] Although the disclosure describes exemplary embodiments of exemplary fields and applications, it should be understood that the disclosure is not limited thereto. Other embodiments and modifications thereof are possible and within the scope and spirit of the disclosure. For example, and without limiting the generality of this paragraph, the embodiments are not limited to the software, hardware, firmware and / or entities shown in the figures and / or described herein. In addition, the embodiments (whether or not explicitly described herein) have significant practical value for fields and applications beyond the examples described herein.

[0075] Embodiments have been described herein by means of functional building blocks illustrating implementations of specified functions and relationships thereof. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries may be defined so long as the specified functions and relationships thereof (or their equivalents) are appropriately performed. Additionally, alternative embodiments may perform functional blocks, steps, operations, methods, etc., in a different order than that described herein.

[0076] References herein to "one embodiment," "an embodiment," "an example embodiment," or similar phrases indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is within the knowledge of those skilled in the relevant art to incorporate such feature, structure, or characteristic into other embodiments, whether or not explicitly mentioned or described herein. Additionally, the expressions "coupled" and "connected," and their derivatives, may be used to describe some embodiments. These terms are not necessarily synonymous with each other. For example, the terms "connected" and / or "coupled" may be used to describe some embodiments to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0077] The breadth and scope of the disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

1. A method comprising: obtaining, by one or more computing devices, contextual information associated with a user and an application on a user device operated by the user, wherein the application includes a plurality of user interface elements; determining, by the one or more computing devices based on the contextual information and the predictive model, that the user is attempting to complete a first task within the application, wherein the first task corresponds to a selected cluster of data structures from a plurality of data structures, each of the plurality of data structures including an identification of a corresponding user interface element that is manipulated to complete the task; obtaining, by the one or more computing devices, a minimum set of user interface elements required for the first task corresponding to the selected cluster of data structures, wherein the minimum set of user interface elements is determined by performing a join operation on a first data structure corresponding to a first set of user interface elements from the selected cluster of data structures and a second data structure corresponding to a second set of user interface elements from the selected cluster of data structures; and A first customized user interface including the minimum set of user interface elements for the first task is transmitted by the one or more computing devices to the user device for display to the user.

2. The method according to claim 1, further comprising: transmitting a prompt for display, the prompt requesting confirmation that the user is attempting to complete the first task; receiving a response to the prompt; and Based on responses to the prompts, the predictive model is updated.

3. The method according to claim 1, further comprising: Obtaining a minimum set of user interface elements required to complete the second task; generating an image of a second custom user interface, the second custom user interface including a minimum set of user interface elements required to complete the second task; and transmitting the image to the user equipment, wherein in response to the user selecting an option corresponding to the second task, displaying the image to the user, The options are displayed together with the first customized user interface.

4. The method according to claim 3, further comprising: obtaining a first age statistic associated with the first task and a second age statistic associated with the second task, wherein the prediction model ranks the first task and the second task based on the first age statistics and the second age statistics, wherein the second task is determined based on the context information and the prediction model, and The first task is ranked higher than the second task.

5. The method according to claim 3, further comprising: The plurality of data structures are obtained by the one or more computing devices, the plurality of data structures comprising: a third data structure that identifies a first subset of a plurality of user interface elements manipulated by a first user while the first user is completing a first task in the application; and a fourth data structure identifying a second subset of the plurality of user interface elements manipulated by a second user while the second user is completing a second task in the application; The plurality of data structures are grouped into a plurality of clusters using a machine learning model being executed on the one or more computing devices, the plurality of clusters comprising: a first cluster corresponding to the first task and including the third data structure; and a second cluster corresponding to the second task and comprising the fourth data structure; Based on the first cluster, determining a minimum set of user interface elements required for the first task; and A minimum set of user interface elements required for the second task is determined based on a subset of the plurality of data structures in the second cluster.

6. The method according to claim 5, further comprising: obtaining a plurality of distances between the plurality of clusters, The generating of the image is based on the second cluster being a nearest neighbor of the first cluster. 7 . The method of claim 1 , wherein the context information includes application objects that have been generated or modified since the user last logged into the application. The method according to claim 1 , wherein the context information includes a current date and application objects having dates within a certain range from the current date.

9. A system comprising: one or more computer processors; a memory storing instructions that, when executed by the one or more computer processors, cause the one or more computer processors to: obtaining contextual information associated with a user and an application on a user device operated by the user, wherein the application includes a plurality of user interface elements; determining, based on the contextual information and the predictive model, that the user is attempting to complete a first task within the application, wherein the first task corresponds to a selected cluster of data structures from a plurality of data structures, each of the plurality of data structures including an identification of a corresponding user interface element that is manipulated to complete the task; obtaining a minimum set of user interface elements required for the first task corresponding to the selected cluster of data structures; determining the minimum set of user interface elements by performing a join operation on a first data structure from the selected cluster of data structures corresponding to a first set of user interface elements and a second data structure from the selected cluster of data structures; and A first customized user interface including the minimum set of user interface elements for the first task is transmitted to the user device for display to the user.

10. The system of claim 9, wherein execution of the instructions further causes the one or more computer processors to: Obtaining a minimum set of user interface elements required to complete the second task; generating an image of a second custom user interface, the second custom user interface including a minimum set of user interface elements required to complete the second task; and transmitting the image to the user device, wherein in response to the user selecting an option corresponding to the second task, displaying the image to the user, The options are displayed together with the first customized user interface.

11. The system of claim 10, wherein executing the instructions further causes the one or more computer processors to: obtaining a first age statistic associated with the first task and a second age statistic associated with the second task, wherein the prediction model ranks the first task and the second task based on the first age statistics and the second age statistics, wherein the second task is determined based on the context information and the prediction model, and The first task is ranked higher than the second task.

12. The system of claim 10, wherein executing the instructions further causes the one or more computer processors to: Obtain the plurality of data structures, the plurality of data structures comprising: a third data structure identifying a first subset of a plurality of user interface elements manipulated by a first user while the first user is completing a first task in the application; and a fourth data structure identifying a second subset of the plurality of user interface elements manipulated by a second user while the second user is completing a second task in the application; The plurality of data structures are grouped into a plurality of clusters using a machine learning model, the plurality of clusters comprising: a first cluster corresponding to the first task and including the third data structure; and a second cluster corresponding to the second task and comprising the fourth data structure; determining a minimum set of user interface elements required for the first task based on the first cluster; and A minimum set of user interface elements required for the second task is determined based on a subset of the plurality of data structures in the second cluster.

13. The system of claim 12, wherein executing the instructions further causes the one or more computer processors to: obtaining a plurality of distances between the plurality of clusters, The generating of the image is based on the second cluster corresponding to the second task being a nearest neighbor of the first cluster corresponding to the first task.

14. The system of claim 9, wherein the context information includes application objects that have been generated or modified since the user last logged into the application. 15 . The system of claim 9 , wherein the context information includes a current date and application objects having dates within a range from the current date.

16. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors, cause the one or more computer processors to: obtaining contextual information associated with a user and an application on a user device operated by the user, wherein the application includes a plurality of user interface elements; determining, based on the contextual information and the predictive model, that the user is attempting to complete a first task within the application, wherein the first task corresponds to a selected cluster of data structures from a plurality of data structures, each of the plurality of data structures including an identification of a corresponding user interface element that is manipulated to complete the task; obtaining a minimum set of user interface elements required for the first task corresponding to the selected cluster of data structures; determining the minimum set of user interface elements by performing a join operation on a first data structure from the selected cluster of data structures corresponding to a first set of user interface elements and a second data structure from the selected cluster of data structures corresponding to a second set of user interface elements; and A first customized user interface including a minimum set of user interface elements for the first task is transmitted to the user device for display to the user.

17. The non-transitory computer-readable medium of claim 16, wherein the instructions further cause the one or more computer processors to: Obtaining a minimum set of user interface elements required to complete the second task; generating an image of a second custom user interface, the second custom user interface including a minimum set of user interface elements required to complete the second task; and transmitting the image to the user device, wherein in response to the user selecting an option corresponding to the second task, displaying the image to the user, The options are displayed together with the first customized user interface.

18. The non-transitory computer-readable medium of claim 17, wherein the instructions further cause the one or more computer processors to: Obtain the plurality of data structures, the plurality of data structures comprising: a third data structure identifying a first subset of a plurality of user interface elements manipulated by a first user while the first user is completing a first task in the application; and a fourth data structure identifying a second subset of the plurality of user interface elements manipulated by a second user while the second user is completing a second task in the application; The plurality of data structures are grouped into a plurality of clusters using a machine learning model, the plurality of clusters comprising: a first cluster corresponding to the first task and including the third data structure; and a second cluster corresponding to the second task and comprising the fourth data structure; determining a minimum set of user interface elements required for the first task based on the first cluster; determining a minimum set of user interface elements required for the second task based on a subset of the plurality of data structures in the second cluster; and obtaining a plurality of distances between the plurality of clusters, The generating of the image is based on the second cluster corresponding to the second task being a nearest neighbor of the first cluster corresponding to the first task.

19. The non-transitory computer-readable medium of claim 16, wherein the context information includes application objects that have been generated or modified since the user last logged into the application. 20 . The non-transitory computer-readable medium of claim 16 , wherein the context information includes a current date and application objects having dates within a range from the current date.

Images