Creating human-readable instructions for user navigation recommendations

Abstract

Techniques are disclosed for generating human-readable explanations (also referred to herein as "reason") for navigation recommendations. Creating a human-readable explanation includes individually selecting words or phrases that are then analyzed, combined, rearranged, modified, or removed to generate a human-readable explanation for the navigation recommendation. A decoder trains a machine learning model to generate human-readable reasons for the navigation recommendation based on (1) past recommendation vectors and (2) past human-readable reasons associated with the recommendation vectors. The system generates a dictionary of human-readable reasons for the recommendation, each entry in the dictionary including (1) a recommendation identifier (ID) associated with the recommended navigation target, (2) a reason identifier (ID) associated with a particular reason for the recommendation, and (3) a human-readable reason associated with the reason ID.

Description

[Technical Field]

[0001] RELATED APPLICATIONS: INCORPORATION BY REFERENCE This application is related to U.S. Patent Application No. 17 / 302,429, filed May 3, 2021, which is incorporated herein by reference.

[0002] The present disclosure relates to creating human readable explanations for user navigation recommendations. In particular, the present disclosure relates to using the products of a neural network that generates user navigation recommendations to create human readable explanations for user navigation recommendations. [Background technology]

[0003] background Various user interface applications provide users with recommendations from which the user can select to navigate to services. To assist users in making selections based on the navigation recommendations, the application may indicate reasons for the navigation recommendations. For example, the application may indicate that the user has made a similar selection in the recent past or that other similar users have made the recommended selection. Typically, applications include only a few categories of reasons that can be provided to users as explanations for the recommendations. A limited, static set of reasons is typically programmed by the developer.

[0004] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Thus, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.

[0005] Embodiments are illustrated by way of example, and not by way of limitation, in the accompanying drawings, in which: It should be noted that references to "an" or "one" embodiment in this disclosure do not necessarily refer to the same embodiment, but rather mean at least one. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 illustrates a system according to one or more embodiments. [Figure 2] FIG. 1 illustrates an example set of operations for generating human-readable reasons for user interface navigation recommendations according to one or more embodiments. [Figure 3] FIG. 1 illustrates an example set of operations for training a machine learning model to generate human-readable reasons for user interface navigation in accordance with one or more embodiments. [Figure 4] FIG. 1 is a block diagram illustrating a computer system according to one or more embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0007] Detailed Description In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding. One or more embodiments may be practiced without these specific details. Features described in one embodiment may be combined with features described in a different embodiment. In some instances, well-known structures and devices are described with reference to block diagram form in order to avoid unnecessarily obscuring the present invention.

[0008] 1.Overview 2. System Architecture 3. Creating human-readable reasons for user interface navigation recommendations 4. Training a sequence-to-sequence machine learning model 5. Computer Networks and Cloud Networks 6. Miscellaneous; Extensions 7. Hardware Overview

[0009] 1.Overview The navigation platform provides users with access to multiple different applications and services. Each application and service may contain many different functions. Each of these functions is a potential navigation target that a user may navigate to within the navigation platform to perform the function.

[0010] One or more embodiments create a human-readable explanation (also referred to herein as a "reason") for the navigation recommendation. Creating a human-readable explanation involves individually selecting words or phrases that are then analyzed, combined, rearranged, modified, or removed to generate the human-readable explanation for the navigation recommendation.

[0011] The system generates navigation recommendations by applying a trained neural network to a set of conditions. The set of conditions may include, for example, previous user activity, a current location in a navigation platform, and a current date and time. In applying the trained neural network, the system generates an initial vector that encodes the set of conditions. The initial vector (also referred to herein as a "layer") is recursively processed to sequentially generate additional vectors (additional "layers") until a final vector is generated that may be mapped to a navigation recommendation. In one embodiment, one of the vectors generated by application of the neural network to generate the navigation recommendation is extracted for use in creating a human-readable explanation for the navigation recommendation. The vector generated by the neural network and extracted for use in creating the human-readable explanation may be referred to herein as a "recommendation vector."

[0012] In one embodiment, the decoder trains a machine learning model to generate human-readable reasons for navigation recommendations based on (1) past recommendation vectors and (2) past human-readable reasons associated with the recommendation vectors. The system generates a dictionary of human-readable reasons for the recommendations, where each entry in the dictionary includes (1) a recommendation identifier (ID) associated with the recommended navigation target, (2) a reason identifier (ID) associated with a particular reason for the recommendation, and (3) a human-readable reason associated with the reason ID. In one or more embodiments, the machine learning model that generates the human-readable reasons for the navigation recommendations is a sequence-to-sequence (seq2seq) machine learning model.

[0013] In one or more embodiments, a machine learning model generates human-readable reasons for navigation recommendations by receiving a recommendation vector as input and generating initial words as output. The recommendation vector and initial words are provided as input to the machine learning model, which generates the next word in the human-readable reasons for the navigation recommendation. The process is repeated until each word in the human-readable reasons for the navigation recommendation has been generated. The words generated by the machine learning model may be represented as embedded vectors associated with words or phrases. A decoder identifies the words or phrases associated with the embedded vectors and generates human-readable reasons for the navigation recommendation.

[0014] Once trained, the machine learning model may generate any number of human-readable reasons for user navigation recommendations customized for a particular user. For example, two users may be provided with different reasons for the same user interface navigation recommendation. Additionally, the same user may be provided with different reasons for the recommendation at different times. Because the human-readable reasons are generated by the machine learning model, the number of human-readable reasons is not limited to the reasons entered into the system by the operator. Additionally, the operator is not required to enter specific rules to display specific human-readable reasons for a particular recommendation. Instead, the machine learning model generates customized human-readable reasons for the customized recommendation based on a set of user attributes and a set of navigation target attributes.

[0015] One or more embodiments described and / or claimed herein may not be included in this General Summary section.

[0016] 2. System Architecture 1 illustrates a system 100 according to one or more embodiments. As shown in FIG. 1, the system 100 includes a navigation platform 110, an encoder 120, a decoder 130, and a data repository 110.

[0017] The navigation platform 110 may be a program or set of programs that provides access to a suite of applications and services. The navigation platform 110 includes a user interface 111 and a navigation target 112.

[0018] In one or more embodiments, user interface 111 refers to hardware and / or software configured to facilitate communication between a user and navigation platform 110. Interface 111 renders user interface elements and receives input via user interface elements. Examples of interfaces include graphical user interfaces (GUIs), command line interfaces (CLIs), tactile interfaces, and voice command interfaces. Examples of user interface elements include check boxes, radio buttons, drop-down lists, list boxes, buttons, toggles, text fields, date and time selectors, command lines, sliders, pages, and forms.

[0019] In an embodiment, different components of interface 111 are specified in different languages. The behavior of user interface elements is specified in a dynamic programming language such as JavaScript. The content of user interface elements is specified in a markup language such as HyperText Markup Language (HTML) or XML User Interface Language (XUL). The layout of user interface elements is specified in a style sheet language such as Cascading Style Sheets (CSS). Alternatively, interface 111 is specified in one or more other languages ​​such as Java, C, or C++.

[0020] Navigation targets 112 include applications, documents, and executable files. Examples of navigation targets include a program for reviewing invoices in an accounting application; a program for storing, editing, and displaying expense reports in an accounting application; a program for storing, editing, and displaying customer and lead contact information in a customer acquisition application; a program for retrieving sales data and generating and displaying sales forecasts in a sales activity application; a program for storing, editing, and displaying project workflow information in a project management application; a program for storing, editing, and displaying employee information in a human resources application; and a program for retrieving and displaying compliance data for compliance with government regulations. The above navigation targets are provided as examples, but they are not an exhaustive list of possible navigation targets. Embodiments encompass any program and any application to which a user may navigate to retrieve information and / or perform functions in the navigation platform. In one or more embodiments, navigation platform 110 generates user interface elements in the GUI of user interface 111 to enable a user to select a user interface element to access navigation target 112.

[0021] Each navigation target 112 is associated with an application 113. Many different navigation targets may belong to the same application. For example, an accounting application may include navigation targets that correspond to different functions in the accounting application, including creating expense reports, creating financial forecasts, creating payroll information, etc. As another example, a human resources application may include navigation targets that include viewing an employee list, viewing and editing employee information, requesting time off, creating employee performance reports, viewing an organizational chart, etc. Each navigation target 112 includes a navigation target attribute.

[0022] In one or more embodiments, navigation platform 110 displays a set of user interface elements in the GUI of user interface 111 to enable a user to navigate to a corresponding set of navigation targets 112. The displayed user interface elements may belong to navigation targets 112 associated with different applications 113. Additionally or alternatively, the displayed user interface elements may belong to different documents or executable files within the same application.

[0023] The encoder 120 retrieves the user attribute data 141 and the navigation target attribute data 142 from the data repository 140. Alternatively, the user attribute data 141 and the navigation target attribute data 142 may be retrieved from the navigation platform 110. For example, a user may log in to the navigation platform 110 and create a user profile that includes the user attribute data. The user attribute data 141 includes data about a particular user. Examples of the user attribute data 141 include job name, role, position, location, and business unit. The above user attributes are provided as examples, but they are not an exclusive list of possible user attributes. Embodiments encompass any information that may be associated with a user to identify the user's attributes. The navigation target attribute data includes data about the navigation target. Examples of navigation target attributes include application stripe, product family, view ID, menu ID, taskflow ID, WebAppName, contextualAreaWidth, RecentItemsID, ​​ObjectCode, CreationDate, LastUpdateLogin, LastUpdataDate, EnterpriseID, ParentID, and SourceFile. The above navigation target attributes are provided as examples, but are not an exclusive list of possible navigation target attributes. Embodiments encompass any information that may be associated with a navigation target to identify the attributes of the navigation target.

[0024] In one or more embodiments, data repository 140 is any type of storage device and / or device for storing data (e.g., a file system, a database, a collection of tables, or any other storage mechanism). Furthermore, data repository 140 may include multiple different storage devices and / or devices. The multiple different storage devices and / or devices may or may not be of the same type and may or may not be located in the same physical location. Furthermore, data repository 140 may be implemented or execute on the same computing system as navigation platform 110, encoder 120, or decoder 130. Alternatively, or additionally, data repository 140 may be implemented or execute on a computing system separate from navigation platform 110, encoder 120, or decoder 130. Data repository 140 may be communicatively coupled to navigation platform 110, encoder 120, or decoder 130 via a direct connection or via a network.

[0025] The information describing user attribute data 141, navigation target attribute data 142, past training data 114, and sorted list 117 may be implemented across any of the components in system 100. However, this information is shown in data repository 140 for clarity and explanation.

[0026] The encoder 120 generates an input vector based on the user attribute data 141 and the navigation target attribute data 142. The encoder may convert the user attribute data 141 and the navigation target attribute data 142 into numerical and vector values. The encoder may concatenate the converted user attribute data 141 and the navigation target attribute data 142 to generate an input vector. The encoder provides the input vector to the machine learning engine 121, which generates the recommendation vector 122. The machine learning engine 121 may include a trained machine learning model that is trained to receive the input vector and generate one or more recommended navigation targets based on the input vector. In one or more embodiments, the machine learning model is a contextual multi-armed bandit type machine learning model.

[0027] In one or more embodiments, the encoder 120 provides the recommendation vector 122 to the recommendation engine 123, which generates one or more recommended navigation targets based on the input vector. The encoder 120 may convert the recommendation vector 122 into a probability distribution using a SoftMax layer before providing the recommendation vector 122 to the recommendation engine 123. The recommendation engine 123 identifies one or more recommended navigation targets for the user based on the information in the recommendation vector 122. Each navigation target is represented by a unique recommendation ID 124. The recommendation engine 123 may generate an ordered list of recommendation IDs 124 for the corresponding list of recommendation vectors 122.

[0028] The encoder 120 also provides the recommendation vector 122 to the decoder 130, which trains another machine learning model to generate human-readable reasons for the user interface navigation recommendations. The recommendation vector 122 is a multidimensional vector having vector values ​​generated by the machine learning engine 121 based on the relationship between the user attribute data, the navigation target attribute data, the recommended user interface navigation, and the reasons for the recommendation. The recommendation vector 122 includes context information for a specific recommendation associated with the specific reason for the recommendation. Thus, the recommendation vector 122 includes context information for a word sequence associated with the specific navigation recommendation. The recommendation vector 122 including the context information for the word sequence is provided to the machine learning engine 132, which trains a machine learning model 132a to learn from the context information for the word sequence included in the recommendation vector and sequentially generates a sequence of output vectors 134 corresponding to the human-readable reasons for the navigation recommendation.

[0029] In one or more embodiments, the machine learning model 132a is a bidirectional sequence-to-sequence (seq2seq) machine learning model 132a, where the input sequence is encoded within the recommendation vector 122. In one or more embodiments, the machine learning model 132a includes a long short-term memory (LTSM) type model, a gated recurrent unit (GRU) type model, or an extended recurrent neural network (RNN) type model.

[0030] The machine learning engine 132 trains a machine learning model 132a to generate human-readable reasons for the recommendation based on (1) the recommendation vector 122 and (2) the word embedding vector 131 for the reason ID associated with the recommendation vector 122. The decoder 130 includes reason ID logic 138 that identifies the reason for the recommendation contained in the context information of the recommendation vector 122. The reason ID logic associates the reason with a reason ID, which is a unique identifier associated with the human-readable reason for the recommendation.

[0031] The recommendation vector 122 is provided as an input vector to the machine learning model 132a to generate a first output vector Y1 in a sequence of output vectors 134. The first output vector Y1 may be generated from among a set of embedded vectors 131 representing words for a particular reason ID. The first output vector Y1 and the recommendation vector 122 are provided as input vectors to the machine learning model 132a to generate a next output vector Y2 in the sequence of output vectors 134. Each output vector 134 represents the embedded vector of one word in the sequence of words 134 that make up the human-readable reason for the recommendation. The machine learning model 132a repeats the process of providing the previously generated output vector and recommendation vector 122 as input vectors, generating each output vector Y1-Y2. n Then, the decoder 130 generates the next output vector in the sequence 134 until the output vector Y1 through Y n sequence 134 is used to generate a human-readable reason for the recommendation.

[0032] The machine learning engine 132 trains the machine learning model 132a based on historical training data 144 stored in the data repository 140. The historical training data 144 includes (1) historical recommendation vectors 145, (2) recommendation IDs 146 associated with the historical recommendation vectors 145, (3) reason IDs associated with the historical recommendation vectors 145, (4) reason attribute data 148, and (5) a text corpus 149. The text corpus 116 may include words previously used to generate human-readable reasons for navigation recommendations, as well as words from any other source, such as a dictionary. For example, the historical training data 144 may include 1,000 reason IDs 147 and 1,000 sequences of words corresponding to each reason ID 147. The decoder 130 converts the sequence of words associated with a particular reason ID 147 for the reason included in the corresponding historical recommendation vector 145 into a word embedding vector 131.

[0033] For a given subset of past training data 144, machine learning model 132a generates a sequence of output vectors 134 that correspond to human-readable sequences of words for past reason IDs 147 associated with past recommendation vectors 145. Machine learning engine 132 may compare the generated human-readable sequences of words to sequences of words associated with past reason IDs 147 to train machine learning model 132a.

[0034] In one or more embodiments, the machine learning engine 132 includes an attention engine 135. For each input vector in the sequence of input vectors 133, other than the recommendation vector 122, the attention engine 135 applies a weight to the vector values ​​of the recommendation vector 122 to focus the machine learning model 132a on words in the reasons for the recommendation. In one embodiment, the attention engine (1) calculates a weight for each vector Y1-Y2 representing words for a particular reason ID. n and (2) performing a dot product operation on the recommendation vector 122 to generate a scalar value. n The attention engine 135 converts the attention distribution into a vector and applies the vector to the recommendation vector 122. As a result, the machine learning model 132a generates an attention distribution based on the scalar values ​​for Y1-Y2. The attention engine 135 converts the attention distribution into a vector and applies the vector to the recommendation vector 122. As a result, the machine learning model 132a generates an attention distribution based on the scalar values ​​for Y1-Y2. n Applying a greater weight to certain words, represented by

[0035] In one or more embodiments, the decoder 130 generates the output vectors Y2 through Y1 by the machine learning model 132a. n The beam search engine 136 is included to improve the efficiency of the generation of the input vectors Y1 to Y nFor each input vector other than recommendation vector 122 in sequence 133, beam search engine 136 generates k sets of words identified as having the highest likelihood of being the next word in the reason for the recommendation. For example, beam search engine 136 may identify four sets of words, each of which is associated with an output vector Y1 through Y2. n The beam search engine 136 provides the k sets of words to the machine learning model 132a and generates output vectors Y2 through Y3 based on the set of words that have the highest likelihood of being the next three words in the sequence of words corresponding to the sequence 134 of the words Y1 through Y2. n For example, the machine learning engine 132 may be configured to generate coefficient values ​​for neurons in a neural network to influence the neural network to generate an output vector corresponding to a particular word or set of words.

[0036] Once trained, the machine learning model 132a generates, for a given input recommendation vector 122, a set 137 of human-readable reasons for the recommendation, including (1) recommendation IDs corresponding to recommendations for navigation targets for the user, (2) reason IDs corresponding to specific reasons for providing the navigation recommendation, and (3) a sequence of words describing the reasons for providing the recommendation. In one or more embodiments, the human-readable reason words are represented by vector values ​​Y1-Y2 converted into words by the navigation platform 110. n It may be stored as

[0037] The system 100 may generate a sorted list 143 of recommendation IDs, reason IDs, and human-readable reasons for the navigation recommendation. When a user logs in to the user interface 111 or a new user profile is entered through the navigation platform 110, the encoder may generate a recommendation vector 122 based on the user's user attribute data 141. In addition, the recommendation engine 123 may generate one or more recommendation IDs 124 based on the recommendation vector 122. If a corresponding reason ID is present in the sorted list, the navigation platform 110 may obtain the human-readable reason for the navigation recommendation from the sorted list 117. Alternatively, or in addition, the decoder 130 may generate a new reason ID and a new human-readable reason for the recommendation based on the new reason ID. The navigation platform 110 displays user interface elements in the user interface 111 that correspond to the recommended navigation target and the human-readable reason for the navigation recommendation.

[0038] In one or more embodiments, the encoder 120, the decoder 130, and the navigation platform 110 refer to hardware and / or software configured to perform the operations described herein for creating human-readable reasons for user interface navigation recommendations. Example operations for creating human-readable reasons for user interface navigation recommendations are described below with reference to FIG. 2.

[0039] In an embodiment, one or more of the encoder 120, the decoder 130, and the navigation platform 110 are implemented on one or more digital devices. The term "digital device" generally refers to any hardware device that includes a processor. A digital device may refer to a physical device or a virtual machine that runs an application. Examples of digital devices include computers, tablets, laptops, desktops, netbooks, servers, web servers, network policy servers, proxy servers, generic machines, specific function hardware devices, hardware routers, hardware switches, hardware firewalls, hardware network address translators (NATs), hardware load balancers, mainframes, televisions, content receivers, set-top boxes, printers, mobile handsets, smartphones, personal digital assistants ("PDAs"), wireless receivers and / or transmitters, base stations, communication management devices, routers, switches, controllers, access points, and / or client devices.

[0040] In one or more embodiments, system 100 may include more or fewer components than those shown in FIG. 1. The components shown in FIG. 1 may be local or remote from one another. The components shown in FIG. 1 may be implemented in software and / or hardware. Each component may be distributed across multiple applications and / or machines. Multiple components may be combined into a single application and / or machine. Operations described in connection with one component may instead be performed by another component.

[0041] Additional embodiments and / or examples related to computer networks are described below in Section 5 entitled "Computer Networks and Cloud Networks."

[0042] 3. Creating human-readable reasons for recommending navigation targets 2 illustrates an example set of operations for creating human-readable reasons for user interface navigation recommendations according to one or more embodiments. One or more of the operations illustrated in FIG. 2 may be modified, rearranged, or omitted entirely. Thus, the particular sequence of operations illustrated in FIG. 2 should not be construed as limiting the scope of one or more embodiments.

[0043] In one or more embodiments, an encoder generates user interface navigation recommendations by providing vectors including user attribute data and navigation attribute data to a trained machine learning model (operation 202). For example, the encoder may convert the user attribute values ​​and navigation attribute data values ​​into numeric and vector values. The encoder may generate input vectors for the trained model based on the converted user attribute values ​​and navigation data values. In one example, the trained machine learning model is a contextual multi-armed bandit type machine learning model.

[0044] Before generating user interface navigation recommendations, the encoder extracts a recommendation vector from the model (operation 204). The recommendation vector includes context information of the recommended user interface navigation and the corresponding reason for the recommendation. In one embodiment, the encoder extracts the recommendation vector before converting the recommendation vector into a probability distribution with a SoftMax layer. The encoder may then generate navigation recommendations and associate the recommendations with recommendation IDs using the probability distribution. The recommendation vector is a hidden layer in the machine learning model that is generated before generating the user interface navigation recommendations and the associated recommendation IDs.

[0045] The encoder provides the recommendation vector to a decoder (operation 206). The decoder identifies a reason ID associated with the recommendation vector. The reason ID corresponds to a human-readable sequence of words that provide a reason for a particular navigation recommendation. The decoder generates a set of embedded vectors that correspond to the words associated with the reason ID.

[0046] The decoder generates human-readable reasons for the recommendations in the recommendation vector using the machine learning model (operation 208). In one embodiment, the machine learning model is a bidirectional deep learning sequence-to-sequence model. For example, the machine learning model may be a long-short-term memory (LTSM) type model, a gated recurrent unit (GRU) type model, or an extended recurrent neural network (RNN) type model. The machine learning model may receive the context information in the recommendation vector as an input sequence and generate a series of embedded vectors representing words as an output sequence. The sequence of words describes the reasons for the user interface navigation recommendation associated with the recommendation vector. The decoder generates the human-readable reasons by providing the recommendation vector and one or more previously generated output vectors as input vectors to the machine learning model. In other words, the machine learning model generates an initial output vector based on the recommendation vector. The machine learning model generates each subsequent output vector based on (1) the recommendation vector and (2) at least one previously generated output vector. In one or more embodiments, the embedded vectors representing words of the reasons for the navigation recommendation define a set of vectors from which the output vector is generated. For example, the machine learning engine may identify, from among the embedded vectors, the embedded vector that has the highest probability of corresponding to the input vector.

[0047] The decoder stores the generated human-readable reason for the recommendation along with the reason ID and recommendation ID (operation 210). The decoder may generate an ordered list of reasons for the user navigation recommendation, in which each entry includes (1) a recommendation ID associated with the recommended navigation target, (2) a reason ID associated with the recommended navigation target, and (3) a human-readable reason associated with the reason ID.

[0048] Before receiving the recommendation vectors from the encoder, the decoder trains a machine learning model based on (1) past recommendation vectors and (2) embedding vectors representing reason words associated with the reason IDs associated with the past recommendation vectors.

[0049] After training the machine learning model, the system identifies a user profile (212). The user profile may be a stored user profile, a newly generated user profile, a user profile of a user logging in to the system, or a user profile generated by any other means. The user profile includes attribute information about the user. For example, the system may generate a set of navigation recommendations and human-readable reasons for the navigation recommendations for every user registered to use the system to access applications or services without requiring the user to log in to the system. Additionally or alternatively, the system may detect user login and generate a set of navigation recommendations and human-readable reasons for the navigation recommendations based on the detection of user login. In another example, the system may generate a set of navigation recommendations and human-readable reasons for the navigation recommendations based on user navigation to a particular graphical user interface. Embodiments include a system that generates a set of navigation recommendations and human-readable reasons for the navigation recommendations based on any predetermined event or trigger.

[0050] The encoder generates a recommendation vector using the user attribute information in the user profile and any navigation attribute information associated with the user (operation 214). The encoder may generate the recommendation vector using, for example, a trained contextual multi-armed bandit type machine learning model.

[0051] The system identifies a recommendation ID and a reason ID associated with the recommendation vector. For example, the system may run the recommendation vector through a SoftMax layer to generate a probability distribution. The system may generate a recommendation ID based on the probability distribution. A decoder may identify a reason ID associated with the recommendation vector. The system determines whether a human-readable reason associated with the recommendation ID and reason ID already exists (operation 216). In particular, the system may refer to a dictionary that stores a set of (1) recommendation IDs, (2) reason IDs, and (3) human-readable reasons associated with the recommendation to determine whether a human-readable reason already exists. If a human-readable reason associated with the recommendation ID and reason ID has been previously generated and stored, the system retrieves the human-readable reason from the dictionary that stores human-readable reasons for navigation recommendations (operation 218).

[0052] If the system determines that a human-readable reason has not yet been generated for the recommendation ID and reason ID, the encoder provides the recommendation vector to a trained machine learning model in the decoder (operation 220). The trained model sequentially generates words for the human-readable reason for the recommendation. The system may store the set of recommendation ID, reason ID, and human-readable reason associated with the recommendation in a dictionary of previously generated human-readable reasons for the recommendation.

[0053] The system displays the navigation recommendation associated with the recommendation ID along with a human-readable reason for the recommendation associated with the reason ID (operation 222). For example, a user may access a graphical user interface to navigate to one or more applications or services. The system may display the navigation recommendation and the corresponding human-readable reason for navigation in the graphical user interface.

[0054] In one or more embodiments, the reason ID associated with a particular sequence of words may include information regarding the ranking for the particular sequence of words. For example, the system may determine that a particular navigation target is recommended based on three separate reason IDs, each associated with a different human-readable reason for the recommendation. The system may select only the highest-ranked recommendations to display human-readable information for the recommendation. Alternatively, the system may select only a predetermined number of human-readable reasons for the recommendation, such as only two or fewer reasons for the recommendation.

[0055] In one or more embodiments, a recommendation vector may include information that restricts human-readable reasons that may be displayed for particular users with certain attributes. An organization may not want its general membership to know that a certain class of users navigates to navigation targets related to information that may be confidential or secret. For example, if a company is acquiring a competitor, the legal department may need to navigate to targets related to the competitor. The system may block users with the user attribute "department" having a value other than "legal" from generating recommendation vectors that result in recommendation IDs related to competitors, or human-readable reasons that identify other employees as navigating to navigation targets related to competitors. Similarly, the system may restrict human-readable reasons that identify searches of other users that disclose confidential information, such as users accessing navigation targets related to payroll, termination, disability, personal leave, parental leave, or medical benefits. In one embodiment, the system may prevent the generation of human-readable reasons for certain recommendations. In another embodiment, the system may allow the generation of human-readable reasons, but the system may assign stored human-readable reasons with rankings or designators that prevent the use or display of the human-readable reasons with recommendations. Thus, a system administrator may identify types of recommendations that generate confidential or secret human-readable reasons.

[0056] 4. Training a sequence-to-sequence machine learning model 3 illustrates an example set of operations for training a machine learning model to sequentially generate words for human-readable reasons for user interface navigation recommendations, according to one or more embodiments. One or more of the operations illustrated in FIG. 3 may be modified, rearranged, or omitted altogether. Thus, the particular sequence of operations illustrated in FIG. 3 should not be construed as limiting the scope of one or more embodiments.

[0057] The machine learning engine obtains past user navigation data (operation 302). The user navigation data includes a user interface recommendation vector associated with the user and the user navigation, past reason IDs associated with the navigation recommendations, and human-readable reasons associated with the reason IDs. For example, a navigation platform may provide a user with access to many different applications and many different processes, documents, and functions within each application. Each process, document, and function that may be accessed by the user may be a separate navigation target. The recommendation vector is generated by the trained machine learning engine and includes contextual information of the past user navigation data, including information about how frequently each user accesses each process, document, and function in each application provided by the navigation platform.

[0058] The machine learning engine generates a training set of recommendation vectors, recommendation IDs, and corresponding human-readable reasons for the recommendations (operation 304). The training set includes specific user navigation recommendations represented by recommendation IDs and associated human-readable reasons represented by reason IDs.

[0059] The machine learning engine uses the training set to train a machine learning model to generate human-readable reasons for the user navigation recommendation (operation 306). The machine learning engine selects a specific recommendation vector corresponding to the specific human-readable reason for the recommendation (operation 308). The machine learning engine converts the human-readable reason words into an embedding vector (operation 310). In one embodiment, the machine learning engine includes a pre-trained word-to-vector (word2vec) model that is trained with all words in a text corpus that includes all human-readable reason words for all user navigation recommendations in the historical data. The machine learning engine converts the specific reason words for the recommendation into the vector generated by the word2vec model.

[0060] The machine learning engine inputs the recommendation vector into another machine learning model to generate an initial output vector (operation 312). For example, the machine learning model may be a bidirectional sequence-to-sequence (seq2seq) machine learning model. The model may be a long-short-term memory (LTSM) type model, a gated recurrent unit (GRU) type model, or an extended recurrent neural network (RNN) type model. In one embodiment, the initial output vector is selected from among an embedded vector representing words associated with past human-readable reasons for recommendations associated with the recommendation vector. Alternatively, the initial output vector may be selected from among a combined set of (1) words associated with past human-readable reasons for recommendations and (2) a set of words identified in a word2vec model as being associated with words in the past human-readable reasons.

[0061] The machine learning engine takes the initial output vector and the recommendation vector and inputs the two vectors into a human-readable reason generation model to generate the next word in the sequence of output vectors that represent human-readable words of the reason for the recommendation (operation 314). The machine learning engine takes the previously generated output vector and the recommendation vector and inputs the two vectors into the model to generate the next output vector (operation 316).

[0062] In one embodiment, the machine learning engine includes an attention engine. The attention engine applies weights to the vector values ​​of the recommendation vector to focus the human-readable reason generation machine learning model on specific next words in the reasons for the recommendation (operation 320). In one embodiment, the attention engine performs a dot product operation on (1) the embedded vectors representing words associated with past human-readable reasons for the recommendation and (2) the recommendation vector associated with the recommendation to generate a scalar value. The attention engine generates an attention distribution based on the scalar values ​​for the embedded vectors. The attention engine converts the attention distribution into a vector and applies the vector to the recommendation vector. As a result, the machine learning engine applies a greater weight to specific words represented by specific embedded vectors in the embedded vectors representing words associated with the recommendation.

[0063] In one embodiment, the machine learning engine also includes a beam search engine. At each stage in which the output vectors and recommendation vectors are applied as inputs to the human-readable reason-generating machine learning model, the beam search engine generates k sets of output vectors identified as having the highest probability of being the next output vector corresponding to a word in the reason for the recommendation (operation 322). For example, the beam search engine may identify four sets of output vectors, each consisting of a sequence of three output vectors having the highest probability of being the next output vector in the sequence of output vectors generated by the machine learning model. The beam search engine provides the k sets of output vectors to the machine learning engine and influences the machine learning model to generate the particular output vector having the highest probability of being the next output vector in the sequence.

[0064] The machine learning engine repeats the process of providing the previously generated output vector along with the recommendation vector as input to the machine learning model to generate the next output vector until all words in the human-readable reason for the recommendation have been generated (operation 318).

[0065] In one embodiment, the generated output vectors are converted into corresponding sequences of words, and the sequences of words are output to a user. The user provides feedback regarding the sequences of words (operation 324). For example, the user may identify words that do not make sense in the sequence of words, words that are out of order, or words that are unnecessary or missing from the sequence. In one or more embodiments, the machine learning engine may provide the user with one or more alternate words based on the relationships between words identified by the word2vec model.

[0066] In one or more embodiments, a user may indicate that certain human-readable reasons should not be generated based on certain users having certain user attributes, for example, a reason may reveal sensitive or confidential information about other users' navigation.

[0067] The machine learning engine updates the machine learning model based on the user feedback (operation 326). For example, if the user indicates words as being in the incorrect order, the machine learning engine may adjust the attention engine to increase the probability that a particular word occurs in a particular position. If the user indicates one or more words as being omitted from the sequence, the machine learning engine may add one or more output vectors to the sequence. If the user indicates that a word is unnecessary in the sequence, the machine learning engine may omit the output vector from the sequence. If the user indicates that a particular reason should not be generated, the machine learning model may refrain from generating a human-readable reason for a particular recommended vector.

[0068] 5. Computer Networks and Cloud Networks In one or more embodiments, a computer network provides connectivity among a set of nodes. The nodes may be local and / or remote from one another. The nodes are connected by a set of links. Examples of links include coaxial cable, unshielded twisted cable, copper cable, optical fiber, and virtual links.

[0069] A subset of nodes implements computer networks. Examples of such nodes include switches, routers, firewalls, and network address translators (NATs). Another subset of nodes uses computer networks. Such nodes (also called "hosts") may run client processes and / or server processes. Client processes make requests for computing services (such as running a particular application and / or storing a particular amount of data). Server processes respond by performing the requested service and / or returning corresponding data.

[0070] A computer network may be a physical network, including physical nodes connected by physical links. A physical node is any digital device. A physical node may be a specific-function hardware device, such as a hardware switch, a hardware router, a hardware firewall, and a hardware NAT. Additionally or alternatively, a physical node may be a generic machine configured to run various virtual machines and / or applications that perform respective functions. A physical link is a physical medium connecting two or more physical nodes. Examples of links include coaxial cable, unshielded twisted cable, copper cable, and optical fiber.

[0071] A computer network may be an overlay network. An overlay network is a logical network implemented on top of another network (e.g., a physical network). Each node in the overlay network corresponds to a respective node in the underlying network. Thus, each node in the overlay network is associated with both an overlay address (for addressing the overlay node) and an underlay address (for addressing the underlay node that implements the overlay node). Overlay nodes may be digital devices and / or software processes (e.g., virtual machines, application instances, or threads). Links connecting overlay nodes are implemented as tunnels that pass through the underlying network. The overlay nodes at either end of the tunnel treat the underlying multi-hop path between them as a single logical link. Tunneling is performed through encapsulation and decapsulation.

[0072] In embodiments, a client may be local to a computer network and / or remote from the computer network. A client may access a computer network over a private network or another computer network, such as the Internet. A client may communicate requests to the computer network using a communication protocol such as Hypertext Transfer Protocol (HTTP). The request is communicated through an interface, such as a client interface (e.g., a web browser), a program interface, or an application programming interface (API).

[0073] In embodiments, a computer network provides connectivity between clients and network resources. The network resources include hardware and / or software configured to run server processes. Examples of network resources include processors, data storage, virtual machines, containers, and / or software applications. The network resources are shared among multiple clients. The clients request computing services from the computer network independently of each other. The network resources are dynamically allocated to requests and / or clients on an on-demand basis. The network resources allocated to each request and / or client may be scaled up or down based on, for example, (a) the computing services requested by a particular client, (b) the aggregate computing services requested by a particular tenant, and / or (c) the aggregate computing services requested of the computer network. Such a computer network may be referred to as a "cloud network."

[0074] In embodiments, a service provider offers a cloud network to one or more end users. Various service models may be implemented by the cloud network, including, but not limited to, Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), and Infrastructure-as-a-Service (IaaS). In SaaS, the service provider offers end users the ability to use the service provider's applications running on the network resources. In PaaS, the service provider offers end users the ability to deploy custom applications to the network resources. The custom applications may be created using programming languages, libraries, services, and tools supported by the service provider. In IaaS, the service provider offers end users the ability to provision processing, storage, network, and other basic computing resources provided by the network resources. Any arbitrary application, including an operating system, may be deployed on the network resources.

[0075] In embodiments, various deployment models may be implemented by a computer network, including, but not limited to, a private cloud, a public cloud, and a hybrid cloud. In a private cloud, network resources are reserved for exclusive use by a specific group of one or more entities (as used herein, the term "entity" refers to a company, organization, individual, or other entity). The network resources may be local to and / or remote from the premises of the specific group of entities. In a public cloud, cloud resources are reserved for multiple entities (also referred to as "tenants" or "customers") that are independent of each other. The computer network and its network resources are accessed by clients corresponding to different tenants. Such a computer network may be referred to as a "multi-tenant computer network." Several tenants may use the same specific network resources at different times and / or simultaneously. The network resources may be local to and / or remote from the tenant's premises. In a hybrid cloud, the computer network includes a private cloud and a public cloud. An interface between the private cloud and the public cloud enables data and application portability. Data stored in the private cloud and data stored in the public cloud may be exchanged via the interface. Applications implemented in the private cloud and applications implemented in the public cloud may have dependencies on each other, and calls from applications in the private cloud to applications in the public cloud (and vice versa) may be made through interfaces.

[0076] In embodiments, tenants of a multi-tenant computer network are independent of one another. For example, the business or operations of one tenant may be separate from the business or operations of another tenant. Different tenants may have different network requirements for the computer network. Examples of network requirements include processing speed, amount of data storage, security requirements, performance requirements, throughput requirements, latency requirements, resiliency requirements, Quality of Service (QoS) requirements, tenant isolation, and / or consistency. The same computer network may need to implement different network requirements required by different tenants.

[0077] In one or more embodiments, in a multi-tenant computer network, tenant isolation is implemented to ensure that applications and / or data of different tenants are not shared with each other. Various tenant isolation approaches may be used.

[0078] In an embodiment, each tenant is associated with a tenant ID. Each network resource in a multi-tenant computer network is tagged with a tenant ID. A tenant is granted access to a particular network resource only if the tenant and the particular network resource are associated with the same tenant ID.

[0079] In an embodiment, each tenant is associated with a tenant ID. Each application implemented by the computer network is tagged with a tenant ID. Additionally or alternatively, each data structure and / or dataset stored by the computer network is tagged with a tenant ID. A tenant is granted access to a particular application, data structure, and / or dataset only if the tenant and the particular application, data structure, and / or dataset are associated with the same tenant ID.

[0080] As one example, each database implemented by a multi-tenant computer network may be tagged with a tenant ID. Only the tenant associated with the corresponding tenant ID may access the data in a particular database. As another example, each entry in a database implemented by a multi-tenant computer network may be tagged with a tenant ID. Only the tenant associated with the corresponding tenant ID may access the data in a particular entry. However, a database may be shared by multiple tenants.

[0081] In an embodiment, the subscription list indicates which tenants have permission to access which applications. For each application, a list of tenant IDs of tenants authorized to access the application is stored. A tenant is authorized to access a particular application only if the tenant's tenant ID is included in the subscription list corresponding to the particular application.

[0082] In an embodiment, network resources (such as digital devices, virtual machines, application instances, and threads) corresponding to different tenants are isolated in specific tenant overlay networks maintained by a multi-tenant computer network. As an example, packets from any source device in a tenant overlay network may be sent only to other devices in the same tenant overlay network. An encapsulation tunnel is used to prohibit any transmission from a source device in a tenant overlay network to a device in another tenant overlay network. In particular, a packet received from a source device is encapsulated in an outer packet. The outer packet is sent from a first encapsulation tunnel endpoint (communicating with a source device in the tenant overlay network) to a second encapsulation tunnel endpoint (communicating with a destination device in the tenant overlay network). The second encapsulation tunnel endpoint decapsulates the outer packet to obtain the original packet sent by the source device. The original packet is sent from the second encapsulation tunnel endpoint to a destination device in the same specific overlay network.

[0083] 7. Miscellaneous; Extensions Embodiments relate to systems having one or more devices that include a hardware processor and are configured to perform any of the operations described herein and / or recited in any of the claims below.

[0084] In an embodiment, a non-transitory computer-readable storage medium includes instructions that, when executed by one or more hardware processors, cause the performance of any of the operations described herein and / or recited in any of the claims.

[0085] Any combination of the features and functions described herein may be used in accordance with one or more embodiments. In the specification, the embodiments are described with reference to numerous specific details that may vary from implementation to implementation. The specification and drawings should therefore be regarded in an illustrative, rather than a restrictive, sense. The sole and exclusive indication of the scope of the invention, and what is intended by the applicant to be the scope of the invention, is the literal and equivalent scope of the set of claims issuing from this application in the specific form in which such claims arise, including any subsequent amendments.

[0086] 8. Hardware Overview According to one embodiment, the techniques described herein are implemented by one or more special-purpose computing devices. The special-purpose computing devices may be hardwired to perform the techniques, or may include digital electronic devices such as one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or network processing units (NPUs) that are persistently programmed to perform the techniques, or may include one or more general-purpose hardware processors programmed to execute the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices may combine custom hardwired logic, ASICs, FPGAs, or NPUs with custom programming to achieve the techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, network devices, or any other devices incorporating hardwired and / or program logic to implement the techniques.

[0087] 4 is a block diagram illustrating a computer system 400 in which embodiments of the invention may be implemented. Computer system 400 includes a bus 402 or other communication mechanism for communicating information, and a hardware processor 404 coupled with bus 402 for processing information. Hardware processor 404 may be, for example, a general-purpose microprocessor.

[0088] Computer system 400 also includes a main memory 406, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 402 for storing information and instructions executed by processor 404. Main memory 406 may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 404. Such instructions, when stored on a non-transitory storage medium accessible to processor 404, render computer system 400 a special-purpose machine customized to perform the operations specified in the instructions.

[0089] Computer system 400 further includes a read-only memory (ROM) 408 or other static storage device coupled to bus 402 for storing static information and instructions for processor 404. A storage device 410, such as a magnetic disk or optical disk, is provided and coupled to bus 402 for storing information and instructions.

[0090] Computer system 400 may be coupled via bus 402 to a display 412, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 414, including alphanumeric and other keys, is coupled to bus 402 for communicating information and command selections to processor 404. Another type of user input device is a cursor control 416, such as a mouse, trackball, or cursor direction keys, for communicating instructional information and command selections to processor 404 and for controlling cursor movement on display 412. The input device generally has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allow the device to specify a position in a plane.

[0091] Computer system 400 may implement the techniques described herein using customized hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic that, in combination with the computer system, causes or programs computer system 400 to be a special-purpose machine. According to one embodiment, the techniques herein are performed by computer system 400 in response to processor 404 executing one or more sequences of one or more instructions contained in main memory 406. Such instructions may be read into main memory 406 from another storage medium, such as storage device 410. Execution of the sequences of instructions contained in main memory 406 causes processor 404 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0092] The term "storage medium" as used herein refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a specific manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 410. Volatile media include dynamic memory, such as main memory 406. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage medium, CD-ROMs, any other optical data storage medium, any physical medium with a pattern of holes, RAM, PROM, and EPROM, FLASH-EPROM, NVRAM, any other memory chip or cartridge, content addressable memory (CAM), and ternary content addressable memory (TCAM).

[0093] Storage media is distinct from but may be used in conjunction with transmission media. Transmission media involves transferring information between storage media. For example, transmission media include coaxial cables, copper wire, and fiber optics, including the wires that comprise bus 402. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0094] Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 404 for execution. For example, the instructions may initially be carried on a magnetic disk or solid state drive of a remote computer. The remote computer can load the instructions into a dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 400 can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector can receive the data carried in the infrared signal and appropriate circuitry can place the data on bus 402. Bus 402 carries the data to main memory 402, from which processor 404 retrieves and executes the instructions. The instructions received by main memory 406 may optionally be stored on storage device 410 either before or after execution by processor 404.

[0095] Computer system 400 also includes a communication interface 418 coupled to bus 402. The communication interface 418 provides a two-way data communication coupling to a network link 420 that is connected to a local network 422. For example, communication interface 418 may be an Integrated Services Digital Network (ISDN) card, cable modem, satellite modem, or a corresponding type of model to provide a data communication connection to a telephone line. As another example, communication interface 418 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. A wireless link may also be implemented. In any such implementation, communication interface 418 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

[0096] Network link 420 typically provides data communication through one or more networks to other data devices. For example, network link 420 may provide a connection through local network 422 to a host computer 424 or to data equipment operated by an Internet Service Provider (ISP) 426. ISP 426 itself provides data communication services through the worldwide packet data communication network now commonly referred to as the "Internet" 428. Both local network 422 and the Internet 428 use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 420 and through communication interface 418, which carry the digital data to and from computer system 400, are exemplary forms of transmission media.

[0097] Computer system 400 can send messages and receive data, including program code, through the network(s), network link 420 and communication interface 418. In the Internet example, a server 430 might transmit a requested code for an application program through Internet 428, ISP 426, local network 422 and communication interface 418.

[0098] The received code may be executed by processor 404 as it is received, and / or stored in memory device 410, or other non-volatile storage for later execution.

[0099] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Accordingly, the specification and drawings should be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indicator of the scope of the invention, and what is intended by the applicant to be the scope of the invention, is the literal and equivalent scope of the set of claims issuing from this application in the specific form in which such claims arise, including any subsequent amendments.

Claims

1. 1. A computer-readable program comprising instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations, the operations including: training a machine learning model to generate human-readable reasons for the target user interface navigation recommendations; training the machine learning model includes obtaining a training dataset of historical user interface navigation information; The training data set is a first vector representing a particular user interface navigation recommendation; a second vector representing human-readable reasons for the particular user interface navigation recommendation; training the machine learning model includes generating, based on the first vector, a sequence of output vectors representing ordered sequences of words that constitute a particular human-readable reason for the particular user interface navigation recommendation; generating the sequence of output vectors comprises: generating a sequence of input vectors; providing each input vector of the sequence of input vectors to the machine learning model to generate a respective output vector of the sequence of output vectors; a first input vector in the sequence of input vectors is generated based on a first vector value of the first vector; a subsequent input vector in the sequence of input vectors is generated based on (1) the first vector and (2) one or more previously generated output vectors in the sequence of output vectors; training the machine learning model, training the machine learning model based on comparing the sequence of output vectors with the second vector representing the human-readable reasons for the particular user interface navigation recommendation; applying the machine learning model to the target user interface navigation recommendation to generate the human-readable reasons for the target user interface navigation recommendation.

2. the targeted user interface navigation recommendations include a plurality of targeted user interface navigation recommendations; the human-readable reasons for the target user interface navigation recommendations include a plurality of human-readable reasons for the plurality of target user interface navigation recommendations; The operation may further include: applying the plurality of target user interface navigation recommendations to the machine learning model to generate the plurality of human-readable reasons for the plurality of target user interface navigation recommendations; storing a plurality of sets of reason pairs; Each set of reason pairs comprises: a target user interface navigation recommendation among the plurality of target user interface navigation recommendations; and and one or more of the plurality of human-readable reasons for the one target user interface navigation recommendation.

3. The step of generating the sequence of output vectors further comprises: For the first input vector in the sequence of input vectors: generating a first set of weighted values ​​for a first plurality of vector values ​​in the first vector; generating the first input vector based on applying the first set of weighted values ​​to the first vector; The step of generating the sequence of output vectors further comprises: For a subsequent input vector in the sequence of input vectors: generating a second set of weighted values ​​for the first vector values ​​in the first vector; and applying the second set of weighted values ​​to vector values ​​of a previously generated output vector in the sequence of output vectors to generate the subsequent input vector.

4. The computer-readable program of claim 1 or 2, wherein the first vector is a hidden layer of a machine learning model for generating user interface navigation recommendations.

5. The step of generating the sequence of output vectors further comprises: For the first input vector in the sequence of input vectors: Computing a first plurality of probabilities for a first plurality of sequences of corresponding output vectors; selecting a first subset of the first plurality of sequences of output vectors as those having the highest probability of corresponding to the input vector; and generating a first output vector in the sequence of output vectors based on the selected first subset of the first plurality of sequences of output vectors; generating a first set of one or more sequences of output vectors; the first plurality of probabilities specifying probabilities that a corresponding first plurality of sequences of output vectors corresponds to the input vector; The step of generating the sequence of output vectors further comprises: For a subsequent input vector in the sequence of input vectors: Computing a second plurality of probabilities for a second plurality of sequences of corresponding output vectors; selecting a second subset of the second plurality of sequences of output vectors as those having the highest probability of corresponding to the subsequent input vector; and generating a subsequent output vector in the sequence of output vectors based on the selected second subset of the second plurality of sequences of output vectors; generating a second set of one or more sequences of output vectors; 3. The computer readable program of claim 1, wherein the second plurality of probabilities specifies probabilities that a corresponding second plurality of sequences of output vectors corresponds to the subsequent input vector.

6. 3. The computer-readable program of claim 1, wherein the second vector is generated by applying a word-to-vector machine learning model to the human-readable reasons for the target user interface navigation recommendation associated with the first vector.

7. the second vector is generated by applying a word-to-vector machine learning model to the human-readable reasons for the target user interface navigation recommendation associated with the first vector; the first vector is a hidden layer of a machine learning model for generating user interface navigation recommendations; The step of generating the sequence of output vectors further comprises: For the first input vector in the sequence of input vectors: generating a first set of weighted values ​​for a first plurality of vector values ​​in the first vector; generating the first input vector based on applying the first set of weighted values ​​to the first vector; The step of generating the sequence of output vectors further comprises: For a subsequent input vector in the sequence of input vectors: generating a second set of weighted values ​​for the first vector values ​​in the first vector; applying the second set of weighted values ​​to vector values ​​of a previously generated output vector in the sequence of output vectors to generate the subsequent input vector; The step of generating the sequence of output vectors further comprises: For the first input vector in the sequence of input vectors: Computing a first plurality of probabilities for a first plurality of sequences of corresponding output vectors; selecting a first subset of the first plurality of sequences of output vectors as those having the highest probability of corresponding to the input vector; and generating a first output vector in the sequence of output vectors based on the selected first subset of the first plurality of sequences of output vectors; generating a first set of one or more sequences of output vectors; the first plurality of probabilities specifying probabilities that a corresponding first plurality of sequences of output vectors corresponds to the input vector; The step of generating the sequence of output vectors further comprises: For the subsequent input vector in the sequence of input vectors: Computing a second plurality of probabilities for a second plurality of sequences of corresponding output vectors; selecting a second subset of the second plurality of sequences of output vectors as those having the highest probability of corresponding to the subsequent input vector; and generating a subsequent output vector in the sequence of output vectors based on the selected second subset of the second plurality of sequences of output vectors; generating a second set of one or more sequences of output vectors; 3. The computer readable program of claim 1, wherein the second plurality of probabilities specifies probabilities that a corresponding second plurality of sequences of output vectors corresponds to the subsequent input vector.

8. 1. A method comprising: training a machine learning model to generate human-readable reasons for the target user interface navigation recommendations; training the machine learning model includes obtaining a training dataset of historical user interface navigation information; The training data set is a first vector representing a particular user interface navigation recommendation; a second vector representing human-readable reasons for the particular user interface navigation recommendation; training the machine learning model includes generating, based on the first vector, a sequence of output vectors representing ordered sequences of words that constitute a particular human-readable reason for the particular user interface navigation recommendation; generating the sequence of output vectors comprises: generating a sequence of input vectors; providing each input vector of the sequence of input vectors to the machine learning model to generate a respective output vector of the sequence of output vectors; a first input vector in the sequence of input vectors is generated based on a first vector value of the first vector; a subsequent input vector in the sequence of input vectors is generated based on the first vector and one or more previously generated output vectors in the sequence of output vectors; training the machine learning model, training the machine learning model based on comparing the sequence of output vectors with the second vector representing the human-readable reasons for the particular user interface navigation recommendation; applying the machine learning model to the target user interface navigation recommendation to generate the human-readable reasons for the target user interface navigation recommendation.

9. the targeted user interface navigation recommendations include a plurality of targeted user interface navigation recommendations; the human-readable reasons for the target user interface navigation recommendations include a plurality of human-readable reasons for the plurality of target user interface navigation recommendations; The method further comprises: applying the plurality of target user interface navigation recommendations to the machine learning model to generate the plurality of human-readable reasons for the plurality of target user interface navigation recommendations; storing a plurality of sets of reason pairs; Each set of reason pairs comprises: a target user interface navigation recommendation among the plurality of target user interface navigation recommendations; and and one or more of the plurality of human-readable reasons for the one target user interface navigation recommendation.

10. The step of generating the sequence of output vectors further comprises: For the first input vector in the sequence of input vectors: generating a first set of weighted values ​​for a first plurality of vector values ​​in the first vector; generating the first input vector based on applying the first set of weighted values ​​to the first vector; The step of generating the sequence of output vectors further comprises: For a subsequent input vector in the sequence of input vectors: generating a second set of weighted values ​​for the first vector values ​​in the first vector; and applying the second set of weighted values ​​to vector values ​​of a previously generated output vector in the sequence of output vectors to generate the subsequent input vector.

11. The method of claim 8 or 9, wherein the first vector is a hidden layer of a machine learning model for generating user interface navigation recommendations.

12. The step of generating the sequence of output vectors further comprises: For the first input vector in the sequence of input vectors: Computing a first plurality of probabilities for a first plurality of sequences of corresponding output vectors; selecting a first subset of the first plurality of sequences of output vectors as those having the highest probability of corresponding to the input vector; and generating a first output vector in the sequence of output vectors based on the selected first subset of the first plurality of sequences of output vectors; generating a first set of one or more sequences of output vectors; the first plurality of probabilities specifying probabilities that a corresponding first plurality of sequences of output vectors corresponds to the input vector; The step of generating the sequence of output vectors further comprises: For a subsequent input vector in the sequence of input vectors: Computing a second plurality of probabilities for a second plurality of sequences of corresponding output vectors; selecting a second subset of the second plurality of sequences of output vectors as those having the highest probability of corresponding to the subsequent input vector; and generating a subsequent output vector in the sequence of output vectors based on the selected second subset of the second plurality of sequences of output vectors; generating a second set of one or more sequences of output vectors; 10. The method of claim 8 or 9, wherein the second plurality of probabilities specifies the probability that a corresponding second plurality of sequences of output vectors corresponds to the subsequent input vector.

13. 10. The method of claim 8 or 9, wherein the second vector is generated by applying a word-to-vector machine learning model to the human-readable reasons for the target user interface navigation recommendation associated with the first vector.

14. 1. A system comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the system to: training a machine learning model to generate human-readable reasons for the target user interface navigation recommendation; training the machine learning model includes obtaining a training dataset of historical user interface navigation information; The training data set is a first vector representing a particular user interface navigation recommendation; a second vector representing human-readable reasons for the particular user interface navigation recommendation; training the machine learning model includes generating a sequence of output vectors representing an ordered sequence of words that constitute a particular human-readable reason for the particular user interface navigation recommendation based on the first vector; generating the sequence of output vectors generating a sequence of input vectors; providing each input vector of the sequence of input vectors to the machine learning model to generate a respective output vector of the sequence of output vectors; a first input vector in the sequence of input vectors is generated based on a first vector value of the first vector; a subsequent input vector in the sequence of input vectors is generated based on the first vector and one or more previously generated output vectors in the sequence of output vectors; Training the machine learning model includes: training the machine learning model based on comparing the sequence of output vectors with the second vector representing the human-readable reasons for the particular user interface navigation recommendation; and applying the machine learning model to the target user interface navigation recommendation to generate the human-readable reasons for the target user interface navigation recommendation.

15. the targeted user interface navigation recommendations include a plurality of targeted user interface navigation recommendations; the human-readable reasons for the target user interface navigation recommendations include a plurality of human-readable reasons for the plurality of target user interface navigation recommendations; The instructions further include: applying the plurality of target user interface navigation recommendations to the machine learning model to generate the plurality of human-readable reasons for the plurality of target user interface navigation recommendations; storing a plurality of sets of reason pairs; Each set of reason pairs comprises: a target user interface navigation recommendation among the plurality of target user interface navigation recommendations; and and one or more of the plurality of human-readable reasons for the one target user interface navigation recommendation.

16. Generating the sequence of output vectors further comprises: For the first input vector in the sequence of input vectors: generating a first set of weighted values ​​for a first plurality of vector values ​​in the first vector; generating the first input vector based on applying the first set of weighted values ​​to the first vector; Generating the sequence of output vectors further comprises: For a subsequent input vector in the sequence of input vectors: generating a second set of weighted values ​​for the first vector values ​​in the first vector; and applying the second set of weighted values ​​to vector values ​​of a previously generated output vector in the sequence of output vectors to generate the subsequent input vector.

17. The system of claim 14 or 15, wherein the first vector is a hidden layer of a machine learning model for generating user interface navigation recommendations.

18. Generating the sequence of output vectors further comprises: For the first input vector in the sequence of input vectors: Computing a first plurality of probabilities for a first plurality of sequences of corresponding output vectors; selecting a first subset of the first plurality of sequences of output vectors as those having the highest probability of corresponding to the input vector; and generating a first output vector in the sequence of output vectors based on the selected first subset of the first plurality of sequences of output vectors; generating a first set of one or more sequences of output vectors; the first plurality of probabilities specifying probabilities that a corresponding first plurality of sequences of output vectors corresponds to the input vector; Generating the sequence of output vectors further comprises: For a subsequent input vector in the sequence of input vectors: Computing a second plurality of probabilities for a second plurality of sequences of corresponding output vectors; selecting a second subset of the second plurality of sequences of output vectors as those having the highest probability of corresponding to the subsequent input vector; and generating a subsequent output vector in the sequence of output vectors based on the selected second subset of the second plurality of sequences of output vectors; generating a second set of one or more sequences of output vectors; 16. The system of claim 14 or 15, wherein the second plurality of probabilities specifies the probability that a corresponding second plurality of sequences of output vectors corresponds to the subsequent input vector.

19. 16. The system of claim 14 or 15, wherein the second vector is generated by applying a word-to-vector machine learning model to the human-readable reasons for the target user interface navigation recommendation associated with the first vector.

Images