Method for retrieving document fragments responsive to the user input and an electronic device thereof

Abstract

In one embodiment, a method includes obtaining a user input including a text string from an external device, identifying a first topic embedding from among multiple topic embeddings based on an embedding associated with the user input, wherein each embedding associated with the user input or each topic embedding is within an n-dimensional embedding space, identifying first entities associated with the user input, identifying entity embeddings based on the first entities, wherein each entity embedding is within the n-dimensional embedding space, generating a respective n-sphere based on each entity embedding and the first topic, wherein each n-sphere is associated with a respective set of document fragments, retrieving one or more sets of document fragments associated with the n-spheres, and transmitting instructions for presenting the retrieved sets of document fragments to the external device responsive to the user input.

Description

METHOD FOR RETRIEVING DOCUMENT FRAGMENTS RESPONSIVE TO THE USER INPUT AND AN ELECTRONIC DEVICE THEREOF

[0001] This disclosure generally relates to information retrieval, and in particular relates to hardware and software for retrieving document fragments responsive to the user input.

[0002] Retrieval augmented generation (RAG) is a framework that combines retrieval-based systems and generative artificial intelligence (AI) models for domain-specific tasks. RAG powered generative AI chatbots may utilize a document store and a method to retrieve contextually relevant document fragments (e.g., pieces of documents that can be used to help supplement context for a chatbot) based on a user input. Distance metrics such as cosine distance or Euclidean distance can be used to compare embeddings between user input and documents. When provided with enough context, the documents returned may prioritize depth (i.e., a quality of the documents returned that looks at the accuracy of documents that relate to the user input) and may be intended to have targeted relevancy with the user input.

[0003] A user input is considered ambiguous if there is little to no context in the input. There are multiple reasons why a user might have ambiguous input. One reason may be clarification. The user is seeking more information about a certain topic and may not have knowledge about the correct language. Another reason may be assumption. The user assumes that a chatbot understands the full nature of their question or issue, even without providing full context. Another reason may be confusion. The user is misinformed about a question or issue.

[0004] A response from a chatbot prioritizing depth may work well when there is an understanding of the differences between services, and a level of comprehension of services. It is beneficial to guide a user in the right direction to increase their engagement.

[0005] In one embodiment, a method performed by an electronic device is disclosed. The method includes obtaining a user input including a text string from an external device, identifying a first topic embedding from among multiple topic embeddings based on an embedding associated with the user input, wherein each embedding associated with the user input or each topic embedding is within an n-dimensional embedding space, identifying first entities associated with the user input, identifying entity embeddings based on the first entities, wherein each entity embedding is within the n-dimensional embedding space, generating a respective n-sphere based on each entity embedding and the first topic, wherein each n-sphere is associated with a respective set of document fragments, retrieving one or more sets of document fragments associated with the n-spheres, and transmitting instructions for presenting the retrieved sets of document fragments to the external device responsive to the user input.

[0006] In one embodiment, an electronic device is disclosed. The electronic device includes at least one memory including one or more instructions; and one or more processors including processing circuitry, wherein the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device to: obtain a user input including a text string from an external device; identify a first topic embedding from among multiple topic embeddings based on an embedding associated with the user input, wherein each embedding associated with the user input or each topic embedding is within an n-dimensional embedding space, identifying first entities associated with the user input; identify entity embeddings based on the first entities, wherein each entity embedding is within the n-dimensional embedding space; generate a respective n-sphere based on each entity embedding and the first topic, wherein each n-sphere is associated with a respective set of document fragments; retrieve one or more sets of document fragments associated with the n-spheres transmit instructions for presenting the retrieved sets of document fragments to the external device responsive to the user input.

[0007] FIG. 1 illustrates an example process for ambiguous-agnostic context disambiguation, in accordance with the presently disclosed embodiments.

[0008] FIG. 2 illustrates an example two-dimensional spatial representation of the AACD retrieval process if embeddings were represented as points, in accordance with the presently disclosed embodiments.

[0009] FIG. 3 illustrates an example two-dimensional vector representation of cosine distance as a distance metric between a topic embedding and an entity embedding, in accordance with the presently disclosed embodiments.

[0010] FIG. 4 illustrates an example process for the AACD system being applied in a chatbot.

[0011] FIG. 5 illustrates a flow diagram of a method for retrieving document fragments responsive to the user input, in accordance with the presently disclosed embodiments.

[0012] FIG. 6 illustrates an embodiment of an electronic device that may be utilized for determining sensing and communication precoders, in accordance with the presently disclosed embodiments.

[0013] For the purpose of promoting an understanding of the principles of the present disclosure, reference will now be made to the various embodiments and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the present disclosure is thereby intended, such alterations and further modifications in the illustrated system, and such further applications of the principles of the present disclosure as illustrated therein being contemplated as would normally occur to one skilled in the art to which the present disclosure relates.

[0014] It will be understood by those skilled in the art that the foregoing general description and the following detailed description are explanatory of the present disclosure and are not intended to be restrictive thereof.

[0015] Embodiments of the present disclosure will be described below in detail with reference to the accompanying drawings. For the sake of clarity, the first digit of a reference numeral of each component of the present disclosure is indicative of the Figure number, in which the corresponding component is shown. For example, reference numerals starting with digit "1" are shown at least in Figure 1. Similarly, reference numerals starting with digit "2" are shown at least in Figure 2.Ambiguous-Agnostic Context Disambiguation

[0016] According to an embodiment of the disclosure, an electronic device 600 may provide ambiguous-agnostic context disambiguation (AACD) system. The description of the electronic device 600 will be provided later with reference to Figure 6.

[0017] In an embodiment, the AACD system may provide handling for ambiguous input by recommending relevant document fragments to a user input. The AACD system may utilize a novel process to locate document fragments that are diverse yet relevant based on the user input. The AACD system may elucidate ambiguous user inputs in RAG retrieval systems by elucidating user input that lacks context and providing context about the related topic as a whole. The AACD system may identify and retrieve documents relevant to ambiguous user input. In an embodiment, the AACD system may utilize a high-dimensional vector space with document fragments and entity embeddings to identify relevant sets of retrieved document fragments. By using relative embedding distance, the AACD system may pinpoint document fragments that prioritize breadth (i.e., a quality of the documents returned that looks at the diversity of documents that relate to a user input) instead of depth. Document fragments may be identified and ordered based on relative distances between entity embeddings, topic embeddings, and input embedding. For domain specific RAG-based chatbots, responding to user input that is ambiguous and lacks context can be a primary example where the disclosed AACD system is useful. With the disclosed AACD system, chatbots may have a more diverse context retrieved to aid users that are using the chatbots to gain understanding about a particular topic. The disclosed AACD system can be used in tandem with other retrieval methods that prioritize depth to handle a variety of user inputs. Although this disclosure describes handling particular input by particular systems in a particular manner, this disclosure contemplates handling any suitable input by any suitable system in any suitable manner.

[0018] In an embodiment, the AACD system may receive, from a client system, a user input comprising a text string. For example, the AACD system may obtain, from an external device, a user input comprising a text string. The AACD system may then identify, based on an embedding associated with the user input, a first topic embedding from among a plurality of topic embeddings. Each of the embeddings associated with the user input and the plurality of topic embeddings may be within an n-dimensional embedding space. The AACD system may identify one or more first entities associated with the user input. The AACD system may further identify a plurality of entity embeddings based on the first entities. Each of the entity embeddings may be within the n-dimensional embedding space. The AACD system may then generate, based on each of the plurality of entity embeddings and the first topic, a respective n-sphere. In an embodiments, each n-sphere may be associated with a respective set of document fragments. The AACD system may retrieve one or more sets of document fragments associated with the plurality of n-spheres. The AACD system may further send (e.g. transmit), to the client system (or, to the external device), instructions for presenting the retrieved sets of document fragments responsive to the user input.

[0019] Certain technical challenges exist for ambiguous-agnostic context disambiguation. One technical challenge may include providing a diverse selection of documents while containing relevant content to a user input. The solution presented by the embodiments disclosed herein to address this challenge may be utilizing both semantic and syntax relevancy to identify and retrieve documents, as the AACD system may identify the most relevant topic based on semantic relevancy, identify related entities based on syntax relevancy, construct an n-sphere based on semantic relevancy between the most relevant topic and the related entities, and retrieve documents having embeddings within the n-sphere.

[0020] An embodiment disclosed herein may provide one or more technical advantages. A technical advantage of the embodiments may include providing diversity to explore a document corpus effectively by retrieving relevant information associated with an ambiguous query from a user, generating an informed and breadth response to allow the user to refine their query and uncover insights that may have been obscured by ambiguity, and recommending options for the user to further provide user context. A technical advantage of the embodiments may include improved stability when dealing with slightly variant user inputs. For similar inputs where there are small variations (e.g., typos or slight rewording of the question) that were introduced into the input string, the embodiments disclosed herein can continue to return identical results if the entities extracted are similar due to the weighting on entity embeddings and topic centroids instead of individual documents. In contrast, related systems based on distance metrics often returned dissimilar results. Certain embodiments disclosed herein may provide none, some, or all of the above technical advantages. One or more other technical advantages may be readily apparent to one skilled in the art in view of the figures, descriptions, and claims of the present disclosure.

[0021] In the event of an ambiguous question, relying solely on a distance metric to retrieve relevant documents can result in either less documents retrieved, or documents potentially containing poor distance scores being retrieved. When semantic similarity is used to retrieve documents, depth may be needed to create a specific answer to a non-ambiguous input. In an embodiment, when the input is ambiguous, breadth may be needed to generalize the answer. Additionally, if there are similar topics in multiple sections of a documentation, such as the topic "firewall" that is mentioned in both in "Security Manage" and "Security Service Plugin" documentation (documentation associated with cloud services), retrieval methods that prioritize depth result in the chatbot may provide a domain-specific response for one service or conflate the two and offer steps from both services.

[0022] Each section in the documentation corpus may have their own ecosystem of terms and topics. Terms such as enrollment may be used in different sections, with the term having a different definition. For example, "Security Manage" documentation may reference enrollment as a method to enroll devices into the Security Manage EMM, but "Security Mobile Enrollment" documentation may reference enrollment as part of its main service offerings for all EMMs.

[0023] There is a need for a system for handling ambiguous user questions that factors both semantic and syntax relevancy that promotes diversity in the documents returned. The disclosed AACD system may provide a diverse selection of documents while containing relevant content to a user input. Returning a breadth selection of documents may allow a chatbot to generate an informed response that recommends relevant documents and topics to direct the user further in conversation.

[0024] In an embodiment, the AACD system may return a selection of documents that prioritize breadth. These documents may be overall related to an ambiguous question from a user, while still containing relevancy based on noun-phrases in the user input. The documents retrieved by the AACD system may be used in a chatbot to create an informed response to the user to gain further context on their intent. The AACD system may ascertain information about the topic as a whole through the retrieved document fragments. The AACD system may retrieve information about the topic such as its significance in each section of documentation.

[0025] In an embodiment, the AACD system can be also an effective retrieval method for unique user context as the AACD system factors syntax distance. Syntax distance is the measure of how similar two or more entities are based on their syntax and sentence-level features. Outside the scope of ambiguous input, if there is user context that is unique to a small subset of documentation, or if there is sufficient context in the user input, the AACD system can return a more specific set of document fragment containing the correct target document fragment.

[0026] In an embodiment, the electronic device 600 may include the AACD system. For example, the memory 604 may store one or more instructions for executing the AACD system. The one or more instructions may be executed by one or more processors, individually or collectively, to cause the electronic device 600 to execute the AACD system. The electronic device 600 will be described in further detail below with reference to FIG. 6.

[0027] FIG. 1 illustrates an example process 100 for ambiguous-agnostic context disambiguation, in accordance with the presently disclosed embodiments. The process 100 may include an embedding phase 110 and a retrieval phase.

[0028] In an embodiment, the AACD system may access a plurality of documents. The AACD system may then generate a plurality of document fragments based on the plurality of documents by partitioning each of the plurality of documents into one or more document fragments. Given a set of document fragments 114 that are of natural language in a document corpus 112, the approach for the embedding phase may be to embed both document fragments 114 and entities 116 into high-dimensional vectors. These embeddings may exist in the same vector space, where a set of all document embeddings 124 and entity embeddings 120 exist. For each document fragment 114, the AACD system may extract all entities 116 from the fragment text at operation 118. As an example and not by way of limitation, the AACD system may use named entity recognition tools to extract entities 116. In an embodiment, entities 116 refer to contexts extracted from the text (e.g., noun-phrases).

[0029] The AACD system may generate entity embeddings 120 for the identified entities 116 by embedding them as word embeddings (e.g., dense vector representations) using a word embedding model at operation 122.

[0030] In an embodiment, the AACD system may generate a plurality of document embeddings for the plurality of document fragments, respectively. Each of the document embeddings may be within the n-dimensional embedding space. For each document fragment 114, the AACD system may generate its document embedding 124 by embedding its fragment text into word embeddings at operation 126. In an embodiment, document embeddings 124 may be representations of documents that are stored as vectors. The AACD may use the same embedding method used for generating the entity embeddings 120. Each document embedding 124 and each entity embedding 120 may be assigned to their respective item. The AACD system may further create mappings between document fragments 114 and document embeddings 124, and between entities 116 and entity embeddings 120.

[0031] In an embodiment, the AACD system may generate a plurality of clusters of the plurality of document embeddings by applying a clustering algorithm to the plurality of document embeddings 124. As illustrated in FIG. 1, the AACD system may apply a clustering algorithm 128 such as K-Means to the document embeddings 124 to group similar document embeddings 124 into clusters. In an embodiment, clusters may be dense groupings of documents in the vector space. The AACD system may further identify a cluster centroid for each of the clusters as a respective topic embedding 130 for that cluster. A cluster centroid may act as the topic embedding 130 for the corresponding cluster. In other words, a topic embedding 130 may be a vector that acts as the centroid of a single cluster.

[0032] In the retrieval phase 140, the AACD system may receive a user input 142. The ACCD system may parse the user input 142 to determine an input string 144 of natural language, such as "what is enrollment?" The AACD system may extract user input entities 146 from the input string 144 at operation 148. As an example and not by way of limitation, the AACD system may use named entity recognition tools to extract all user input entities 146 from the input string 144. Continuing with the example of "what is enrollment", the user input entity 146 "enrollment" may be extracted from the input string 144.

[0033] In an embodiment, the AACD system may generate a user input embedding 150 by embedding the input string 144 as a word embedding at operation 152. The AACD system may convert the input string 144 into a dense vector representation (i.e., word embedding).

[0034] In an embodiment, identifying the first topic embedding may comprise comparing a respective semantic distance between the embedding associated with the user input and each cluster centroid in the n-dimensional embedding space and identifying the first topic embedding based on a determination that the semantic distance between the embedding associated with the user input and a first cluster centroid corresponding to the first topic embedding is shortest among the semantic distances between the embedding associated with the user input and the plurality of cluster centroids. As illustrated in FIG. 1, the AACD system may compare the distance between the user input embedding 150 for the input string 144 and all topic embeddings 130 (i.e., centroids) at operation 154 to find the closest topic embedding 156 that matches the user input embedding 150. As an example and not by way of limitation, the distance may be calculated with a distance metric such as cosine distance. As another example and not by way of limitation, the distance may be calculated with a distance metric such as Euclidian distance. In an embodiment, cosine distance may be less sensitive to magnitude compared to Euclidian distance.

[0035] Once the closest topic embedding 156 has been identified, the AACD system may compare syntax distances between the user input entities 146 and the entity embedding text associated with the entities 116 at operation 158 to identify related entities 160. In an embodiments, the syntax distance may be determined based on an edit distance algorithm such as Levenshtein distance or hamming distance along with substring matches, but the present disclosure is not limited to the examples listed. In an embodiment, identifying the plurality of entity embeddings 120 may comprise comparing edit distances between the first entities 146 and text strings comprising entities and identifying a plurality of text strings comprising entities 116 as related to the first entities 146 based on the comparison. The plurality of entity embeddings 120 may correspond to the plurality of identified text strings.

[0036] Continuing with the example of "what is enrollment", the user input entity 146 "enrollment" may match with the following entities 160:

[0037] [Security Mobile Enrollment account', ' Security Manage enrollment guide', 'auto-enrollment feature', ' Security Mobile Enrollment admin role', ' devices with Security Mobile Enrollment', ' device enrollment enhancements', ' enrollment option for devices', ' enrollment for Windows devices', ' KM-Enrollment-QR-Code-5.jpg', ' convenience of device-based enrollment', ' enrollment times', ' Screen During Enrollment', ' contents during unenrollment', ' Limited Enrollment page', ' Enrollment screen', ' enrollment', ' Security Mobile Enrollment Direct download', ' Android enrollment type', ' Language selection for gesture-based enrollment with Android'].

[0038] FIG. 2 illustrates an example two-dimensional spatial representation of the AACD retrieval process 200 if embeddings were represented as points, in accordance with the presently disclosed embodiments. In the illustrated example of FIG. 2, the user input 142 is "I want to know more about my firewall". After embedding the input string 144 with a word embedding model, the closest cluster may be identified by comparing the semantic distance between the user input embedding 150 and topic embeddings 130. Once the closest topic embedding 156 has been identified, the user input entity 146 "firewall" may be extracted from the input string 144. For this example, the user input entity 146 "firewall" sufficiently matches with three entities 160: firewall exceptions, firewall, and firewall proxy. With these entities 160, the semantic distance between each corresponding entity embedding 120 and the cluster centroid of the closest topic embedding 156 may be assessed at operation 162.

[0039] The AACD system may further determine a n-sphere search radius 164. In an embodiment, an n-sphere (hypersphere) may be a n-dimensional generalization of a sphere where n refers to the number of dimensions in the vector space. An n-sphere may include a locus of points equidistant from a center. The center may be the center point of an n-sphere, where all other points around the sphere are equidistant. The radius 164 may be the distance between the center and the perimeter of the n-sphere. At operation 166, the AACD system may calculate the number of overlaps for document fragments inside the n-spheres. At operation 168, the AACD system may order the document fragments based on the overlaps associated with the respective document fragments. At operation 170, the AACD system may present the ordered document fragments responsive to the user input 142.

[0040] Utilizing both semantic and syntax relevancy to identify and retrieve documents may be an effective solution for addressing the technical challenge of providing a diverse selection of documents while containing relevant content to a user input, as the AACD system may identify the most relevant topic based on semantic relevancy, identify related entities based on syntax relevancy, construct an n-sphere based on semantic relevancy between the most relevant topic and the related entities, and retrieve documents having embeddings within the n-sphere.

[0041] In an embodiment, generating each n-sphere may comprise determining a semantic distance between each entity embedding 120 and the first topic embedding 156 in the n-dimensional embedding space, calculating a midpoint of the semantic distance, determining a radius 164 for the n-sphere as a distance between the midpoint and the first topic embedding 156, and generating the n-sphere based on the radius 164 and a centroid corresponding to the first topic embedding 156.

[0042] FIG. 3 illustrates an example two-dimensional vector representation 300 of cosine distance as a distance metric between a topic embedding and an entity embedding, in accordance with the presently disclosed embodiments. As described above, the AACD system may further construct an n-sphere to act as a search radius 164 for all entity embeddings 120 that match with the user input entity 146. For each related entity embedding 120, to construct an n-sphere, the AACD system may calculate the distance between the entity embedding 120 and the topic embedding 156 using a distance metric to form the diameter d. With the diameter between the entity embedding 120 and the topic embedding 156, the AACD system may calculate the center (e.g., midpoint 310) and the search radius 164. In an embodiment, the minimum search radius 164 may be the distance between the midpoint 310 and the topic embedding 156 but can be increased to allow for a greater breadth of documents. After the midpoint 310 and search radius 164 are identified, document fragments 114 that are within the search radius 164 can be retrieved.

[0043] As described above, for each center 310 and search radius 164, an n-sphere is constructed where n is the number of dimensions held by the embeddings. In an embodiment, retrieving the one or more sets of document fragments 114 associated with the plurality of n-spheres may comprise identifying an intersection of a cluster associated with the first topic embedding 156 and a union of the plurality of n-spheres and retrieving the one or more sets of document fragments from the intersection of the cluster and the union of the plurality of n-spheres. The AACD system may further retrieve a set of document fragments 114 that exist in the identified cluster and the intersection of the cluster and a union of all the n-spheres as below:

[0044] Document fragments = ,

[0045] whereClusteris the set of document fragments 114 that exist in the cluster, andSiis the set of document fragments 114 that exist in all n-spheres.

[0046] In an embodiment, the AACD system may determine, for each document fragment 114 associated with the retrieved sets of document fragments 114, a ranking order based on a number of overlaps of the document fragment 114 being associated with the plurality of n-spheres. Accordingly, the instructions for presenting the retrieved sets of document fragments 114 may comprise the ranking order for each document fragment 114 associated with the retrieved sets of document fragments 114. In other words, the AACD system may order the retrieved document fragments 114 based on the number of overlaps a document fragment 114 has for all n-spheres.

[0047] In an embodiment, the AACD system can be integrated with a chatbot that functions based on a large language model (LLM). The AACD system can provide the breadth to retrieved documents for ambiguous input for an LLM to generate a conversation that will guide the user to find what they are looking for. With the AACD system, the chatbot can retrieve more contextually relevant documents, and prioritize referenced services. The AACD system can allow the chatbot to follow up with the user to provide contextually aware recommendations for further conversation. As a result, a technical advantage of the AACD system may include providing diversity to explore a document corpus effectively by retrieving relevant information associated with an ambiguous query from a user, generating an informed and breadth response to allow the user to refine their query and uncover insights that may have been obscured by ambiguity, and recommending options for the user to further provide user context.

[0048] FIG. 4 illustrates an example process 400 for the AACD system being applied in a chatbot. At operation 410, the chatbot may perform targeted question and answering, where the chatbot may check question and answer pairs from SFDC. At operation 420, the chatbot may prioritize depth by checking question and answer pairs from live documents. In one scenario, the chatbot may return documents with high similarity scores at operation 430. In another scenario, the chatbot may determine that no documents found, or return documents with low similarity scores at operation 440.

[0049] After operation 440, the chatbot may prioritize breadth by utilizing the AACD system at operation 450. The AACD system may perform ambiguous resolution through conversation, question refinement, and live document retrieval.

[0050] After operation 450, the chatbot may generate a conversational response 460 based on the documents returned by the AACD system. The process 400 may then return to operation 410 to repeat the targeted question and answering, followed by operations 420.

[0051] After operation 430 where the chatbot returns documents with high similarity scores, the chatbot may generate a one-shot / zero-shot response 470. Then the process 400 may proceed to operation 480 for depth comprehension escalation, where the chatbot may generate a prompt engineered response. In dept comprehension escalation, more context may be required from the user before a response with depth can be generated.

[0052] The embodiments disclosed herein further conducted tests on various user inputs to evaluate the performance. One example user input was "I want to know about firewall." The AACD system extracted "firewall" as the entity. The AACD further found entities with syntax distance as: ['Firewall_config1', 'device firewalls', ‘domain firewall policy', ‘company-security-firewall-exceptions#security-manage', 'company-security-firewall-exceptions#security-remote-support', ‘firewall policies to Kiosk Browsers', 'firewall exception', 'firewall', 'concept-options#messaging-ports-and-your-firewall>See documentation for hostnames', ‘Security firewall policy', ‘Firewall policies', ‘firewall-management>Firewall', ‘firewall exceptions for KM', ‘firewall exemptions', ‘company-security-firewall-exceptions#security-mobile-enrollment', ‘firewall configuration', ‘Firewall exceptions for Security Configure in China', ‘concept-options#messaging-ports-and-your-firewall', ‘firewall information', ‘td id=firewall-prohibited-policy-ip>>', 'list of firewall exceptions for Security Configure', ‘firewall policies', ‘faq#our-corporate-devices-are-behind-a-firewall-how-do-i-allow-security-services-to-contact-company-servers', 'firewall restrictions on bookmarks', ‘firewall policy for webbrowser, ‘client firewalls', ‘firewall ports', ‘client firewall allowlist', ‘Firewall exceptions', ‘list of firewall exceptions for Company Care+ for Business', ‘company-security-firewall-exceptions#security-e-fota', ‘ firewall policies', ‘firewall exceptions for webbrowser', ‘firewall exceptions', ‘Name of firewall configuration to user', ‘kba-842-how-to-configure-firewall-exceptions-for-webbrowser', ‘On-device firewall management', ‘firewall', ‘Company Security firewall', ‘Firewall configuration profiles', 'Client firewall', ‘Client firewall', ‘Firewall policy', ‘company-security-firewall-exceptions#security-configure', ‘company-security-firewall-exceptions#security-asset-intelligence', ‘Firewall_internalOnly', ‘list of firewall exceptions for Security Remote Support', ‘list of firewall exceptions for Security Mobile Enrollment', ‘Firewall Rules', ‘list of firewall exceptions for Security Asset Intelligence', ‘Server firewall', ‘firewall configurations', ‘firewall issue.', ‘minimum-requirements#disable-firewall>Disable', 'firewall-allowlist-config', ‘firewalls', ‘firewall domain', ‘firewall rules', ‘id=firewall-prohibited-policy-domain>>', ‘Security firewall package', ‘Company Security firewall exceptions', ‘firewall allowlist', ‘server firewall allowlist', ‘firewall configuration events', ‘firewall settings', ‘list of firewall exceptions for Security E-FOTA', ‘Firewall Management', ‘Firewall network type', ‘firewall access', ‘firewall for servers', ‘firewall policy', ‘Firewall', ‘firewall information of L4 domains', ‘company-security-firewall-exceptions', ‘package for firewall restrictions', ‘concept-options#messaging-ports-and-your-firewall>documentation', ‘Security firewall', ‘company-security-firewall-exceptions#company-care-for-business', ‘firewall exception for True Single SKU devices', ‘control over firewall settings', ‘Firewall controls', ‘on-device firewall rules', ‘list of firewall exceptions for Security Manage', ‘Firewall requirements for Security license servers'].

[0053] The top 10 retrieved document fragments included:

[0054] https: / docs.companysecurity.com / admin / security-manage / kbas / kba-360001261908, https: / docs.companysecurity.com / admin / security-manage / kbas / kba-360001261908#security-remote-support-exceptions,

[0055] https: / docs.companysecurity.com / admin / security-mobile-enrollment / appendix / firewall-exceptions

[0056] https: / docs.companysecurity.com / admin / security-manage / configure / profile / configure-profile-policies / company-security-running-android-enterprise-policies#firewall

[0057] https: / docs.companysecurity.com / admin / security-admin-portal / get-started / company-security-firewall-exceptions#security-mobile-enrollment

[0058] https: / docs.companysecurity.com / admin / security-admin-portal / get-started / company-security-firewall-exceptions#security-manage

[0059] https: / docs.companysecurity.com / admin / security-admin-portal / get-started / company-security-firewall-exceptions#security-configure

[0060] https: / docs.companysecurity.com / admin / security-admin-portal / get-started / company-security-firewall-exceptions#company-care+-for-business

[0061] https: / docs.companysecurity.com / admin / security-admin-portal / get-started / company-security-firewall-exceptions#security-e-fota

[0062] https: / docs.companysecurity.com / admin / security-admin-portal / get-started / company-security-firewall-exceptions.

[0063] The AACD also identified other topic information. For the topic firewall, the following services were identified: [('SM', 14), ('SAP', 14), ('SPE', 5), ('SC', 3), ('USP', 3), ('SAI', 2), ('SSP', 2), ('SME', 1), ('SCPE', 1), ('EFOTA', 1), ('SRS', 1)]. With this information, information that the services Security Manage (SM) and Security Admin Portal (SAP) have the most information about firewall. Having this overall knowledge can be helpful for a chatbot to generate an informed response to further conversation. Additionally, for this case, the service Security Admin Portal contains document fragments that focus on various firewall exceptions for multiple services.

[0064] Another example user input was "I have an issue with an error code." The AACD system extracted "issue" and "error code" as the entities. The AACD further found entities with syntax distance as: [‘compromise issues', 'SysDump issue', ‘Server issue', ‘Email formatting issues', ‘Exchange Active Sync configuration issues on Security devices', ‘app-issues', ‘breakdown by issue type', ‘Wi-Fi-related issues', ‘assets / 22.08-groups-app-issue-events.png', ‘enrollment flow issue', ‘battery charging issues', ‘issues with Security licenses', ‘issues with firmware rollout', ‘sync issues with search engine, ‘Privacy issue', ‘Security Manage Command issue', ‘Issues', ‘Charge issues event', ‘issue reports', ‘server issue.', ‘issues in countries', ‘support for issues', ‘system issues', ‘ISSUES', ‘Issue report dashboard settings', ‘at-a-glance view of device issues', ‘Factory Reset issue', ‘license seat count issues', ‘exchange issues', ‘issues with Security Service Plugin', ‘security issue', ‘Security Service Plugin issue', ‘Issues with Kiosk Browser mode', ‘barcode capture issues', ‘Wi-Fi connectivity issues', ‘assets / 22.05-app-issues-expanded.png', ‘issues through analyses of logs', ‘issue as #',' ERROR_iOS_ISSUE_TEST', ‘enrollment issues.', ‘device driver issues', ‘permissions issues', 'date of issue', ‘issues with Bluetooth', ‘sync issue', ‘issue thresholds', ‘Issues with Company e-mail app', ‘Groups with app issue', ‘Sync Service issue for AD Groups', ‘Cloud connector issue', ‘thresholds-todays-issues', ‘signs of issues', ‘KSP-related issues', ‘network issue', ‘configuration issues', ‘license utilization issue', ‘issues with Security E-FOTA', ‘Device Issue Report', ‘network timing issue', ‘Groups with app issues events', ‘server issue', ‘Security Platform for Enterprise issues', ‘Enrollment issues', ‘Policy combination issue', ‘company-issued', ‘security-code-bytes-troubleshooting-device-issues', ‘issue within SyncManager', ‘Security Manage Security Manage console login issue', ‘nature of issue with device', ‘application issues', ‘issue events by app', ‘client app issues', ‘ Issuer', ‘Storage issue', 'security issues', ‘issuer sign value', ‘bug issues', ‘network connectivity issues', ‘battery consumption issues', ‘liability issues', ‘license seat count issue', ‘Security Manage Bookmark issue', ‘compliance issue', ‘Android issues', ‘Issue', ‘data collection issues', 'issue', ‘battery issues', ‘device issue report', ‘issue events by firmware version', 'wildcard issue in Security Manage', ‘scan issues', ‘nature of device issue', ‘issues on devices', ‘data about battery charging issues', ‘Security Platform for Enterprise issue', ‘categories of app issues', ‘sign-in credentials issue', ‘Network issue', ‘issue reports for device groups', ‘Issues with Kiosk Browser Screensave', ‘IssuerDN', ‘storage issue', 'server network latency issues', ‘data battery charging issue data', 'domain issue', ‘issue links', ‘connectivity issue', ‘support issue', ‘Issue date', ‘deployment issues', ‘Custom Animation issue', ‘Language issue', ‘workflow issues', ‘security issues', ‘ enrollment issue with', ‘number of issue events', ‘Security Manage profile configuration issue', ‘ issues', ‘patches for Android-related security issues', ‘account issues', ‘Troubleshoot issues with DeX customization features', ‘Device environment issue', ‘app issue', ‘issue events', ‘Issue opening PDF from Kiosk', ‘issues with EMM group sync', ‘Kiosk Browser Mode Issue', ‘Issue with message templates', ‘enrollment issues', ‘assets / 22.08-app-issues-firmware-version.png', ‘issues with setup', ‘App issues', ‘network traffic issues', ‘devices with issues', ‘conflict issues', 'sync issues', ‘email alerts for battery-related issues', ‘App installation issue', ‘Charge issue events', ‘copyright issues', ‘known-issue-missing-certificates-after-updating-company-work', 'Issue with Devices', ‘Manage App Block Issue', ‘Cloud Connector sync issue', ‘troubleshooting of network issues', ‘device list issue', ‘issue category', ‘assets / 22.08-todays-issues.png', ‘feedback issue', 'battery issues', ‘search issue', ‘device issues', ‘Issue with JavaScript alert on Kiosk Multi App Bookmarks', ‘ Issue Date', ‘sync issues between Security Manage', ‘assets / 22.05-app-issues-app-version.png', ‘issues with firmware updates', ‘Charge issues', ‘app issues', ‘issue first hand', ‘boot issue', ‘battery-related issues', ‘ Issue with user creation process in', ‘battery charging issue events', ‘battery drainage issues', 'case of connectivity issues', ‘compatibility issues', 'User Certificate Issue', ‘issues', 'app-issues', ‘ stability issue', 'connection issue', ‘Mobile Admin portal issue', ‘connectivity issues', ‘assets / 22.05-app-issues-app-os-versions.png', ‘device issue', ‘Android issue', ‘issues per day', ‘console issues', ‘Security Manage agent installation issues', ‘Support for battery charging issues', ‘ Issue on', ‘Connector issues', ‘alt=Groups with app issue events dashboard chart', ‘issue alongside search engine', ‘mobile network connectivity issues', ‘issues with Security Configure', ‘events by issue type', ‘Device issues', ‘password compliance issue', ‘performance issues', ‘issue in detail', ‘urgency of issues', ‘issue types', ‘Number Issue on Wi-Fi', ‘issue type', ‘Date issues', ‘action on issues', ‘Groups with app issue events', ‘issues-related-to-specific-deployments', ‘charging issues', ‘firewall issue.', ‘sync issues', ‘ISSUE', ‘Company License server connection issues', ‘issue.', ‘ Issue', ‘issue events by app versions', ‘issues events', ‘instability issues', ‘ issues with', ‘app issue data', ‘policies without issue', ‘Security Manage Remote Support Client issues', ‘account issues', ‘issues per device per day', 'customer support issues', ‘assets / 22.05-app-most-issues.png', ‘Issue type', ‘issue events dashboard chart', ‘network issues', ‘network issue.', ‘issue', ‘accordion id=how-do-i-troubleshoot-vpn-configuration-issues', ‘Apps with issue events', ‘issue in Android', ‘Security Remote Support agent error codes', ‘error code KMA_F5007', ‘error codes', ‘script error code', ‘Wear OS client error codes', ‘sample of error codes details', ‘Security Enhanced Attestation Error code', ‘console error codes', 'Client error codes', ‘Security Manage console error codes', ‘Console error codes', 'Error code', ‘details on error codes', ‘type of error code', ‘ error codes', ‘client error codes reference', ‘Error codes', ‘error code definitions', 'license error code', ‘in-console error codes', ‘Client error codes', ‘error code', ‘AppWrapper error codes'].

[0065] The top 10 retrieved document fragments included:

[0066] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / security-service-plugin / references / error-codes#android-or-security-platform-for-enterprise-issue

[0067] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / security-service-plugin / references / error-codes#exception

[0068] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / security-service-plugin / references / error-codes#device-environment-issue

[0069] https: / docs.companysecurity.com / admin / security-manage / appendix / lists-of-error-codes / client-error-codes#client

[0070] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / security-service-plugin / references / error-codes#schema-errors

[0071] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / security-service-plugin / references / error-codes#network-issue

[0072] https: / docs.companysecurity.com / admin / security-manage / appendix / lists-of-error-codes / client-error-codes#agent-enrollment

[0073] https: / docs.companysecurity.com / admin / security-manage / appendix / lists-of-error-codes / client-error-codes#provisioning-library

[0074] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / security-service-plugin / references / error-codes#creation-fail

[0075] https: / docs.companysecurity.com / admin / security-efota / troubleshoot / error-codes

[0076] The AACD also identified other topic information including related projects as: [('SSP', 14), ('SM', 12), ('EFOTA', 4), ('SC', 3), ('WSC', 2)].

[0077] The embodiments disclosed herein further compared the results from the AACD system to results generated based on cosine distance. For example, given the user input "I want to know more about Kiosk mode," a RAG system with cosine distance retrieval returned the following top 20 document fragments:

[0078] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / introduction / about-kiosk-mode

[0079] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / install-kiosk-application / install-a-kiosk-application-using-a-profile

[0080] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / introduction / manage-windows-kiosks#kiosk-use-cases-experiences

[0081] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / configure-a-kiosk-in-a-profile

[0082] https: / docs.companysecurity.com / admin / security-manage / configure / profile / configure-profile-policies / wear-os-policies#kiosk

[0083] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / introduction / types-of-kiosk-applications

[0084] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / create-kiosk-overview

[0085] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / exit-kiosk-mode#to-re-enter-kiosk-mode

[0086] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / introduction / kiosk-minimum-requirements

[0087] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / introduction / manage-windows-kiosks

[0088] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / configure-kiosk-mode-for-smartwatches

[0089] https: / docs.companysecurity.com / admin / security-manage / release-notes / 22-08#chrome-os-policy-additions

[0090] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / install-kiosk-application / install-a-kiosk-application-using-a-device-command

[0091] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / kiosk-wizard / kiosk-wizard-menu-items

[0092] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / set-up-a-windows-kiosk#configure-an-app-for-a-multi-app-kiosk

[0093] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / introduction / manage-windows-kiosks#see-also

[0094] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / create-a-kiosk-using-the-kiosk-wizard

[0095] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / introduction / manage-windows-kiosks#multi-app-kiosk-use-cases

[0096] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / kiosk-wizard / configure-kiosk-device-settings#access-kiosk-settings

[0097] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / set-up-a-windows-kiosk#see-also

[0098] In this example, document fragments with a strong correlation to Security Manage were returned. With these results, a chatbot could provide a depth response about Kiosk mode for Security Manage. However, in Company Security documentation, there exists other types of kiosk modes (e.g., web kiosk mode, multi-app kiosk mode, ProKiosk mode, integration from other services into kiosk mode). As the input was ambiguous, a breadth selection of document fragments may be required to generate an informed response.

[0099] Given the user input "I want to know more about Kiosk mode," the AACD system returned the following top 20 document fragments:

[0100] https: / docs.companysecurity.com / admin / security-manage / release-notes / 20-08#kiosk

[0101] https: / docs.companysecurity.com / admin / security-configure / security-configure-wearables / get-started / customize-your-profile / dynamic-edition-prokiosk-mode#exit-professional-kiosk-mode-ui

[0102] https: / docs.companysecurity.com / admin / security-configure / release-notes / 19-09#web-kiosk-mode

[0103] https: / docs.companysecurity.com / admin / security-configure / release-notes / 18-02

[0104] https: / docs.companysecurity.com / admin / security-configure / release-notes / 21-06#configure-enhanced-kiosk-mode-multi-app

[0105] https: / docs.companysecurity.com / admin / security-mobile-enrollment / release-notes / 18-02

[0106] https: / docs.companysecurity.com / admin / security-configure / how-to-guides / profiles / about-prokiosk-mode

[0107] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / set-up-a-windows-kiosk#set-up-a-multi-app-windows-kiosk

[0108] https: / docs.companysecurity.com / admin / security-manage / kbas / kba-360048885374#additional-information

[0109] https: / docs.companysecurity.com / admin / security-manage / new-console / references / android-policies#kiosk

[0110] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / create-kiosk-overview

[0111] https: / docs.companysecurity.com / admin / security-efota / kbas / kba-713-how-to-deploy-security-efota-client-kiosk-ksp#how-to-set-up-kiosk-mode-configure-ksp-security-manage

[0112] https: / docs.companysecurity.com / admin / security-manage / release-notes / 20-08#to-reenter-kiosk-mode

[0113] https: / docs.companysecurity.com / admin / security-configure / security-configure-wearables / get-started / customize-your-profile / dynamic-edition-prokiosk-mode#set-home-activity

[0114] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / introduction / about-kiosk-mode

[0115] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / create-kiosk-devices / set-up-a-windows-kiosk#set-up-a-single-app-windows-kiosk

[0116] https: / docs.companysecurity.com / admin / security-manage / release-notes / 19-03#simplified-kiosk-wizard-workflow

[0117] https: / docs.companysecurity.com / admin / security-configure / kbas / kba-360046494354#workaround

[0118] https: / docs.companysecurity.com / admin / security-manage / kiosk-devices / kiosk-wizard / configure-kiosk-device-settings#access-kiosk-settings

[0119] https: / docs.companysecurity.com / admin / security-manage / kbas / kba-360048885374

[0120] The AACD system also identified related projects: [('SM', 54), ('SC', 9), ('WSC', 2), ('SME', 1), ('EFOTA', 1)].

[0121] As can be seen, the AACD system provided high diversity in the retrieved document fragments, including Security Configure wearables ProKiosk mode, Security Mobile Enrollment, Web kiosk mode, multi-app kiosk mode, how to deploy Security E-FOTA to a kiosk, along with Security Manage Kiosk mode information. Given the context in the retrieved document fragments, a chatbot could provide a breadth response and recommend a couple options for the user to further provide user context. Additionally, topic-based information can be collected from the retrieved document fragments and retained in the field of related projects. Given this information, a chatbot may have further context to recommend an ordered list of relevant services to the user for this topic.

[0122] In addition, the AACD system had improved stability compared to solely using cosine distance as a distance metric when dealing with slightly variant user inputs. For similar inputs where there are small variations (e.g., typos or slight rewording of the question) that were introduced into the input string, the AACD system continued to return identical results if the entities extracted are similar due to its weighting on entity embeddings and topic centroids instead of individual documents. The improved stability when dealing with slightly variant user inputs may be another technical advantage of the AACD system. In contrast, traditional systems based on distance metrics often returned dissimilar results.

[0123] Another comparison of the results from the AACD system to results generated based on cosine distance used the input "what is Security" as an example. The top 10 retrieved document fragments with cosine distance include:

[0124] https: / docs.companysecurity.com / admin / security-asset-intelligence / faq / #purpose-of-kai

[0125] https: / docs.companysecurity.com / admin / security-suite / faq / #what-is-security-suite

[0126] https: / docs.companysecurity.com / admin / security-configure / faq / #what-is-the-purpose-of-security-configure

[0127] https: / docs.companysecurity.com / admin / security-manage / faq / #which-types-companies-best-suited-employ-security-manage

[0128] https: / docs.companysecurity.com / admin / security-manage / faq / #what-is-security-manage

[0129] https: / docs.companysecurity.com / admin / fundamentals / whitepaper / company-security-for-android / the-company-security-platform

[0130] https: / docs.companysecurity.com / admin / fundamentals / whitepaper / company-security-for-android / the-company-security-platform#security-platform-highlights

[0131] https: / docs.companysecurity.com / admin / security-guard / faq / #what-is-the-purpose-of-security-guard

[0132] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / faq / #what-version-of-security-platform-for-enterprise-is-running-on-my-device

[0133] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / faq / #video-overview-of-company-security-workspace

[0134] By contrast, the top 10 retrieved document fragments by the AACD system include:

[0135] https: / docs.companysecurity.com / admin / security-suite / faq / #what-is-security-suite

[0136] https: / docs.companysecurity.com / admin / fundamentals / kbas / kba-349-about-android-others-android-go-devices#which-security-solutions-do-android---others-devices-support?

[0137] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / kbas / kba-360048230734

[0138] https: / docs.companysecurity.com / admin / security-manage / release-notes / 18-09#miscellaneous-enhancements

[0139] https: / docs.companysecurity.com / admin / fundamentals / whitepaper / company-security-for-android / the-company-security-platform#security-platform-highlights

[0140] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / faq / #what-information-is-collected-when-security-platform-for-enterprise-communicates-with-the-server

[0141] https: / docs.companysecurity.com / admin / security-manage / configure / profile / configure-profile-policies / company-security-running-android-enterprise-policies#security-service-plugin

[0142] https: / docs.companysecurity.com / admin / security-manage / configure / basic-setup / basic-setup-overview

[0143] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / before-you-begin / user-agreements-for-android-device-management#see-also

[0144] https: / docs.companysecurity.com / admin / fundamentals / whitepaper / company-security-for-android / the-company-security-platform#learn-more

[0145] Then, given a subsequent user input "I want to know about Security," the top 10 retrieved document fragments with cosine distance include:

[0146] https: / docs.companysecurity.com / admin / security-efota / troubleshoot / get-support

[0147] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / faq / #what-version-of-security-platform-for-enterprise-is-running-on-my-device

[0148] https: / docs.companysecurity.com / admin / fundamentals / whitepaper / company-security-for-android / appendix / about-this-white-paper#revision-history

[0149] https: / docs.companysecurity.com / admin / security-manage / faq / #which-types-companies-best-suited-employ-security-manage

[0150] https: / docs.companysecurity.com / admin / security-suite / faq / #what-is-security-suite

[0151] https: / docs.companysecurity.com / admin / security-remote-support / manage-security-remote-support-devices#security-admin-portal-device-search

[0152] https: / docs.companysecurity.com / admin / security-configure / troubleshoot / get-support#documentation

[0153] https: / docs.companysecurity.com / admin / security-configure / release-notes / 23-03#security-configure-accessory-improvements

[0154] https: / docs.companysecurity.com / admin / security-admin-portal / get-started / tour-the-portal#navigate-homepage

[0155] https: / docs.companysecurity.com / admin / fundamentals / whitepaper / company-security-for-android / the-company-security-platform#security-platform-highlights

[0156] By contrast, given the subsequent user input "I want to know about Security," the top 10 retrieved document fragments by the AACD system include:

[0157] https: / docs.companysecurity.com / admin / security-suite / faq / #what-is-security-suite

[0158] https: / docs.companysecurity.com / admin / fundamentals / kbas / kba-349-about-android-others-android-go-devices#which-security-solutions-do-android---others-devices-support?

[0159] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / kbas / kba-360048230734

[0160] https: / docs.companysecurity.com / admin / security-manage / release-notes / 18-09#miscellaneous-enhancements

[0161] https: / docs.companysecurity.com / admin / fundamentals / whitepaper / company-security-for-android / the-company-security-platform#security-platform-highlights

[0162] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / faq / #what-information-is-collected-when-security-platform-for-enterprise-communicates-with-the-server

[0163] https: / docs.companysecurity.com / admin / security-manage / configure / profile / configure-profile-policies / company-security-running-android-enterprise-policies#security-service-plugin

[0164] https: / docs.companysecurity.com / admin / security-manage / configure / basic-setup / basic-setup-overview

[0165] https: / docs.companysecurity.com / admin / security-platform-for-enterprise / before-you-begin / user-agreements-for-android-device-management#see-also

[0166] https: / docs.companysecurity.com / admin / fundamentals / whitepaper / company-security-for-android / the-company-security-platform#learn-more

[0167] Although the overall intent of both user inputs is the same ("What is Security?", "I want to know about Security."), the AACD system provided consistency in the retrieved document fragments, compared to retrieval based on cosine distance which retrieved a different combination of document fragments.

[0168] The embodiments disclosed herein additionally compared depth comprehension escalation and breadth comprehension. As described earlier (e.g., with reference to FIG. 4), a chatbot may perform depth comprehension escalation. The following describes an example based on the user input "I want to know more about Kiosk mode." For comprehension, more context may be required from the user before a response with depth can be generated. Documents retrieved with semantic similarity can identify a group of relevant documents, but they may not provide any extra cues or ranking to help a user choose the most important ones when the user input is ambiguous. With semantic similarity, small changes to the sentence can have a big effect on the semantic distance result, making it less reliable for certain applications such as the ambiguous input.

[0169] As described earlier (e.g., with reference to FIG. 4), the AACD system may provide breadth comprehension. Continuing with the previous example of "I want to know more about Kiosk mode," the AACD system can include a wide range of documents relating to the full user input embedding. Compared to semantic similarity, the AACD system provided more breadth with the ambiguous input with the set of documents that were proportional per service. The AACD system also provided more granularity over document selection. The AACD system can allow users to adjust weights in both syntax and semantic matching when working with an ambiguous input for a specific dataset.

[0170] The following describes another example based on the user input "I want to know more about enrollment." Similarly, for comprehension, more context may be required from the user before a response with depth can be generated. For this user input, there were no documents returned. The response from the chatbot provided general information and advised the user to submit a support ticket. In an embodiment, the AACD system determined that the term enrollment was used in a wide variety of services. For enrollment, SME was returned as the service with the most results from the AACD system. This information can be useful in providing further context when following up with a user in conversation.

[0171] The embodiments disclosed herein additionally compared depth assumption lacking breadth and breadth assumption. The following describes an example based on the user input "I want to know more about firewall." The depth assumption was that given an ambiguous input and if enough documents are returned, the chatbot may generate an answer based on those documents. Given documents about SSP, the chatbot responded with information about how to configure Security Service Plugin. By contrast, the AACD system determined that the term firewall was used in multiple services. The AACD system identified firewall documentation across the services and could retrieve these documents as a breadth approach.

[0172] An embodiment disclosed herein additionally compared depth confusion escalation and breadth confusion. The following describes an example based on the user input "Why do I need a firewall to make a phone call?" The confusion may be that if an ambiguous input does not make sense, a depth retrieval should not be used to respond to the user. As no documents were returned, the chatbot recognized the confusion and guided users to submit a support ticket instead. In an embodiment, the AACD system can provide a breadth collection of documents across documentation in instances of confusion. The chatbot could request more information from the user, while recommending related services and documents that fall within the scope of the original input.

[0173] FIG. 5 illustrates a flow diagram of a method 500 for ambiguous-agnostic context disambiguation, in accordance with the presently disclosed embodiments. The method 500 may be performed utilizing one or more processing devices (e.g., a computing system associated with the AACD system) that may include hardware (e.g., a general purpose processor, a graphic processing unit (GPU), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a microcontroller, a field-programmable gate array (FPGA), a central processing unit (CPU), an application processor (AP), a visual processing unit (VPU), a neural processing unit (NPU), a neural decision processor (NDP), or any other processing device(s) that may be suitable for processing wireless communication data, software (e.g., instructions running / executing on one or more processors), firmware (e.g., microcode), or some combination thereof.

[0174] The method 500 may begin at operation 510 with the one or more processing devices (e.g., the AACD system). For example, in an embodiment, the AACD system may obtain, from an external device, a user input comprising a text string. For example, in an embodiment, the AACD system may receive, from a client system, a user input comprising a text string. The method 500 may then continue at operation 520 with the one or more processing devices (e.g., the AACD system). For example, in an embodiments, the AACD system may identify, based on an embedding associated with the user input, a first topic embedding from among a plurality of topic embeddings, wherein each of the embeddings associated with the user input and the topic embeddings is within an n-dimensional embedding space. In an embodiment, the identifying a first topic embedding may include comprises generating clusters of document embeddings by applying a clustering algorithm to the document embeddings. In an embodiment, the identifying a first topic embedding may include identifying a cluster centroid for each cluster as a respective topic embedding for that cluster. In an embodiment, the identifying a first topic embedding may include comparing a respective semantic distance between the embedding associated with the user input and each cluster centroid. In an embodiment, the identifying a first topic embedding may include identifying the first topic embedding based on a determination that the semantic distance between the embedding associated with the user input and a first cluster centroid corresponding to the first topic embedding is shortest among the semantic distances between the embedding associated with the user input and the cluster centroids. The method 500 may then continue at operation 530 with the one or more processing devices (e.g., the AACD system). For example, in an embodiment, the AACD system may identify one or more first entities associated with the user input. The method 500 may then continue at operation 540 with the one or more processing devices (e.g., the AACD system). For example, in an embodiments, the AACD system may identify entity embeddings based on the first entities. In an embodiment, each identified entity embedding may be within the n-dimensional embedding space. In an embodiment, the identifying may include comparing edit distances between the first entities and text strings comprising entities and identifying text strings comprising entities as related to the first entities based on the comparison, wherein the identified entity embeddings correspond to the identified text strings. The method 500 may then continue at operation 550 with the one or more processing devices (e.g., the AACD system). For example, in an embodiments, the AACD system may generate, based on each entity embedding and the first topic, a respective n-sphere, wherein each n-sphere is associated with a respective set of document fragments. In an embodiment, the generating each n-sphere may include determining a semantic distance between each entity embedding and the first topic embedding in the n-dimensional embedding space. In an embodiment, the generating each n-sphere may include calculating a midpoint of the semantic distance. In an embodiment, the generating each n-sphere may include determining a radius for the n-sphere as a distance between the midpoint and the first topic embedding. In an embodiment, the generating each n-sphere may include generating the n-sphere based on the radius and a centroid corresponding to the first topic embedding. The method 500 may then continue at operation 560 with the one or more processing devices (e.g., the AACD system). For example, in an embodiments, the AACD system may retrieve one or more sets of document fragments associated with the n-spheres. In an embodiment, the retrieving may include identifying an intersection of a cluster associated with the first topic embedding and a union of the n-spheres. In an embodiment, the retrieving may include retrieving the one or more sets of document fragments from the intersection of the cluster and the union of the n-spheres. The method 500 may further include then continue at operation 562 (not mentioned in FIG. 5) with the one or more processing devices (e.g., the AACD system). For example, in an embodiments, the AACD system may determine, for each document fragment associated with the retrieved sets of document fragments, a ranking order based on a number of overlaps of the document fragment being associated with the n-spheres. The method 500 may then continue at operation 570 with the one or more processing devices (e.g., the AACD system). For example, in an embodiment, the AACD system may transmit, to the external device, instructions for presenting the retrieved sets of document fragments based on the ranking order for each document fragment responsive to the user input. For example, in an embodiment, the AACD system may send, to the client system, instructions for presenting the retrieved sets of document fragments based on the ranking order for each document fragment responsive to the user input. An embodiment may repeat one or more operations of the method of FIG. 5, where appropriate. Although this disclosure describes and illustrates an operations of the method of FIG. 5 as occurring in an order, this disclosure contemplates any suitable operations of the method of FIG. 5 occurring in any suitable order. Moreover, although this disclosure describes and illustrates an example method for ambiguous-agnostic context disambiguation including the one or more operations of the method of FIG. 5, this disclosure contemplates any suitable method for ambiguous-agnostic context disambiguation including any suitable operations, which may include all, some, or none of the operations of the method of FIG. 5, where appropriate. Furthermore, although this disclosure describes and illustrates particular components, devices, or systems carrying out particular operations of the method of FIG. 5, this disclosure contemplates any suitable combination of any suitable components, devices, or systems carrying out any suitable operations of the method of FIG. 5.

[0175] Systems and Methods

[0176] FIG. 6 illustrates an embodiment of an electronic device 600 that may be utilized for determining sensing and communication precoders, in accordance with the presently disclosed embodiments. In an embodiment, one or more electronic system 600 perform one or more operations of one or more methods described or illustrated herein. In an embodiment, one or more electronic device 600 provide functionality described or illustrated herein. In an embodiment, software running on one or more electronic device 600 performs one or more operations of one or more methods described or illustrated herein or provides functionality described or illustrated herein. An embodiment include one or more portions of one or more electronic device 600. Herein, reference to an electronic device may encompass a computing device, and vice versa, where appropriate. Moreover, reference to an electronic device may encompass one or more electronic devices, where appropriate.

[0177] This disclosure contemplates any suitable number of one or more electronic device 600. This disclosure contemplates electronic device 600 taking any suitable physical form. As example and not by way of limitation, electronic device 600 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (e.g., a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented / virtual reality device, or a combination of two or more of these. Where appropriate, electronic device 600 may include one or more electronic devices 600; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks.

[0178] Where appropriate, one or more electronic device 600 may perform without substantial spatial or temporal limitation one or more operations of one or more methods described or illustrated herein. As an example, and not by way of limitation, one or more electronic device 600 may perform in real time or in batch mode one or more operations of one or more methods described or illustrated herein. One or more electronic device 600 may perform at different times or at different locations one or more operations of one or more methods described or illustrated herein, where appropriate.

[0179] In an embodiment, electronic device 600 includes a processor 602, memory 604, an input / output (I / O) interface 608, a communication interface 610, and a bus 612. Although this disclosure describes and illustrates an electronic device having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable electronic device having any suitable number of any suitable components in any suitable arrangement.

[0180] In an embodiment, processor 602 includes hardware for executing instructions, such as those making up a computer program. As an example, and not by way of limitation, to execute instructions, processor 602 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 604; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 604. In an embodiment, processor 602 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor 602 including any suitable number of any suitable internal caches, where appropriate. As an example, and not by way of limitation, processor 602 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory 604, and the instruction caches may speed up retrieval of those instructions by processor 602.

[0181] Data in the data caches may be copies of data in memory 604 for instructions executing at processor 602 to operate on; the results of previous instructions executed at processor 602 for access by subsequent instructions executing at processor 602 or for writing to memory 604; or other suitable data. The data caches may speed up read or write operations by processor 602. The TLBs may speed up virtual-address translation for processor 602. In an embodiment, processor 602 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 602 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor 602 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 602. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0182] In an embodiment, memory 604 includes main memory for storing instructions for processor 602 to execute or data for processor 602 to operate on. As an example, and not by way of limitation, electronic device 600 may load instructions from storage 606 or another source (such as, for example, another electronic device 600) to memory 604. Processor 602 may then load the instructions from memory 604 to an internal register or internal cache. To execute the instructions, processor 602 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor 602 may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor 602 may then write one or more of those results to memory 604. In an embodiment, processor 602 executes only instructions in one or more internal registers or internal caches or in memory 604 (as opposed to storage 606 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 604 (or elsewhere).

[0183] One or more memory buses (which may each include an address bus and a data bus) may couple processor 602 to memory 604. Bus 612 may include one or more memory buses, as described below. In an embodiment, one or more memory management units (MMUs) reside between processor 602 and memory 604 and facilitate accesses to memory 604 requested by processor 602. In an embodiment, memory 604 includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory 604 may include one or more memory devices, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

[0184] In an embodiment, memory 604 includes mass storage for data or instructions. As an example, and not by way of limitation, memory 604 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Memory 604 may include removable or non-removable (or fixed) media, where appropriate. Memory 604 may be internal or external to electronic device 600, where appropriate. In an embodiment, memory 604 is non-volatile, solid-state memory. In an embodiment, memory 604 includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass memory 604 taking any suitable physical form. Memory 604 may include one or more storage control units facilitating communication between processor 602 and memory 604, where appropriate. Where appropriate, memory 604 may include one or more memory 604. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

[0185] In an embodiment, I / O interface 608 includes hardware, software, or both, providing one or more interfaces for communication between electronic device 600 and one or more I / O devices. Electronic device 600 may include one or more of these I / O devices, where appropriate. One or more of these I / O devices may enable communication between a person and electronic device 600. As an example, and not by way of limitation, an I / O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I / O device or a combination of two or more of these. An I / O device may include one or more sensors. This disclosure contemplates any suitable I / O devices and any suitable I / O interfaces 608 for them. Where appropriate, I / O interface 608 may include one or more device or software drivers enabling processor 602 to drive one or more of these I / O devices. I / O interface 608 may include one or more I / O interfaces 608, where appropriate. Although this disclosure describes and illustrates a particular I / O interface, this disclosure contemplates any suitable I / O interface.

[0186] In an embodiment, communication interface 610 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between electronic device 600 and one or more other electronic device 600 or one or more networks. As an example, and not by way of limitation, communication interface 610 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface 610 for it.

[0187] As an example, and not by way of limitation, electronic device 600 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), an ultra-wideband network (UWB), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, electronic device 600 may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these. Electronic device 600 may include any suitable communication interface 610 for any of these networks, where appropriate. Communication interface 610 may include one or more communication interfaces 610, where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface.

[0188] In an embodiment, bus 612 includes hardware, software, or both coupling components of electronic device 600 to each other. As an example, and not by way of limitation, bus 612 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus 612 may include one or more buses 612, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

[0189] Miscellaneous

[0190] Herein, "or" is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, "A or B" means "A, B, or both," unless expressly indicated otherwise or indicated otherwise by context. Moreover, "and" is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, "A and B" means "A and B, jointly or severally," unless expressly indicated otherwise or indicated otherwise by context.

[0191] Herein, "automatically" and its derivatives means "without human intervention," unless expressly indicated otherwise or indicated otherwise by context.

[0192] The embodiments disclosed herein are only examples, and the scope of this disclosure is not limited to them. Embodiments according to the invention are in particular disclosed in the attached claims directed to a method, a storage medium, a system and a computer program product, wherein any feature mentioned in one claim category, e.g. method, can be claimed in another claim category, e.g. system, as well. The dependencies or references back in the attached claims are chosen for formal reasons only. However, any subject matter resulting from a deliberate reference back to any previous claims (in particular multiple dependencies) can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims. The subject-matter which can be claimed comprises not only the combinations of features as set out in the attached claims but also any other combination of features in the claims, wherein each feature mentioned in the claims can be combined with any other feature or combination of other features in the claims. Furthermore, any of the embodiments and features described or depicted herein can be claimed in a separate claim and / or in any combination with any embodiment or feature described or depicted herein or with any of the features of the attached claims.

[0193] The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, feature, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative. Additionally, although this disclosure describes or illustrates particular embodiments as providing particular advantages, particular embodiments may provide none, some, or all of these advantages.

[0194] The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps but may include other steps not expressly listed or inherent to such process or method. Similarly, one or more devices or sub-systems or elements or structures or components proceeded by "comprises... a" does not, without more constraints, preclude the existence of other devices or other sub-systems or other elements or other structures or other components or additional devices or additional sub-systems or additional elements or additional structures or additional components.

[0195] It should be appreciated that the blocks in each flowchart and combinations of the flowcharts may be performed by one or more computer programs which include computer-executable instructions. The entirety of the one or more computer programs may be stored in a single memory or the one or more computer programs may be divided with different portions stored in different multiple memories.

[0196] Any of the functions or operations described herein can be processed by one processor or a combination of processors. The one processor or the combination of processors is circuitry performing processing and includes circuitry like an application processor (AP), a communication processor (CP), a graphical processing unit (GPU), a neural processing unit (NPU), a microprocessor unit (MPU), a system on chip (SoC), an IC, or the like.

[0197] The processor may include various processing circuitry and / or multiple processors.　 For example, as used herein, including the claims, the term "processor" may include various processing circuitry, including at least one processor, wherein one or more of at least one processor, individually and / or collectively in a distributed manner, may be configured to perform various functions described herein. As used herein, when "a processor", "at least one processor", and "one or more processors" are described as being configured to perform numerous functions, these terms cover situations, for example and without limitation, in which one processor performs some of recited functions and another processor(s) performs other of recited functions, and also situations in which a single processor may perform all recited functions. Additionally, the at least one processor may include a combination of processors performing various of the recited / disclosed functions, e.g., in a distributed manner.　 At least one processor may execute program instructions to achieve or perform various functions.

[0198] According to an embodiment of the disclosure, a method is disclosed. The method, by an electronic device may include obtaining, from an external device, a user input comprising a text string. The method may include receiving, from a client system, a user input comprising a text string. The method may include identifying, based on an embedding associated with the user input, a first topic embedding from among a plurality of topic embeddings, wherein each of the embeddings associated with the user input and the plurality of topic embeddings is within an n-dimensional embedding space. The method may include identifying one or more first entities associated with the user input. The method may include identifying a plurality of entity embeddings based on the first entities, wherein each of the entity embeddings is within the n-dimensional embedding space. The method may include identifying a plurality of entity embeddings based on the one or more first entities, wherein each of the plurality of entity embeddings is within the n-dimensional embedding space. The method may include generating, based on each of the plurality of entity embeddings and the first topic, a respective n-sphere, wherein each n-sphere is associated with a respective set of document fragments. The method may include retrieving one or more sets of document fragments associated with the plurality of n-spheres. The method may include transmitting, to the external device, instructions for presenting the retrieved sets of document fragments responsive to the user input. The method may include sending, to the client system, instructions for presenting the retrieved sets of document fragments responsive to the user input.

[0199] According to an embodiment of the disclosure, the method may include accessing a plurality of documents. The method may include generating a plurality of document fragments based on the plurality of documents by partitioning each of the plurality of documents into one or more document fragments. The one or more sets of document fragments may be retrieved from the plurality of document fragments.

[0200] According to an embodiment of the disclosure, the method may include generating a plurality of document embeddings for the plurality of document fragments, respectively, wherein each of the document embeddings is within the n-dimensional embedding space. The method may include generating a plurality of clusters of the plurality of document embeddings by applying a clustering algorithm to the plurality of document embeddings. The method may include identifying a cluster centroid for each of the clusters as a respective topic embedding for that cluster.

[0201] According to an embodiment of the disclosure, the method may include comparing a respective semantic distance between the embedding associated with the user input and each cluster centroid in the n-dimensional embedding space. The method may include identifying the first topic embedding based on a determination that the semantic distance between the embedding associated with the user input and a first cluster centroid corresponding to the first topic embedding is shortest among the semantic distances between the embedding associated with the user input and the plurality of cluster centroids.

[0202] According to an embodiment of the disclosure, the method may include comparing edit distances between the first entities and text strings comprising entities. The method may include identifying a plurality of text strings comprising entities as related to the first entities based on the comparison. The plurality of entity embeddings may correspond to the plurality of identified text strings.

[0203] According to an embodiment of the disclosure, the method may include determining a semantic distance between each entity embedding and the first topic embedding in the n-dimensional embedding space. The method may include determining a midpoint of the semantic distance. The method may include calculating a midpoint of the semantic distance. The method may include determining a radius for the n-sphere as a distance between the midpoint and the first topic embedding. The method may include generating the n-sphere based on the radius and a centroid corresponding to the first topic embedding.

[0204] According to an embodiment of the disclosure, the method may include identifying an intersection of a cluster associated with the first topic embedding and a union of the plurality of n-spheres. The method may include retrieving the one or more sets of document fragments from the intersection of the cluster and the union of the plurality of n-spheres.

[0205] According to an embodiment of the disclosure, the method may include determining, for each document fragment associated with the retrieved sets of document fragments, a ranking order based on a number of overlaps of the document fragment being associated with the plurality of n-spheres. The instructions for presenting the retrieved sets of document fragments may include the ranking order for each document fragment associated with the retrieved sets of document fragments.

[0206] According to an embodiment of the disclosure, an electronic device 600 is disclosed. The electronic device 600 may include at least one memory 604 including one or more instructions; and one or more processors 602 including processing circuitry. According to an embodiment of the disclosure, the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device 600 to: obtain, from an external device, a user input comprising a text string; identify, based on an embedding associated with the user input, a first topic embedding from among a plurality of topic embeddings, wherein each of the embeddings associated with the user input and the plurality of topic embeddings is within an n-dimensional embedding space; identify one or more first entities associated with the user input; identify a plurality of entity embeddings based on the one or more first entities, wherein each of the plurality of entity embeddings is within the n-dimensional embedding space; generate, based on each of the plurality of entity embeddings and the first topic, a respective n-sphere, wherein each n-sphere is associated with a respective set of document fragments; retrieve one or more sets of document fragments associated with the plurality of n-spheres; and transmit, to the external device, instructions for presenting the retrieved sets of document fragments responsive to the user input.

[0207] According to an embodiment of the disclosure, the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device 600 to: access a plurality of documents; and generate a plurality of document fragments based on the plurality of documents by partitioning each of the plurality of documents into one or more document fragments; wherein the one or more sets of document fragments are retrieved from the plurality of document fragments.

[0208] According to an embodiment of the disclosure, the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device 600 to: generate a plurality of document embeddings for the plurality of document fragments, respectively, wherein each of the document embeddings is within the n-dimensional embedding space; generate a plurality of clusters of the plurality of document embeddings by applying a clustering algorithm to the plurality of document embeddings; and identify a cluster centroid for each of the clusters as a respective topic embedding for that cluster.

[0209] According to an embodiment of the disclosure, the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device 600 to: compare a respective semantic distance between the embedding associated with the user input and each cluster centroid in the n-dimensional embedding space; and identify the first topic embedding based on a determination that the semantic distance between the embedding associated with the user input and a first cluster centroid corresponding to the first topic embedding is shortest among the semantic distances between the embedding associated with the user input and the plurality of cluster centroids.

[0210] According to an embodiment of the disclosure, the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device 600 to: compare edit distances between the first entities and text strings comprising entities; and identify a plurality of text strings comprising entities as related to the first entities based on the comparison; wherein the plurality of entity embeddings correspond to the plurality of identified text strings.

[0211] According to an embodiment of the disclosure, the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device 600 to: determine a semantic distance between each entity embedding and the first topic embedding in the n-dimensional embedding space; determine a midpoint of the semantic distance; determine a radius for the n-sphere as a distance between the midpoint and the first topic embedding; and generate the n-sphere based on the radius and a centroid corresponding to the first topic embedding.

[0212] According to an embodiment of the disclosure, the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device 600 to: identify an intersection of a cluster associated with the first topic embedding and a union of the plurality of n-spheres; and retrieve the one or more sets of document fragments from the intersection of the cluster and the union of the plurality of n-spheres.

[0213] According to an embodiment of the disclosure, the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device 600 to: determine, for each document fragment associated with the retrieved sets of document fragments, a ranking order based on a number of overlaps of the document fragment being associated with the plurality of n-spheres, wherein the instructions for presenting the retrieved sets of document fragments comprise the ranking order for each document fragment associated with the retrieved sets of document fragments.

[0214] According to an embodiment of the disclosure, a computer-readable medium containing instructions, wherein the instructions, when executed by at least one processor, cause the electronic device 600 to perform the method is disclosed.

[0215] According to an embodiment of the disclosure, a computing system may include, in communication with the memory 204, one or more non-transitory computer-readable storage media including instructions; and one or more processors coupled to the storage media, the one or more processors configured to execute the instructions to: receive, from a client system, a user input comprising a text string; identify, based on an embedding associated with the user input, a first topic embedding from among a plurality of topic embeddings, wherein each of the embeddings associated with the user input and the plurality of topic embeddings is within an n-dimensional embedding space; identify one or more first entities associated with the user input; identify a plurality of entity embeddings based on the first entities, wherein each of the entity embeddings is within the n-dimensional embedding space; generate, based on each of the plurality of entity embeddings and the first topic, a respective n-sphere, wherein each n-sphere is associated with a respective set of document fragments; retrieve one or more sets of document fragments associated with the plurality of n-spheres; and send, to the client system, instructions for presenting the retrieved sets of document fragments responsive to the user input.

[0216] According to an embodiment of the disclosure, the processors are operable when executing the instructions to access a plurality of documents; and generate a plurality of document fragments based on the plurality of documents by partitioning each of the plurality of documents into one or more document fragments; wherein the one or more sets of document fragments are retrieved from the plurality of document fragments.

[0217] According to an embodiment of the disclosure, the processors are operable when executing the instructions to generate a plurality of document embeddings for the plurality of document fragments, respectively, wherein each of the document embeddings is within the n-dimensional embedding space; generate a plurality of clusters of the plurality of document embeddings by applying a clustering algorithm to the plurality of document embeddings; and identify a cluster centroid for each of the clusters as a respective topic embedding for that cluster.

[0218] According to an embodiment of the disclosure, identifying the first topic embedding comprises: comparing a respective semantic distance between the embedding associated with the user input and each cluster centroid in the n-dimensional embedding space; and identifying the first topic embedding based on a determination that the semantic distance between the embedding associated with the user input and a first cluster centroid corresponding to the first topic embedding is shortest among the semantic distances between the embedding associated with the user input and the plurality of cluster centroids.

[0219] According to an embodiment of the disclosure, identifying the plurality of entity embeddings comprises: comparing edit distances between the first entities and text strings comprising entities; and identifying a plurality of text strings comprising entities as related to the first entities based on the comparison; wherein the plurality of entity embeddings correspond to the plurality of identified text strings.

[0220] According to an embodiment of the disclosure, generating each n-sphere comprises: determining a semantic distance between each entity embedding and the first topic embedding in the n-dimensional embedding space; calculating a midpoint of the semantic distance; determining a radius for the n-sphere as a distance between the midpoint and the first topic embedding; and generating the n-sphere based on the radius and a centroid corresponding to the first topic embedding.

[0221] According to an embodiment of the disclosure, retrieving the one or more sets of document fragments associated with the plurality of n-spheres may include identifying an intersection of a cluster associated with the first topic embedding and a union of the plurality of n-spheres; and retrieving the one or more sets of document fragments from the intersection of the cluster and the union of the plurality of n-spheres.

[0222] According to an embodiment of the disclosure, the processors are further operable when executing the instructions to: determine, for each document fragment associated with the retrieved sets of document fragments, a ranking order based on a number of overlaps of the document fragment being associated with the plurality of n-spheres, wherein the instructions for presenting the retrieved sets of document fragments comprise the ranking order for each document fragment associated with the retrieved sets of document fragments.

[0223] According to an embodiment of the disclosure, a computer-readable non-transitory storage media comprising instructions executable by a processor associated with a computing system to: receive, from a client system, a user input comprising a text string; identify, based on an embedding associated with the user input, a first topic embedding from among a plurality of topic embeddings, wherein each of the embeddings associated with the user input and the plurality of topic embeddings is within an n-dimensional embedding space; identify one or more first entities associated with the user input; identify a plurality of entity embeddings based on the first entities, wherein each of the entity embeddings is within the n-dimensional embedding space; generate, based on each of the plurality of entity embeddings and the first topic, a respective n-sphere, wherein each n-sphere is associated with a respective set of document fragments; retrieve one or more sets of document fragments associated with the plurality of n-spheres; and send, to the client system, instructions for presenting the retrieved sets of document fragments responsive to the user input.

[0224] According to an embodiment of the disclosure, the software is operable when executed to: access a plurality of documents; and generate a plurality of document fragments based on the plurality of documents by partitioning each of the plurality of documents into one or more document fragments; wherein the one or more sets of document fragments are retrieved from the plurality of document fragments.

[0225] According to an embodiment of the disclosure, the software is operable when executed to: generate a plurality of document embeddings for the plurality of document fragments, respectively, wherein each of the document embeddings is within the n-dimensional embedding space; generate a plurality of clusters of the plurality of document embeddings by applying a clustering algorithm to the plurality of document embeddings; and identify a cluster centroid for each of the clusters as a respective topic embedding for that cluster.

[0226] According to an embodiment of the disclosure, identifying the first topic embedding comprises: comparing a respective semantic distance between the embedding associated with the user input and each cluster centroid in the n-dimensional embedding space; and identifying the first topic embedding based on a determination that the semantic distance between the embedding associated with the user input and a first cluster centroid corresponding to the first topic embedding is shortest among the semantic distances between the embedding associated with the user input and the plurality of cluster centroids.

Claims

1.A method comprising, by an electronic device:obtaining, from an external device, a user input comprising a text string;identifying, based on an embedding associated with the user input, a first topic embedding from among a plurality of topic embeddings, wherein each of the embeddings associated with the user input and the plurality of topic embeddings is within an n-dimensional embedding space (510);identifying one or more first entities associated with the user input (520);identifying a plurality of entity embeddings based on the one or more first entities, wherein each of the plurality of entity embeddings is within the n-dimensional embedding space (530);generating, based on each of the plurality of entity embeddings and the first topic, a respective n-sphere, wherein each n-sphere is associated with a respective set of document fragments (540);retrieving one or more sets of document fragments associated with the plurality of n-spheres (550); andtransmitting, to the external device, instructions for presenting the retrieved sets of document fragments responsive to the user input (560).2.The method of Claim 1, further comprising:accessing a plurality of documents; andgenerating a plurality of document fragments based on the plurality of documents by partitioning each of the plurality of documents into one or more document fragments;wherein the one or more sets of document fragments are retrieved from the plurality of document fragments.3.The method of Claim 2, further comprising:generating a plurality of document embeddings for the plurality of document fragments, respectively, wherein each of the document embeddings is within the n-dimensional embedding space;generating a plurality of clusters of the plurality of document embeddings by applying a clustering algorithm to the plurality of document embeddings; andidentifying a cluster centroid for each of the clusters as a respective topic embedding for that cluster.4.The method of any one of claims 1 to 3, wherein identifying the first topic embedding comprises:comparing a respective semantic distance between the embedding associated with the user input and each cluster centroid in the n-dimensional embedding space; andidentifying the first topic embedding based on a determination that the semantic distance between the embedding associated with the user input and a first cluster centroid corresponding to the first topic embedding is shortest among the semantic distances between the embedding associated with the user input and the plurality of cluster centroids.5.The method of any one of claims 1 to 4, wherein identifying the plurality of entity embeddings comprises:comparing edit distances between the first entities and text strings comprising entities; andidentifying a plurality of text strings comprising entities as related to the first entities based on the comparison;wherein the plurality of entity embeddings correspond to the plurality of identified text strings.6.The method of any one of claims 1 to 5, wherein generating each n-sphere comprises:determining a semantic distance between each entity embedding and the first topic embedding in the n-dimensional embedding space;determining a midpoint of the semantic distance;determining a radius for the n-sphere as a distance between the midpoint and the first topic embedding; andgenerating the n-sphere based on the radius and a centroid corresponding to the first topic embedding.7.The method of any one of claims 1 to 6, wherein retrieving the one or more sets of document fragments associated with the plurality of n-spheres comprises:identifying an intersection of a cluster associated with the first topic embedding and a union of the plurality of n-spheres; andretrieving the one or more sets of document fragments from the intersection of the cluster and the union of the plurality of n-spheres.8.The method of any one of claims 1 to 7, further comprising:determining, for each document fragment associated with the retrieved sets of document fragments, a ranking order based on a number of overlaps of the document fragment being associated with the plurality of n-spheres, wherein the instructions for presenting the retrieved sets of document fragments comprise the ranking order for each document fragment associated with the retrieved sets of document fragments.9.An electronic device (600) comprising:at least one memory (604) including one or more instructions; andone or more processors (602) including processing circuitry, wherein the one or more instructions executed by the one or more processor individually or collectively, to cause the electronic device (600) to:obtain, from an external device, a user input comprising a text string;identify, based on an embedding associated with the user input, a first topic embedding from among a plurality of topic embeddings, wherein each of the embeddings associated with the user input and the plurality of topic embeddings is within an n-dimensional embedding space;identify one or more first entities associated with the user input;identify a plurality of entity embeddings based on the one or more first entities, wherein each of the plurality of entity embeddings is within the n-dimensional embedding space;generate, based on each of the plurality of entity embeddings and the first topic, a respective n-sphere, wherein each n-sphere is associated with a respective set of document fragments;retrieve one or more sets of document fragments associated with the plurality of n-spheres; andtransmit, to the external device, instructions for presenting the retrieved sets of document fragments responsive to the user input.10.The electronic device (600) of Claim 9, wherein the one or more instructions executed by the one or more processors individually or collectively, to cause the electronic device (600) to:access a plurality of documents; andgenerate a plurality of document fragments based on the plurality of documents by partitioning each of the plurality of documents into one or more document fragments;wherein the one or more sets of document fragments are retrieved from the plurality of document fragments.11.The electronic device (600) of Claim 10, wherein the one or more instructions executed by the one or more processors individually or collectively, to cause the electronic device (600) to:generate a plurality of document embeddings for the plurality of document fragments, respectively, wherein each of the document embeddings is within the n-dimensional embedding space;generate a plurality of clusters of the plurality of document embeddings by applying a clustering algorithm to the plurality of document embeddings; andidentify a cluster centroid for each of the clusters as a respective topic embedding for that cluster.12.The electronic device (600) of any one of claims 9 to 11, wherein the one or more instructions executed by the one or more processors individually or collectively, to cause the electronic device (600) to:compare a respective semantic distance between the embedding associated with the user input and each cluster centroid in the n-dimensional embedding space; andidentify the first topic embedding based on a determination that the semantic distance between the embedding associated with the user input and a first cluster centroid corresponding to the first topic embedding is shortest among the semantic distances between the embedding associated with the user input and the plurality of cluster centroids.13.The electronic device (600) of any one of claims 9 to 12, wherein the one or more instructions executed by the one or more processors individually or collectively, to cause the electronic device (600) to:compare edit distances between the first entities and text strings comprising entities; andidentify a plurality of text strings comprising entities as related to the first entities based on the comparison;wherein the plurality of entity embeddings correspond to the plurality of identified text strings.14.The electronic device (600) of any one of claims 9 to 13, wherein the one or more instructions executed by the one or more processors individually or collectively, to cause the electronic device (600) to:determine a semantic distance between each entity embedding and the first topic embedding in the n-dimensional embedding space;determine a midpoint of the semantic distance;determine a radius for the n-sphere as a distance between the midpoint and the first topic embedding; andgenerate the n-sphere based on the radius and a centroid corresponding to the first topic embedding.15.A computer-readable medium containing instructions, wherein the instructions, when executed by at least one processor, cause the electronic device (600) to perform the method of any one of claims 1 to 8.

Images