System, computer implementation method, and computer program (asymmetric relevance via fragment-level execution of a neural sentence encoder)
Asymmetric execution of a neural sentence encoder on sentence fragments addresses the issue of imposter sentences in natural language processing by ensuring embeddings accurately reflect the semantic content of the query, reducing misidentification.
Patent Information
- Application Number
- JP2026072644
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-08
- Filing Date
- 2026-04-24
- Publication Date
- 2026-11-19
AI Technical Summary
Existing natural language processing techniques for embedding searches often identify imposter sentences, which are undesirable due to their high probability of incorrectly identifying sentences that share many words with the query but do not actually answer it.
Implementing asymmetric relevance via fragment-wise execution of a neural sentence encoder, where the encoder is run on individual fragments of the sentence instead of the entire sentence, generating embeddings for each fragment and fusing them to produce a sentence embedding, reducing the likelihood of identifying imposter sentences.
This approach significantly reduces the misidentification of imposter sentences by focusing on the spatial relationship of important words within the sentence, ensuring that embeddings accurately reflect the semantic content of the query.
Smart Images

Figure 2026182787000001_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The subject matter of the present disclosure relates to natural language processing. Given a natural language query, sentences that answer the natural language query can be identified via embedding search through a sentence database. Unfortunately, existing techniques for performing such embedding searches often identify imposter sentences, which may be undesirable. SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION
[0002] Systems or techniques that can facilitate embedding searches that are less susceptible to imposter sentences may be desirable. MEANS FOR SOLVING THE PROBLEM
[0003] The following presents a summary to provide a basic understanding of one or more embodiments. This summary is not intended to identify key or critical elements, or delineate any scope of particular embodiments or any scope of the claims. Its sole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments described herein, a device, system, method, or apparatus is described that can facilitate asymmetric relevance via fragment-wise execution of a neural sentence encoder.
[0004] A system is provided according to one or more embodiments. In various embodiments, the system may include a processor capable of executing computer executable instructions stored in non-temporary computer-readable memory. In various cases, such execution may facilitate various operations for the processor. In various cases, such operations may include accessing a first text passage and a neural statement encoder. In various embodiments, such operations may include generating a first embedding for the first text passage based on independently executing the neural statement encoder on multiple fragments of the first text passage. In various cases, independently executing the neural statement encoder on multiple fragments may result in multiple fragment embeddings, and the first embedding for the first text passage may be based on fusing multiple fragment embeddings. In various cases, fusing may involve concatenating multiple fragment embeddings, or dimensionally reduced versions thereof, together.
[0005] In various forms, the above system can be reformatted, reformatted, or otherwise implemented as a computer implementation method or as a computer program product. [Brief explanation of the drawing]
[0006] [Figure 1] A block diagram of an exemplary and non-limiting system that facilitates asymmetric relationships through the execution of a neural sentence encoder in fragment units, according to one or more embodiments described herein, is shown.
[0007] [Figure 2] An exemplary and non-limiting block diagram illustrating how a neural sentence encoder may be trained according to one or more embodiments described herein is shown.
[0008] [Figure 3]A block diagram of an exemplary and non-limiting system, including sentence embeddings and query embeddings, which facilitates asymmetric relationships through fragment-unit execution of a neural sentence encoder, is shown according to one or more embodiments described herein.
[0009] [Figure 4] An exemplary and non-limiting block diagram illustrating how sentence embeddings and query embeddings may be generated asymmetrically according to one or more embodiments described herein is shown. [Figure 5] An exemplary and non-limiting block diagram illustrating how sentence embeddings and query embeddings may be generated asymmetrically according to one or more embodiments described herein is shown. [Figure 6] An exemplary and non-limiting block diagram illustrating how sentence embeddings and query embeddings may be generated asymmetrically according to one or more embodiments described herein is shown. [Figure 7] An exemplary and non-limiting block diagram illustrating how sentence embeddings and query embeddings may be generated asymmetrically according to one or more embodiments described herein is shown.
[0010] [Figure 8] An exemplary and non-limiting block diagram is provided below illustrating how similarity can be calculated between sentence embeddings and query embeddings according to one or more embodiments described herein.
[0011] [Figure 9] An exemplary and non-limiting block diagram is shown below illustrating how the localization of subsentences in a query response may be facilitated according to one or more embodiments described herein.
[0012] [Figure 10]A flowchart of an exemplary and non-limiting computer implementation method for facilitating asymmetric relevance through fragment-unit execution of a neural sentence encoder is shown according to one or more embodiments described herein.
[0013] [Figure 11] A block diagram of an exemplary and non-limiting operating environment is shown, which may facilitate one or more embodiments described herein. [Modes for carrying out the invention]
[0014] The following detailed description is illustrative only and is not intended to limit the embodiments or the application or use of the embodiments. Furthermore, it is not intended to be bound by any express or implied information presented in the preceding background or summary section or the detailed description section.
[0015] One or more embodiments are described herein with reference to the drawings, and similar reference numerals are used here to refer to similar elements throughout. Numerous specific details are given in the following description for illustrative purposes and to provide a better understanding of one or more embodiments. However, it is clear that in various cases one or more embodiments may be carried out without these specific details.
[0016] Given a natural language query (e.g., a plaintext question or interrogative), it may be desirable to identify a sentence in a sentence database (e.g., a document corpus) that semantically or substantially answers the natural language query. Such identification can be facilitated through embedding lookup. In particular, a neural sentence encoder can be trained to generate embeddings (e.g., latent vector representations) for input sentences. Thus, the neural sentence encoder can be run on each sentence in the sentence database, thereby producing its respective embedding for each sentence in the sentence database. In addition, the neural sentence encoder can be run on a natural language query, thereby producing an embedding for the natural language query. The embedding for the natural language query can then be compared (e.g., via cosine similarity or Euclidean distance) to that of each sentence in the sentence database; and any sentence in the sentence database with the closest or most similar embedding to that of the natural language query may be considered to semantically or substantially answer the natural language query.
[0017] Unfortunately, as the inventors of the various embodiments described herein have recognized, existing techniques for performing such embedded searches often or frequently identify imposter sentences. In other words, existing techniques can be considered to have an unacceptably high probability of returning sentences that, while initially appearing to be related to the natural language query, do not actually answer the natural language query. As an example, consider the natural language query "Is Jane Doe a blacksmith?". A sentence that correctly or accurately answers such a query might be "Renowned blacksmith Jane Doe honed her craft during a years-long apprenticeship in her youth." On the other hand, the imposter sentence could be, "Over the weekend, Jane Smith, a noted veterinarian in the tri-county area, tirelessly worked to save a black-tailed doe and newborn fawn." Because the natural language query shares many words with both the correct sentence and the imposter sentence (e.g., "jane," "doe," "black," "smith"), any degree of similarity between the embedding in the natural language query and that of the correct sentence can be close to or nearly the same as the degree of similarity between the embedding in the natural language query and that of the imposter sentence. In some cases, the latter may slightly surpass the former. In such situations, the imposter sentence is incorrectly selected as the answer to the natural language query. Such vulnerability to imposter sentences can be considered undesirable or disadvantageous.
[0018] Accordingly, systems or techniques that can facilitate embedding search that is less susceptible to imposter sentences may be desirable.
[0019] Various embodiments described herein may address one or more of these technical problems. Various embodiments described herein may include systems, computer-implemented methods, apparatuses, or computer program products that can facilitate asymmetric relevance via fragment-wise execution of a neural sentence encoder. Indeed, the present inventors have recognized why existing techniques are vulnerable to imposter sentences, and have devised various embodiments described herein to reduce or eliminate such vulnerabilities. Specifically, the present inventors have recognized that such existing techniques are susceptible to the influence of imposter sentences because existing techniques symmetrically execute the neural sentence encoder on natural language queries and sentences that can potentially answer those natural language queries. In other words, existing techniques retrieve symmetric relevance between sentences and queries.
[0020] More specifically, given a query and a sentence that can answer that query, existing techniques generate a first embedding by feeding the entire query into a neural sentence encoder, and similarly generate a second embedding by feeding the entire sentence into the neural sentence encoder. In other words, such existing techniques run the neural sentence encoder on the sentence in the same way they run it on the query, hence the terms “symmetric execution” and “symmetric relevance.” The inventors have recognized that the symmetric execution of such a neural sentence encoder can be seen as causing, or forcing, the subsequent embedding search to look for the sentence most similar to the query. For example, if the similarity score between the first and second embeddings is above a threshold, the sentence can be seen as semantically or substantially similar to the query and therefore answer the query. In contrast, if the similarity score between the first and second embeddings is below a threshold, the sentence can be seen as semantically or substantially dissimilar to the query and therefore not answer the query. Because imposter statements often share many words with queries, their embeddings are likely to be similar to those in the query, which can lead to them being inaccurately or incorrectly identified as answers to the query.
[0021] However, the present inventors have made the following observations: any major or important words in a query can be considered to have some fixed spatial relationship to each other; any of those major or important words that are present in a correct sentence tend to have the same fixed spatial relationship to each other; and any of those major or important words that are present in an imposter sentence tend to be physically significantly further apart from each other than that fixed spatial relationship. By way of example, consider again the above example where the query is "Is Jane Doe a blacksmith?". In such an example, "jane" and "doe" are physically close to each other in the query (e.g., are adjacent words), in a correct sentence "jane" and "doe" have the same physical proximity to each other, but in an imposter sentence "jane" and "doe" are not physically close to each other (e.g., there are a large number of intervening words between them). Similarly, in such an example, "black" and "smith" are physically close to each other in the query, in a correct sentence "black" and "smith" have the same physical proximity to each other, but in an imposter sentence "black" and "smith" are not physically close to each other.
[0022] With these observations in mind, the inventors recognized that by performing a neural sentence encoder asymmetrically, imposter sentences can be more reliably distinguished from correct sentences, and as a result, any subsequent embedding search performed can be considered to be searching for a sentence containing the query rather than a sentence similar to the query. In other words, the various embodiments described herein can search for asymmetric relationships between sentences and queries. Given a query and a sentence that can answer the query, such asymmetric execution can be achieved by: performing a neural sentence encoder on the query, thereby producing an embedding of the query; performing a neural sentence encoder on individual or independent fragments of the sentence, rather than on the entire sentence, thereby producing an embedding for each fragment; and generating an embedding of the sentence by fusing the embeddings of the fragments together. In such cases, the similarity calculation between the embedding of the query and that of the sentence can be considered to reveal whether something similar to the query is located somewhere in the sentence, as opposed to revealing whether the query is similar to the entire sentence. In fact, if the main or important words of a query are present in a sentence and are physically close to or adjacent to each other within the sentence (for example, if the sentence correctly answers the query), then there is a very high probability that these main or important words are located within the same fragment, which is why at least one embedding in the fragment will have a high degree of similarity to that of the query. In contrast, if the main or important words of a query are present in a sentence but are physically separated or far apart from each other within the sentence (for example, if the sentence is an imposter), then there is a very high probability that these main or important words are located within different fragments or separate fragments, which is why neither embedding in any of the fragments will have a high degree of similarity to that of the query.Therefore, by performing a neural sentence encoder on a sentence in a different way than it is performed on a query (for example, by physically subdividing the sentence into fragments and encoding each fragment independently, while the query as a whole is encoded), any subsequent embedding lookup can be considered more robust or resistant to imposter sentences (for example, the probability of incorrectly returning an imposter sentence as an answer to a query may be reduced).
[0023] The various embodiments described herein can be considered as computerized tools (e.g., any suitable combination of computer executable hardware or computer executable software) that can facilitate asymmetric relationships through the execution of a neural sentence encoder on a fragment-by-fragment basis. In various embodiments, such computerized tools may have an access component or an execution component.
[0024] In various embodiments, a neural sentence encoder may exist. In various embodiments, a neural sentence encoder may exhibit any suitable deep learning internal architecture. For example, a neural sentence encoder may include any suitable number of any suitable type of layers (e.g., an input layer, one or more hidden layers, and an output layer, any of which may be a convolutional layer, a fully connected layer, a long short-term memory (LSTM) layer, a transformer layer, a nonlinear layer, a pooling layer, a batch normalization layer, or a padding layer). As another example, a neural sentence encoder may include any suitable number of neurons in various layers (e.g., different layers may have the same or different numbers of neurons). As yet another example, a neural sentence encoder may include any suitable activation function in various neurons (e.g., softmax, sigmoid, hyperbolic tangent, normalized linear unit) (e.g., different neurons may have the same or different activation functions). As yet another example, a neural sentence encoder may include any appropriate interneuron connections or interlayer connections (e.g., forward connections, skip connections, recurrent connections).
[0025] Regardless of its specific internal architecture, a neural sentence encoder may be configured to take text sentences as input and produce embeddings of those text sentences as output. In various embodiments, the embeddings produced by the neural sentence encoder in response to input sentences can be considered as any suitable mathematical quantity (e.g., scalar, vector, matrix, tensor, tokenization, or any suitable combination thereof) that numerically represents at least some substantial or semantic aspects of the input sentence in a fixed-dimensional form. In other words, the embedding may have a fixed number of numerical elements, regardless of the number of words that make up the input sentence. To put it another way, the embedding may be considered a latent vector representation of such an input sentence. In some non-restrictive examples, the embedding may be a vector of 384 elements, a vector of 768 elements, or a vector of 1024 elements.
[0026] In various embodiments, sentences and natural language queries can exist. In various embodiments, a sentence may be an unstructured or plain text declarative clause or description that semantically or substantially describes something (e.g., some factoid). In various cases, a natural language query may be an unstructured or plain text interrogative clause or question that asks about something or otherwise seeks confirmation. In various cases, a sentence may or may not substantially or semantically answer a natural language query.
[0027] In various embodiments, it may be desirable to utilize a neural sentence encoder to determine whether a sentence answers a natural language query. As described herein, computerized tools can achieve such determination.
[0028] In various embodiments, the access component of a computerized tool may have electronic access to sentences and natural language queries. For example, the access component may receive, retrieve, or otherwise obtain sentences and natural language queries from any suitable centralized or distributed data structure (e.g., graph data structures, relational data structures, hybrid data structures). Similarly, the access component may have electronic access to a neural sentence encoder. For example, the access component may electronically interface with or communicate with a neural sentence encoder (e.g., send electronic commands to a neural sentence encoder and read electronic signals from a neural sentence encoder). In any case, the access component may be considered a conduit through which other components of the computerized tool can electronically interact with (e.g., read, write, edit, copy, manipulate, execute, enable, disable, modify) sentences, natural language queries, or neural sentence encoders.
[0029] In various embodiments, the execution component of a computerized tool may electronically utilize a neural sentence encoder to generate embeddings for sentences and natural language queries. In various embodiments, the execution component may do so through the asymmetric execution of the neural sentence encoder.
[0030] In particular, the execution component may run a neural sentence encoder on a natural language query, and such execution may result in query embedding. More specifically, the execution component may feed a natural language query to the input layer of a neural sentence encoder, the natural language query may complete a forward pass through one or more hidden layers of the neural sentence encoder, and the output layer of the neural sentence encoder may compute query embedding based on the activations generated by one or more hidden layers. In either case, the query embedding may be a fixed-size numerical or latent vector representation of the natural language query.
[0031] Here, in various ways, the execution component may refrain from running a neural sentence encoder on a sentence. Instead, the execution component may subdivide or decompose a sentence into multiple sentence fragments. In various cases, the execution component may achieve this through any suitable text analysis technique. In any case, a sentence can be considered as a sequence of words, and each of the multiple sentence fragments can be considered as an unbroken subsequence of words from the sentence. That is, any given sentence fragment may contain fewer words than a sentence, any words in that given sentence fragment may be ordered with respect to one another in the same way that those words are ordered in a sentence, and two words of a sentence in that given sentence fragment cannot be separated by any word of a sentence that is not present in that given sentence fragment.
[0032] In various ways, an execution component may run a neural sentence encoder independently or separately on each of multiple sentence fragments, and such an execution may result in multiple fragment embeddings. More specifically, for any given sentence fragment, the execution component may feed the sentence fragment to the input layer of a neural sentence encoder, the sentence fragment may complete forward propagation through one or more hidden layers of the neural sentence encoder, and the output layer of the neural sentence encoder may calculate each fragment embedding based on the activations generated by one or more hidden layers, where such fragment embedding may be a fixed-size numerical or latent vector representation of the given sentence fragment. By running a neural sentence encoder on each of multiple sentence fragments in this way, the execution component can generate multiple fragment embeddings. Note that each of the multiple fragment embeddings may have the same form, size, or dimensions as the query embedding (for example, after all, the embedding is a fixed-dimensional numerical representation of the text).
[0033] In various cases, the execution component can merge multiple fragment embeddings together, thereby producing a sentence embedding that can be considered a latent vector representation of the statement. In some cases, such merging may involve concatenating multiple fragment embeddings together to form a sentence embedding. In such situations, the dimension of the sentence embedding (e.g., the number of internal numeric elements) may be an integer multiple of that of the query embedding. In other cases, such merging may involve projecting each of the multiple fragment embeddings into a lower-dimensional form or space, thereby producing multiple compressed fragment embeddings; and concatenating multiple compressed fragment embeddings together to form a sentence embedding. In such situations, the lower-dimensional form or space may be selected based on how many fragments the statement has been subdivided into, and as a result, the dimension of the concatenated sentence embedding is equal to that of the query embedding. Regardless of which of these fusion techniques is implemented, sentence embeddings can be considered to be generated asymmetrically or differently from query embeddings (for example, sentence embeddings are generated by running a neural sentence encoder on independent fragments of a sentence; query embeddings are not generated by running a neural sentence encoder on independent fragments of a natural language query).
[0034] In various embodiments, the execution component can calculate a similarity score between a query embedding and a sentence embedding. In situations where the dimension of the sentence embedding is an integer multiple of that of the query embedding, the execution component can concatenate copies of the query embedding together to produce a concatenation having dimensions equal to those of the sentence embedding, and the similarity score may be equal to, or otherwise based on, the cosine similarity or the reciprocal of the Euclidean distance between the concatenation of the sentence embedding and the copies of the query embedding. On the other hand, in situations where the dimension of the sentence embedding is equal to that of the query embedding, the similarity score may be equal to the cosine similarity or the reciprocal of the Euclidean distance between the sentence embedding and the query embedding. In either case, the similarity score may be a scalar, and its magnitude indicates the level of semantic or substantial similarity between the natural language query and any individual fragment of the sentence (e.g., a larger similarity score may indicate a higher similarity; a smaller similarity score may indicate a lower similarity instead). In various cases, if the similarity score exceeds an arbitrary appropriate threshold, the execution component can conclude that at least one of the sentence fragments is similar to the natural language query, meaning that the sentence is likely to answer the natural language query. Conversely, if the similarity score falls below an arbitrary appropriate threshold, the execution component can conclude that none of the sentence fragments are similar to the natural language query, meaning that the sentence is likely not to answer the natural language query.
[0035] In some embodiments, in response to a similarity score exceeding a threshold, the execution component can finely localize which particular fragment of the sentence is similar to the natural language query. In situations where the dimension of the sentence embedding is an integer multiple of that of the query embedding, such localization can be facilitated by calculating the respective similarity score between the query embedding and each of several fragment embeddings (e.g., this enables the identification of the particular fragment most similar to the natural language query). In situations where the dimension of the sentence embedding is equal to that of the query embedding, such localization can be facilitated by projecting the query embedding into a lower-dimensional form or space and calculating the respective similarity score between its compressed version and each of several compressed fragment embeddings (e.g., as above, this enables the identification of the particular fragment most similar to the natural language query).
[0036] In either case, the asymmetric execution of the neural sentence encoder (e.g., execution on the entire natural language query, as opposed to execution on individual fragments of the sentence) can reduce the likelihood that sentence embeddings are similar to query embeddings in situations where the sentence is imposter (e.g., the sentence shares many words with the natural language query but does not actually answer the natural language query).
[0037] Various embodiments described herein may be employed to use hardware or software to solve problems that are highly technical in nature (e.g., to facilitate asymmetric relevance through the execution of a neural sentence encoder in fragment units), are not abstract, and cannot be performed as a set of mental actions by humans. Furthermore, some of the processes to be performed may be executed by specialized computers (e.g., a neural sentence encoder is a particular type of deep learning neural network).
[0038] For example, such a defined task may include: accessing a sentence and a neural sentence encoder by a device operably coupled to a processor; and generating a first embedding for the sentence based on the device independently executing the neural sentence encoder on multiple sentence fragments of the sentence. In various embodiments, independently executing the neural sentence encoder on multiple sentence fragments may result in multiple fragment embeddings, and the first embedding for the sentence may be based on fusing the multiple fragment embeddings. In various cases, such fusing may involve the device concatenating the multiple fragment embeddings together. In other cases, such fusing may involve the device projecting each of the multiple fragment embeddings into a dimensionality-reduced form, thereby resulting in multiple compressed fragment embeddings; and concatenating the multiple compressed fragment embeddings together. In various cases, such defined actions may include: the device accessing a natural language query; generating a second embedding for the natural language query based on running a neural sentence encoder on the natural language query, and not based on independently running a neural sentence encoder on multiple fragments of the natural language query; calculating a similarity score between a first embedding and a second embedding; and determining, in response to the similarity score exceeding a threshold, that the sentence semantically answers the natural language query. In some embodiments, such defined actions may involve the device, in response to the determination that the sentence semantically answers the natural language query, calculating a second similarity score between the second embedding and each of the multiple fragment embeddings, thereby generating multiple second similarity scores; and locating the most semantically relevant of the multiple sentence fragments based on identifying which of the multiple second similarity scores exceeds a threshold.
[0039] Such defined actions are inherently computerized. Indeed, artificial neural networks (e.g., neural sentence encoders) are inherently computerized constructs that include a specific software-oriented architecture (e.g., input layers, hidden layers, or output layers, each of which may consist of trainable or untrainable internal parameters such as convolutional or LSTM layers). Artificial neural networks cannot be trained or performed in any reasonable or practical way by human mind or by a human using only pen and paper, without a computer. Furthermore, the field of natural language processing itself concerns enabling computers to semantically understand and substantially respond to natural language queries, comments, commands, or instructions. Discussing the field of natural language processing without computers would be utterly meaningless.
[0040] Furthermore, one or more embodiments described herein may integrate the teachings described herein into practical applications. As mentioned above, given a natural language query, it may be desirable to identify a sentence that semantically answers that natural language query. Existing techniques facilitate this through a neural sentence encoder. Specifically, the natural language query is encoded into a query embedding by the neural sentence encoder, the sentence is encoded into a sentence embedding by the neural sentence encoder, and whether a sentence semantically answers a natural language query is determined based on the degree of similarity between the query embedding and the sentence embedding. Unfortunately, as the inventors have recognized, such existing techniques frequently identify imposter sentences as answers to natural language queries. Thus, existing techniques may be considered to have technical problems (e.g., due to insufficiently high answer accuracy).
[0041] Fortunately, the various embodiments described herein address or otherwise improve upon this technical problem. In fact, the inventors have discovered why existing techniques are particularly susceptible to imposter sentences and have devised the various embodiments described herein to reduce or eliminate such susceptibility. Specifically, the inventors have recognized that existing techniques implement symmetric execution of a neural sentence encoder, meaning that such existing techniques execute a neural sentence encoder on a sentence in the same way that they execute a neural sentence encoder on a natural language query. More specifically, such existing techniques generate query embeddings by feeding the entire natural language query to the neural sentence encoder, and similarly generate sentence embeddings by feeding the entire sentence to the neural sentence encoder. The inventors have recognized that such symmetric execution forces subsequent similarity calculations to substantially determine or measure how similar the entire sentence is to the natural language query. Because imposter statements often share many words with natural language queries, embeddings of such imposter statements are more likely to resemble embeddings of natural language queries when generated via symmetric execution.
[0042] To address this problem, various embodiments described herein may instead implement asymmetric execution of the neural sentence encoder, meaning that such embodiments may execute the neural sentence encoder on a sentence in a different way than how the neural sentence encoder is executed on a natural language query. More specifically, various embodiments may generate query embeddings by feeding the entire natural language query to the neural sentence encoder, but may refrain from generating sentence embeddings by feeding the entire sentence to the neural sentence encoder. Instead, various embodiments may generate sentence embeddings by subdividing the sentence into multiple fragments and feeding each of those fragments independently to the neural sentence encoder. This may result in an embedding for each fragment of the sentence, and the sentence embedding may be obtained by fusing the embeddings of the fragments together. A degree of similarity may then be calculated between the query embedding and the sentence embedding, where the degree of similarity may be considered to indicate how similar the natural language query is to any of the fragments of the sentence. Imposter sentences often share many words with natural language queries, but these shared words are often located farther apart in the imposter sentence than they are in the natural language query. Therefore, it is unlikely that all of these shared words in an imposter sentence will be present within a single fragment of the imposter sentence, which in turn means that sentence embeddings of imposter sentences are less likely to resemble query embeddings when generated via asymmetric execution. In contrast, correct sentences often share many words with natural language queries, but these shared words are often located as close together in the correct sentences as they are in the natural language query. Therefore, it is likely that all of these shared words in a correct sentence will be present within a single fragment of the correct sentence, which in turn means that sentence embeddings of correct sentences are more likely to resemble query embeddings when generated via asymmetric execution.
[0043] In other words, asymmetric execution of a neural sentence encoder can force subsequent embedding searches to look for sentences that contain something similar to the natural language query, and can be considered to misidentify imposter sentences less frequently. In contrast, symmetric execution of a neural sentence encoder taught by existing techniques can force subsequent embedding searches to look for sentences that are similar to the natural language query as a whole, and can be considered to misidentify imposter sentences more frequently.
[0044] For at least these reasons, the various embodiments described herein provide concrete and tangible technical improvements or effects in the field of natural language processing. Thus, such embodiments certainly constitute useful and practical applications of computers.
[0045] It should be understood that the drawings and disclosure herein illustrate non-limiting examples of various embodiments. Furthermore, it should be understood that the drawings are not necessarily drawn to scale.
[0046] Figure 1 shows a block diagram of an exemplary and non-limiting system 100 that can facilitate asymmetric relationships through the execution of a neural sentence encoder in fragment units according to one or more embodiments described herein.
[0047] In various embodiments, a neural sentence encoder 104 may exist. In various embodiments, the neural sentence encoder 104 may exhibit any suitable deep learning internal architecture. In fact, in various cases, the neural sentence encoder 104 may have an input layer, one or more hidden layers, and an output layer. In various cases, any of such layers may be connected to one another by any suitable interneuron connections or interlayer connections, such as forward connections, skip connections, or recurrent connections. Furthermore, in various cases, any of such layers may be any suitable type of neural network layer having any suitable learnable or trainable internal parameters. For example, any of such an input layer, one or more hidden layers, or output layer may be a convolutional layer, and its learnable or trainable parameters may be a convolutional kernel. As another example, any of such an input layer, one or more hidden layers, or output layer may be a fully connected layer, and its learnable or trainable parameters may be a weight matrix or bias values. As yet another example, any of such an input layer, one or more hidden layers, or output layer could be a batch normalization layer, and its learnable or trainable parameter could be a shift coefficient or a scale coefficient. As yet another example, any of such an input layer, one or more hidden layers, or output layer could be an LSTM layer, and its learnable or trainable parameter could be an input state weight matrix or a hidden state weight matrix. As yet another example, any of such an input layer, one or more hidden layers, or output layer could be a transformer layer, and its learnable or trainable parameter could be a single-head or multi-head attention block or other weight matrix. Furthermore, in various cases, any of such layers could be any suitable type of neural network layer with any suitable fixed or untrainable internal parameters. For example, any of such an input layer, one or more hidden layers, or output layer could be a nonlinear layer, a padding layer, a pooling layer, or a concatenated layer.
[0048] Regardless of the specific internal architecture implemented within the neural sentence encoder 104 (e.g., a particular number, type, or configuration of layers), the neural sentence encoder 104 may be configured to accept a sentence of text as input and to produce, as output, a fixed-dimensional embedding that numerically represents such input text sentence. To ensure that such fixed-dimensional embedding is correct or accurate (e.g., to ensure that such fixed-dimensional embedding actually reflects the semantic or substantial content of those input text sentences), the neural sentence encoder 104 may undergo any appropriate type of training. Various non-limiting aspects of such training are illustrated with reference to Figure 2.
[0049] Figure 2 shows an exemplary and non-limiting block diagram 200 illustrating how the neural sentence encoder 104 may be trained according to one or more embodiments described herein. In particular, the neural sentence encoder 104 may be trained in an unsupervised manner (for example, by the asymmetric relevance system 102 or by any other suitable computation device) by utilizing the neural sentence decoder 206.
[0050] In various embodiments, the neural sentence decoder 206 can exhibit any suitable deep learning internal architecture. In fact, in various cases, the neural sentence decoder 206 may have an input layer, one or more hidden layers, and an output layer. In various cases, any of such layers may be connected to one another by any suitable interneuron connections or interlayer connections, such as forward connections, skip connections, or recurrent connections. Furthermore, in various cases, any of such layers may be any suitable type of neural network layer having any suitable learnable or trainable internal parameters. For example, any of the input layer, one or more hidden layers, or output layer may be a convolutional layer, and its learnable or trainable parameters may be a convolutional kernel. As another example, any of the input layer, one or more hidden layers, or output layer may be a fully connected layer, and its learnable or trainable parameters may be a weight matrix or bias values. As yet another example, any of the input layer, one or more hidden layers, or output layer may be a batch normalization layer, and its learnable or trainable parameters may be a shift coefficient or a scale coefficient. As yet another example, any of such an input layer, one or more hidden layers, or output layer may be an LSTM layer, and its learnable or trainable parameters may be an input state weight matrix or a hidden state weight matrix. As yet another example, any of such an input layer, one or more hidden layers, or output layer may be a transformer layer, and its learnable or trainable parameters may be a single-head or multi-head attention block or other weight matrix. Furthermore, in various cases, any of such layers may be any suitable type of neural network layer having any suitable fixed or untrainable internal parameters. For example, any of such an input layer, one or more hidden layers, or output layer may be a nonlinearity layer, a padding layer, a pooling layer, or a concatenated layer.
[0051] Regardless of the specific internal architecture implemented within the neural sentence decoder 206, it may be configured to accept a fixed-dimensional embedding as input and produce a text sentence as output, which is numerically represented by such input fixed-dimensional embedding. In other words, the neural sentence decoder 206 can be considered to perform the functional inverse of the neural sentence encoder 104.
[0052] In various embodiments, the neural sentence encoder 104 and the neural sentence decoder 206 can be trained simultaneously with each other or separately in cooperation with each other. Before training begins, the trainable internal parameters of the neural sentence encoder 104 and the neural sentence decoder 206 (e.g., convolution kernel, weight matrix, bias values) can be initialized in any suitable manner (e.g., via random initialization).
[0053] In various embodiments, training sentence 202 may exist. In various embodiments, training sentence 202 may be any suitable grammatically consistent plain sentence or sequence of words in natural language. In various cases, training sentence 202 may contain or consist of any suitable number of words. In various cases, training sentence 202 may be any suitable semantic type. As some non-restrictive examples, training sentence 202 may be an interrogative sentence (e.g., a question), a declarative sentence (e.g., a description), an imperative sentence (e.g., a command), or an exclamatory sentence (e.g., a strong expression of emotion).
[0054] In either case, the neural sentence encoder 104 may be executed on the training sentence 202, thereby causing the neural sentence encoder 104 to generate an output 204. More specifically, the training sentence 202 may be supplied to or routed to the input layer of the neural sentence encoder 104, the training sentence 202 may complete forward propagation through one or more hidden layers of the neural sentence encoder 104, and the output layer of the neural sentence encoder 104 may calculate the output 204 based on the activation map or feature map provided by one or more hidden layers of the neural sentence encoder 104 during such forward propagation.
[0055] It should be noted that the format, size, or dimensions of output 204 may be determined by the number, arrangement, size, or other characteristics of the neurons, convolutional kernels, attention blocks, or other internal parameters of the output layer (or any other layer) of the neural sentence encoder 104. Therefore, by adding, removing, or otherwise adjusting the characteristics of the output layer (or any other layer) of the neural sentence encoder 104, output 204 may be forced to have any desired format, size, or dimensions. In particular, output 204 may be forced to have a fixed format, size, or dimensions (for example, to be a fixed-length vector). Thus, output 204 may be considered a predicted or inferred embedding that potentially captures (in the view of the neural sentence encoder 104) or otherwise numerically represents the substantial or semantic content of the training sentence 202. It should be noted that if the neural sentence encoder 104 has not been trained at all or with little training, output 204 may be extremely inaccurate. In other words, output 204 may fail to truly or accurately capture, or otherwise numerically represent, the substantive or semantic content of training sentence 202.
[0056] In various embodiments, the neural sentence decoder 206 may be performed on the output 204, thereby causing the neural sentence decoder 206 to generate an output 208. More specifically, the output 204 may be fed to or routed to the input layer of the neural sentence decoder 206, the output 204 may complete forward propagation through one or more hidden layers of the neural sentence decoder 206, and the output layer of the neural sentence decoder 206 may compute the output 208 based on the activation map or feature map provided by one or more hidden layers of the neural sentence decoder 206 during such forward propagation.
[0057] As described above, it should be noted that the format, size, or dimensions of output 208 may be determined by the number, arrangement, size, or other characteristics of the neurons, convolutional kernels, attention blocks, or other internal parameters of the output layer (or any other layer) of the neural sentence decoder 206. Therefore, by adding, removing, or otherwise adjusting the characteristics of the output layer (or any other layer) of the neural sentence decoder 206, output 208 may be forced to have any desired format, size, or dimensions. In particular, output 208 may be forced to be a variable-length word sequence. Thus, output 208 may be considered a predicted or inferred plaintext or natural language sentence that conveys any substantial or semantic content (in the view of the neural sentence decoder 206) that is potentially captured by output 204 or otherwise represented numerically. As described above, it should be noted that if the neural sentence decoder 206 has not been trained at all or with little training, output 208 may be extremely inaccurate. In other words, output 208 may fail to truly or accurately convey, through plain text or natural language, any semantic or substantive content that is potentially captured by output 204 or otherwise represented numerically.
[0058] Furthermore, it should be noted that the neural sentence encoder 104 and the neural sentence decoder 206 may collectively form a bottleneck structure: the neural sentence encoder 104 attempts to convert the training sentence 202 into a predicted embedding; and the neural sentence decoder 206 attempts to reconstruct the training sentence 202 from that predicted embedding.
[0059] In any case, an error 210 (e.g., mean absolute error, mean squared error, cross-entropy error) can be calculated between the output 208 and the training sentence 202. In various cases, the trainable intrinsic parameters of both the neural sentence encoder 104 and the neural sentence decoder 206 can be incrementally updated based on the error 210 via backpropagation (e.g., stochastic gradient descent).
[0060] In various cases, such execution and updating procedures may be repeated for any appropriate number of training sentences. This may eventually lead to the trainable internal parameters of the neural sentence decoder 206 being iteratively optimized to accurately reconstruct sentences from input embeddings; and the trainable internal parameters of the neural sentence encoder 104 being iteratively optimized to accurately generate embeddings from input sentences. In various embodiments, any appropriate training batch size, any appropriate error / loss function, or any appropriate training termination criterion may be used during such training.
[0061] The disclosure herein primarily describes the neural sentence encoder 104 as being trained unsupervised using a bottleneck pipeline, but this is merely a non-limiting example for the sake of explanation and illustration. In various embodiments, any other suitable training paradigm, such as supervised training, semi-supervised training, or reinforcement learning, may be used to train the neural sentence encoder 104, any of which may be associative or unassociative. In fact, in some cases, the neural sentence encoder 104 may be trained on pairs of sentences using a contrast error, loss, or objective function. Training based on such pairs may be implemented instead of, or in addition to, any of the training techniques described above.
[0062] Referring again to Figure 1, sentence 106 may exist. In various forms, sentence 106 may be any suitable grammatically consistent plain text or sequence of words in natural language. In various cases, sentence 106 may contain or consist of any suitable number of words. In various cases, sentence 106 may be a declarative semantic type. As an unrestricted example, sentence 106 may describe or explain a factoid (e.g., any suitable fragment of factual trivia about any suitable subject or topic).
[0063] Natural language queries 108 can exist in various forms. In various cases, a natural language query 108 can be any appropriate grammatically consistent plain text or sequence of natural language words. In various cases, a natural language query 108 can contain or consist of any appropriate number of words. In some situations, a natural language query 108 can be an interrogative semantic type. As an unrestricted example, a natural language query 108 can be a question. In other situations, a natural language query 108 can be a declarative semantic type. As an unrestricted example, a natural language query 108 can explain or describe a factoid.
[0064] In any case, it may be desirable to determine whether sentence 106 is substantially or semantically related to the natural language query 108. As an unrestricted example, if the natural language query 108 is a question, it may be desirable to determine whether sentence 106 answers that question. As another unrestricted example, if the natural language query 108 is a description of some factid, it may be desirable to determine whether sentence 106 either confirms (e.g., is consistent with) or negates (e.g., is inconsistent with) the natural language query 108; or whether it does neither confirm nor negate the natural language query 108. In any of these situations, the asymmetric relevance system 102 can facilitate such determinations as described herein.
[0065] In various embodiments, the asymmetric relationship system 102 may include a processor 110 (e.g., a computer processing unit, a microprocessor) and a non-temporary computer-readable memory 112 operably connected to or coupled to the processor 110. The memory 112 can store computer-executable instructions, which, when executed by the processor 110, can cause the processor 110 or other components of the asymmetric relationship system 102 (e.g., an access component 114, an execution component 116) to perform one or more operations. In various embodiments, the memory 112 can store computer-executable components (e.g., an access component 114, an execution component 116), and the processor 110 can execute these computer-executable components.
[0066] In various embodiments, the asymmetric relation system 102 may include an access component 114. In various embodiments, the access component 114 can electronically access the neural sentence encoder 104 in any suitable manner. That is, the access component 114 can establish any suitable type of electronic communication with the neural sentence encoder 104, thereby enabling the asymmetric relation system 102 to electronically execute, electronically modify, or otherwise electronically control the neural sentence encoder 104. Furthermore, in various cases, the access component 114 can electronically receive, retrieve, acquire, import, or otherwise access sentences 106 or natural language queries 108 from any suitable data structure or any suitable computing device. Thus, the access component 114 can be considered a proxy or conduit through which any other component of the asymmetric relation system 102 can electronically interact with or otherwise manipulate the neural sentence encoder 104, sentences 106, or natural language queries 108.
[0067] In various embodiments, the asymmetric relevance system 102 may include an execution component 116. In various embodiments, the execution component 116 can leverage the neural sentence encoder 104, as described herein, to asymmetrically generate embeddings for sentences 106 and natural language queries 108. In various cases, the execution component 116 can use these asymmetrically generated embeddings, as described herein, to determine whether sentence 106 is semantically or substantially relevant to a natural language query 108.
[0068] It should be noted that in various cases, the access component 114 and the execution component 116 may be collectively considered as one or more software components 113 of the asymmetric relationship system 102. In various embodiments, it should be understood that, for the sake of ease of explanation and illustration, one or more software components 113 are described primarily herein as comprising two components (e.g., the access component 114 and the execution component 116). However, one or more software components 113 are not limited to being implemented strictly as such two components in every embodiment. In fact, in some embodiments, the functions of such two components described herein may be combined in any suitable manner so as to be implemented in or by fewer than two components (for example, in some cases, a single component may perform all the functions described herein with respect to the access component 114 and the execution component 116). In other embodiments, the functions of such two components described herein may instead be distributed, separated, divided, or fragmented in any suitable manner so as to be implemented in or by two or more components (for example, two or more components may facilitate the functions that can be performed by the access component 114; two or more components may facilitate the functions that can be performed by the execution component 116).
[0069] Figure 3 shows a block diagram of an exemplary and non-limiting system 300, including sentence embeddings and query embeddings, which may facilitate asymmetric relationships through the execution of a neural sentence encoder in fragment units, according to one or more embodiments described herein. As shown, system 300 may in some cases include the same components as system 100, and may further include sentence embeddings 302 and query embeddings 304.
[0070] In various embodiments, the execution component 116 may asymmetrically execute the neural sentence encoder 104 with respect to the sentence 106 and the natural language query 108 to generate the sentence embedding 302 and the query embedding 304. Various non-limiting embodiments are described with reference to Figures 4-7.
[0071] Figures 4–7 show exemplary and non-limiting block diagrams illustrating how the sentence embedding 302 and query embedding 304 may be generated asymmetrically according to one or more embodiments described herein.
[0072] First, we consider Figure 4, which shows a block diagram 400. In various embodiments, the execution component 116 can electronically execute the neural sentence encoder 104 for a natural language query 108. In various embodiments, such execution can produce a query embedding 304. More specifically, the execution component 116 can feed or route the natural language query 108 to the input layer of the neural sentence encoder 104. In various cases, the natural language query 108 can therefore complete forward propagation through one or more hidden layers of the neural sentence encoder 104. In various cases, the output layer of the neural sentence encoder 104 can, accordingly, compute or compute the query embedding 304 based on any hidden activation maps or hidden feature maps generated by one or more hidden layers during its forward propagation. In various embodiments, the query embedding 304 may be a latent vector representation of any semantic or substantial content conveyed by the natural language query 108. As a non-restrictive example, query embedding 304 can be a d-element vector for any suitable positive integer d > 1.
[0073] Next, we consider Figure 5, which shows a block diagram 500. In various embodiments, the execution component 116 may refrain from performing the neural sentence encoder 104 on the sentence 106. Instead, the execution component 116 may electronically decompose, electronically split, or electronically subdivide the sentence 106 into a plurality of sentence fragments 502. In various embodiments, the execution component 116 may achieve such decomposition, splitting, or subdivision through any suitable text analysis technique. Some non-exclusive examples of text analysis techniques may include named entity recognition analysis; tokenization analysis; syntactic analysis; dependency analysis; or phrase structure analysis.
[0074] In various cases, multiple sentence fragments 502 may contain n fragments: sentence fragment 502(1) to sentence fragment 502(n) for any suitable positive integer n>1. In various cases, sentence 106 can be considered as a sequence of words, and each of the multiple sentence fragments 502 may be a continuous, uninterrupted, or unaltered subsequence of words from sentence 106. That is, for any given one of the multiple sentence fragments 502, that given sentence fragment may contain one or more words present in sentence 106, such one or more words are ordered with respect to each other in that given sentence fragment in the same way they are ordered with respect to each other in sentence 106, and such one or more words are not separated from each other in sentence 106 by any word that is not a member of that given sentence fragment. In any case, the union of the multiple sentence fragments 502 may be equal to sentence 106, that is, each word in sentence 106 may be assigned to or belong to at least one of the multiple sentence fragments 502. In some cases, the multiple sentence fragments 502 may be disjoint or non-overlapping with each other. That is, each word in sentence 106 may be assigned to exactly one of the multiple sentence fragments 502, or belong to one another. However, in other cases, the multiple sentence fragments 502 may not be relatively prime or may overlap with one another. That is, each word in sentence 106 may be assigned to one or more of the multiple sentence fragments 502, or belong to one another (for example, some words may belong to multiple fragments). Note that different sets of multiple sentence fragments 502 may consist of the same or different numbers of words. In addition, note that different sets of multiple sentence fragments 502 may be considered to represent or be limited to different physical locations or locations within sentence 106. That is, each of the multiple sentence fragments 502 may be considered to be a separate group of adjacent words within sentence 106.
[0075] As a non-restrictive example, suppose sentence 106 is "Over the weekend, Jane Smith, a noted veterinarian in the tri-county area, tirelessly worked to save a black-tailed doe and newborn fawn." In such a case, "over the weekend" could be considered the first fragment of sentence 106, "jane smith, a noted veterinarian in the tri-county area" could be considered the second fragment of sentence 106, "tirelessly worked to save" could be considered the third fragment of sentence 106, and "a black-tailed doe and newborn fawn" could be considered the fourth fragment of sentence 106.
[0076] Here, we consider Figure 6, which shows a block diagram 600. In various embodiments, the execution component 116 can electronically execute the neural sentence encoder 104 for each of the multiple sentence fragments 502, and such execution can collectively produce multiple fragment embeddings 602.
[0077] As a non-limiting example, the execution component 116 can electronically execute the neural sentence encoder 104 on a sentence fragment 502(1). In various embodiments, such execution may result in a fragment embedding 602(1). More specifically, the execution component 116 may feed or route the sentence fragment 502(1) to the input layer of the neural sentence encoder 104. In various cases, the sentence fragment 502(1) may thus complete forward propagation through one or more hidden layers of the neural sentence encoder 104. In various cases, the output layer of the neural sentence encoder 104 may, accordingly, compute or compute the fragment embedding 602(1) based on any hidden activation map or hidden feature map generated by one or more hidden layers during its forward propagation. In various embodiments, the fragment embedding 602(1) may be a latent vector representation of any semantic or substantial content propagated by the sentence fragment 502(1). Note that fragment embedding 602(1) is not affected by or influenced by any words of sentence 106 that are not present in sentence fragment 502(1). In addition, note that fragment embedding 602(1) may have the same form, size, or dimensions as query embedding 304. Therefore, fragment embedding 602(1), like query embedding 304, can be a d-element vector.
[0078] As another non-limiting example, the execution component 116 can electronically execute the neural sentence encoder 104 on a sentence fragment 502(n). In various embodiments, such execution may result in a fragment embedding 602(n). More specifically, the execution component 116 may feed or route the sentence fragment 502(n) to the input layer of the neural sentence encoder 104. In various cases, the sentence fragment 502(n) may thus complete forward propagation through one or more hidden layers of the neural sentence encoder 104. In various cases, the output layer of the neural sentence encoder 104 may, accordingly, compute or compute the fragment embedding 602(n) based on any hidden activation map or hidden feature map generated by one or more hidden layers during its forward propagation. In various embodiments, the fragment embedding 602(n) may be a latent vector representation of any semantic or substantial content propagated by the sentence fragment 502(n). Note that fragment embedding 602(n) is not affected by or influenced by any words in sentence 106 that are not included in sentence fragment 502(n). In addition, note that fragment embedding 602(n) may have the same form, size, or dimensions as query embedding 304. Therefore, fragment embedding 602(n) can be a d-element vector, just like query embedding 304.
[0079] In various cases, fragment embeddings 602(1) through 602(n) can be considered collectively to form multiple fragment embeddings 602.
[0080] Next, we will consider Figure 7, which shows block diagrams 700 and 701. In various embodiments, the execution component 116 can electronically generate a sentence embedding 302 by fusing multiple fragment embeddings 602.
[0081] In some situations, as shown with respect to block diagram 700, such fusion may involve concatenating multiple fragment embeddings 602 together. In such cases, the sentence embedding 302 may be n times the length of the query embedding 304. That is, the sentence embedding 302 may be an nd element vector.
[0082] However, in other situations, as shown with respect to block diagram 701, such fusion may instead involve projecting each of the multiple fragment embeddings 602 into a lower-dimensional space. More specifically, the execution component 116 may, in various embodiments, apply any suitable dimensionality reduction technique to each of the multiple fragment embeddings 602, thereby resulting in multiple compressed fragment embeddings 702. As some non-restrictive examples, such dimensionality reduction techniques may be principal component analysis (PCA); singular value decomposition (SVD); linear discriminant analysis (LDA); t-distributed stochastic neighbor embedding (t-SNE); or uniform manifold approximation and projection (UMAP).
[0083] In any case, each of the multiple compressed fragment embeddings 702 may be a dimensionally smaller or shorter version of each of the multiple fragment embeddings 602. As an unrestricted example, a compressed fragment embedding 702(1) may be formed by applying any suitable dimensionality reduction technique to a fragment embedding 602(1). Thus, a compressed fragment embedding 702(1) may represent, convey, or otherwise represent numerically the same substantial or semantic content as a fragment embedding 602(1), but it may do so through c numerical elements for any suitable positive integer c>d (e.g., it may be a c-element vector rather than a d-element vector). As another non-restrictive example, a compressed fragment embedding 702(n) can be formed by applying any suitable dimensionality reduction technique to a fragment embedding 602(n). Thus, a compressed fragment embedding 702(n) can represent, convey, or otherwise represent numerically the same substantial or semantic content as a fragment embedding 602(n), but can do so through c numerical elements.
[0084] It should be noted that compressed fragment embeddings 702(1) through 702(n) can be collectively considered as multiple compressed fragment embeddings 702.
[0085] In some cases, c = d / n. In such situations, the execution component 116 may concatenate multiple compressed fragment embeddings 702 together to form a statement embedding 302. Note that in such situations, the statement embedding 302 may therefore have the same form, size, or dimensions as the query embedding 304. In other words, when the statement embedding 302 is formed by concatenating multiple compressed fragment embeddings 702, the statement embedding 302 may be a d-element vector.
[0086] It should be noted that sentence embedding 302 can be considered asymmetrically generated compared to query embedding 304. In fact, query embedding 304 can be obtained by running the neural sentence encoder 104 over the entire natural language query 108 (for example, the natural language query 108 is not subdivided or broken down into fragments over which the neural sentence encoder 104 is run independently). In contrast, sentence embedding 302 can instead be obtained by subdividing sentence 106 into multiple sentence fragments 502 and running the neural sentence encoder 104 independently or separately over each of the multiple sentence fragments 502. Thus, while query embedding 304 can be considered to represent the semantic or substantial content of the natural language query 108, sentence embedding 302 can instead be considered to represent the semantic or substantial content of the individual parts of the multiple sentence fragments 502 at a finer granularity. In other words, sentence embeddings 302 may not be affected by semantic or lexical relationships that may exist between words of sentence 106 that are far apart from each other (e.g., between words of sentence 106 that are not in the same fragment). Instead, if the neural sentence encoder 104 were run on the entire sentence 106 (e.g., without first fragmenting sentence 106), sentence embeddings 302 would instead be significantly affected by semantic or lexical relationships that may exist between words of sentence 106 that are far apart from each other.
[0087] Figure 8 shows exemplary and non-limiting block diagrams 800 and 801 illustrating how similarity can be calculated between the sentence embedding 302 and the query embedding 304 according to one or more embodiments described herein.
[0088] In various embodiments, the execution component 116 may electronically calculate a similarity score 802 based on the sentence embeddings 302 and the query embeddings 304.
[0089] In situations where sentence embedding 302 is already identical in form, size, or dimensions to query embedding 304, the similarity score 802 may be equal to, or otherwise based on, the cosine similarity (or the reciprocal or complement of the Euclidean distance) between sentence embedding 302 and query embedding 304, as shown by block diagram 801. Note that block diagram 801 may be considered to correspond to or be a continuation of block diagram 701.
[0090] However, in situations where the sentence embedding 302 is n times longer than the query embedding 304, the execution component 116 may generate n copies of the query embedding 304 and concatenate those n copies together, as shown in block diagram 800. In such a case, the similarity score 802 may be equal to, or otherwise based on, the cosine similarity (or the reciprocal or complement of the Euclidean distance) between the sentence embedding 302 and its n-th concatenation of the query embedding 304. Note that block diagram 800 may be considered to correspond to or a continuation of block diagram 700.
[0091] In either case, the similarity score 802 may be a scalar whose magnitude can be considered to represent how semantically the natural language query 108 is similar to at least one fragment within the sentence 106, rather than how semantically the natural language query 108 is similar to the sentence 106. In response that the similarity score 802 meets any appropriate threshold, the execution component 116 may conclude that the sentence 106 contains something similar to the natural language query 108, and therefore conclude that the sentence 106 is related to the natural language query 108 (e.g., answer; confirm or invalidate). On the other hand, in response that the similarity score 802 does not meet any appropriate threshold, the execution component 116 may conclude that the sentence 106 contains nothing similar to the natural language query 108, and therefore conclude that the sentence 106 is irrelevant to the natural language query 108 (e.g., do not answer; do not confirm or invalidate).
[0092] Figure 9 shows illustrative and non-limiting block diagrams 900 and 901 illustrating how subsentence localization of query answers may be facilitated according to one or more embodiments described herein.
[0093] In various embodiments, in response to determining that a statement 106 is related to a natural language query 108, the execution component 116 can electronically locate which particular part of the statement 106 is most relevant to the natural language query 108. That is, the execution component 116 can identify which of the multiple statement fragments 502 is most similar to the natural language query 108.
[0094] As shown in block diagram 900, in situations where dimensionality reduction is not applied to multiple fragment embeddings 602, the execution component 116 may generate multiple fragment similarity scores 902 based on comparing the query embedding 304 with each of the multiple fragment embeddings 602. As an unrestricted example, fragment similarity score 902(1) may be equal to, or otherwise based on, the cosine similarity (or the reciprocal or complement of the Euclidean distance) between the query embedding 304 and fragment embedding 602(1). As another unrestricted example, fragment similarity score 902(n) may be equal to, or otherwise based on, the cosine similarity (or the reciprocal or complement of the Euclidean distance) between the query embedding 304 and fragment embedding 602(n). Note that fragment similarity scores 902(1) through 902(n) may be considered collectively to form multiple fragment similarity scores 902.
[0095] As shown in block diagram 901, in situations where dimensionality reduction is applied to multiple fragment embeddings 602, the execution component 116 may generate multiple fragment similarity scores 902 as described above, or the execution component 116 may instead generate multiple fragment similarity scores 902 based on comparing the compressed query embedding 904 with each of the multiple compressed fragment embeddings 702. In various embodiments, the compressed query embedding 904 may be obtained by applying any suitable dimensionality reduction technique to the query embedding 304, resulting in the compressed query embedding 904 conveying or representing the same substantial or semantic content as the query embedding 304, but doing so in dimension c instead of dimension d. In such cases, the fragment similarity score 902(1) may be equal to, or otherwise based on, the cosine similarity (or the reciprocal or complement of the Euclidean distance) between the compressed query embedding 904 and the compressed fragment embedding 702(1). Similarly, the fragment similarity score 902(n) may be equal to, or otherwise based on, the cosine similarity (or the reciprocal or complement of the Euclidean distance) between the compressed query embedding 904 and the compressed fragment embedding 702(n).
[0096] In any case, each of the multiple fragment similarity scores 902 can be considered to indicate how semantically or substantially similar the natural language query 108 is to each of the multiple sentence fragments 502. As an unrestricted example, the fragment similarity score 902(1) may be a scalar whose magnitude indicates how similar the natural language query 108 is to sentence fragment 502(1). As another unrestricted example, the fragment similarity score 902(n) may be a scalar whose magnitude indicates how similar the natural language query 108 is to sentence fragment 502(n). In various embodiments, among the multiple sentence fragments 502, the one with the highest, largest, or otherwise exceeding any appropriate threshold may be considered to be the portion of sentence 106 most similar to the natural language query 108.
[0097] In various embodiments, the execution component 116 may electronically transmit any of the following: the statement embedding 302, the query embedding 304, the multiple fragment embeddings 602, the multiple compressed fragment embeddings 702, the compressed query embedding 904, the similarity score 802, or the multiple fragment similarity scores 902, to any suitable computing device (for example, for downstream or subsequent processing). In various embodiments, the execution component 116 may electronically display any of the following: the statement embedding 302, the query embedding 304, the multiple fragment embeddings 602, the multiple compressed fragment embeddings 702, the compressed query embedding 904, the similarity score 802, or the multiple fragment similarity scores 902, on any suitable computer screen or electronic display (for example, to notify a user or operator).
[0098] In various embodiments, the asymmetric relational system 102 can generate a statement embedding 302 in response to the receipt of an electronic instruction or command (e.g., by an access component 114) for executing the neural statement encoder 104 on a statement 106 (e.g., via an execution component 116). In fact, any user or computing client issuing such an instruction or command may not be aware of the advantages of the asymmetric execution of the neural statement encoder 104 as described herein. Thus, such a user or computing client may request the normal or standard execution of the neural statement encoder 104 on the statement 106 (e.g., request that the neural statement encoder 104 be executed on the entire statement 106 without first fragmenting the statement 106). The execution component 116 can respond to such a request for standard or normal execution not by executing the neural sentence encoder 104 on the sentence 106 in a standard or normal manner, but instead by executing the neural sentence encoder 104 asymmetrically with respect to the sentence 106 as described herein (for example, by fragmenting the sentence 106, independently executing the neural sentence encoder 104 on those fragments, and then fusing the resulting fragment embeddings).
[0099] The various embodiments described herein primarily involve fusing together multiple fragment embeddings 602 (or multiple compressed fragment embeddings 702) via concatenation, but these are merely non-limiting examples for the sake of clarity and illustration. In other embodiments, such fusing may be achieved via any other suitable method, such as vector addition, unweighted vector averaging, or weighted vector averaging.
[0100] Figure 10 shows a flowchart of an exemplary and non-limiting computer implementation method 1000 that can facilitate asymmetric associations through the execution of a neural sentence encoder in fragment units, according to one or more embodiments described herein. In various cases, an asymmetric association system 102 can facilitate the computer implementation method 1000.
[0101] In various embodiments, operation 1002 may include accessing a device operably coupled to a processor (e.g., 110) (by, for example, via 114), a statement (e.g., 106), and a neural statement encoder (e.g., 104).
[0102] In various embodiments, operation 1004 may include the device generating a first embedding (e.g., 302) for a sentence (e.g., via 116) based on independently running a neural sentence encoder on multiple sentence fragments (e.g., 502) of the sentence.
[0103] Although not explicitly shown in Figure 10, independently running a neural sentence encoder on multiple sentence fragments may result in multiple fragment embeddings (e.g., 602), and the first embedding for a sentence may be based on fusing multiple fragment embeddings. In some cases, such fusing may involve the device (e.g., via 116) projecting each of the multiple fragment embeddings into a dimensionally reduced form, thereby resulting in multiple compressed fragment embeddings (e.g., 702); and the device (e.g., via 116) concatenating the multiple compressed fragment embeddings together. In other cases, such fusing may involve the device (e.g., via 116) concatenating or adding the multiple fragment embeddings together.
[0104] Although not explicitly shown in Figure 10, the computer implementation method 1000 may include: accessing a natural language query (e.g., 108) by the device (e.g., via 114); generating a second embedding (e.g., 304) for the natural language query by the device (e.g., via 116) based on running a neural sentence encoder on the natural language query, and not based on independently running a neural sentence encoder on multiple fragments of the natural language query; calculating a similarity score (e.g., 802) between the first and second embeddings by the device (e.g., via 116); and determining by the device (e.g., via 116) that the sentence semantically answers the natural language query in response to the similarity score exceeding a threshold.
[0105] Although not explicitly shown in Figure 10, the computer implementation method 1000 may include, in response to a determination that a sentence semantically answers a natural language query, the device (e.g., via 116) calculating a second similarity score between the second embedding and each of the multiple fragment embeddings, thereby generating a plurality of second similarity scores (e.g., 902); and, based on identifying which of the plurality of second similarity scores exceeds a threshold, the device (e.g., via 116) locating the most semantically relevant of the multiple sentence fragments.
[0106] The various embodiments described herein involve the neural sentence encoder 104 being performed on a grammatical sentence or a fragment thereof, but these are merely non-limiting examples for the sake of explanation and illustration. The various embodiments described herein can be readily extrapolated for use with any other suitable type of text passage (e.g., not limited to grammatical sentences). In fact, the neural sentence encoder can take any string, chain, or passage of text as input and produce as output a fixed-dimensional embedding that numerically represents the substantial or semantic content of the string, chain, or passage of text. Thus, in various embodiments, sentence 106 can be extended or generalized to any suitable passage of text consisting of any suitable number of words. As some non-limiting examples, sentence 106 can be replaced or substituted with a paragraph of text; two or more paragraphs; an entire chapter of a paragraph; multiple chapters; or even an entire web page or book of text. In such cases, multiple sentence fragments 502 may be extended or generalized to multiple passage fragments (also referred to as multiple text subpassages), where each passage fragment may be any suitable consecutive or uninterrupted exact subsequence of words from a text passage. Similarly, in various ways, a natural language query 108 may be extended or generalized to any suitable text passage (e.g., any suitable number of paragraphs, chapters, web pages, or books). Regardless of what particular grammatical form such text passages take, the neural sentence encoder 104 may be performed asymmetrically on such text passages to determine more accurately or reliably whether they are semantically related to one another (e.g., the neural sentence encoder 104 may be performed on the entire text passage that replaces the natural language query 108; or the neural sentence encoder 104 may instead be performed on the parsed fragments or subpassages of any text passage that replaces the sentence 106).
[0107] Accordingly, various embodiments described herein may involve accessing a first text passage (e.g., 106) and a neural sentence encoder (e.g., 104) by a device (e.g., 114) operably coupled to a processor (e.g., 110); and generating a first embedding (e.g., 302) for the first text passage based on independently executing the neural sentence encoder on multiple fragments (e.g., 502) of the first text passage by the device (e.g., via 116). In various embodiments, independently executing the neural sentence encoder on multiple fragments may result in multiple fragment embeddings (e.g., 602), and the first embedding for the first text passage may be based on fusing the multiple fragment embeddings. In some cases, such fusing may include concatenating or adding the multiple fragment embeddings to each other. In other cases, the fusion may involve projecting each of the multiple fragment embeddings into a dimensionally reduced form, thereby producing multiple compressed fragment embeddings (e.g., 702), and concatenating the multiple compressed fragment embeddings together. In various cases, such embodiments may further involve the device accessing a second text passage (e.g., 108) (e.g., via 114); generating a second embedding (e.g., 304) for the second text passage (e.g., via 116) based on running a neural sentence encoder on the second text passage, and not based on independently running a neural sentence encoder on multiple fragments of the second text passage; the device calculating a similarity score (e.g., 802) between the first and second embeddings (e.g., via 116); and, in response to the similarity score exceeding a threshold, the device (e.g., 116) determining that the first text passage is semantically related to the second text passage.In various embodiments, such embodiments may further include, in response to determining that the first text passage is semantically related to the second text passage, the device (e.g., via 116) calculating a second similarity score between the second embedding and each of the multiple fragment embeddings, thereby generating a plurality of second similarity scores (e.g., 902); and, in response to determining that the device (e.g., via 116) is semantically related to the second similarity score, locating the most semantically relevant fragment among the multiple fragments based on identifying which of the multiple second similarity scores exceeds a threshold. In various cases, the neural sentence encoder may be executed independently for the multiple fragments in response to the receipt of an electronic command requesting the execution of the neural sentence encoder on the first text passage.
[0108] Figure 11 and the following discussion are intended to provide a brief and general description of a suitable computing environment 1100 in which one or more embodiments described herein may be implemented. For example, various aspects of this disclosure are described by descriptive text, flowcharts, block diagrams of computer systems, or block diagrams of machine logic included in computer program product (CPP) embodiments. In any flowchart, depending on the technology involved, operations may be performed in a different order than that shown in a given flowchart. For example, again depending on the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that overlaps at least partially in time.
[0109] Computer program product embodiments ("CPP embodiments" or "CPP") are terms used in this disclosure to describe any set of one or more storage media ("mediums") that are collectively included in a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. Without limitation, computer-readable storage media may be electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any suitable combination thereof. Some known types of storage devices, including these media, include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices (e.g., pits / lands formed on the main surface of punch cards or disks), or any appropriate combination of the above. Where the terms are used in this disclosure, computer-readable storage media should not be interpreted as storage of transient signals in themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, light pulses passing through optical fiber cables, electrical signals transmitted through wires, or other transmission media. As will be understood by those skilled in the art, data is typically moved at several occasional points during the normal operation of the storage device, such as during access, defragmentation, or garbage collection, but this is not transient while the data is stored, and therefore the storage device should not be considered transient.
[0110] The computing environment 1100 includes an example of an environment for executing at least a portion of the computer code involved in performing the inventive method, such as the asymmetric related fragment unit embedded code 1180. In addition to block 1180, the computing environment 1100 includes, for example, a computer 1101, a wide area network (WAN) 1102, an end-user device (EUD) 1103, a remote server 1104, a public cloud 1105, and a private cloud 1106. In this embodiment, the computer 1101 includes a processor set 1110 (including processing circuits 1120 and a cache 1121), a communication fabric 1111, volatile memory 1112, persistent storage 1113 (including the operating system 1122 and block 1180 as identified above), a peripheral device set 1114 (including a user interface (UI), a device set 1123, storage 1124, and an Internet of Things (IoT) sensor set 1125), and a network module 1115. The remote server 1104 includes the remote database 1130. The public cloud 1105 includes the gateway 1140, the cloud orchestration module 1141, the host physical machine set 1142, the virtual machine set 1143, and the container set 1144.
[0111] Computer 1101 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device currently known or to be developed in the future that is capable of running programs, accessing networks, or querying databases such as remote database 1130. As is well understood in the technical field of computer technology, and depending on the technology, the execution of a computer implementation method may be distributed among multiple computers or across multiple locations. On the other hand, in this presentation of the computing environment 1100, in order to keep the presentation as simple as possible, the detailed discussion focuses on a single computer, specifically computer 1101. Computer 1101 may be located in the cloud, although it is not shown in the cloud in Figure 11. On the other hand, computer 1101 is not required to be in the cloud, except to any extent that can be definitively shown.
[0112] The processor set 1110 includes one or more computer processors of any type currently known or to be developed in the future. The processing circuitry 1120 may be distributed across multiple packages, for example, multiple coordinated integrated circuit chips. The processing circuitry 1120 may implement multiple processor threads or multiple processor cores. The cache 1121 is memory located within the processor chip package and is typically used for data or code that should be available for rapid access by threads or cores operating on the processor set 1110. The cache memory is typically organized into multiple levels depending on its relative proximity to the processing circuitry. Alternatively, some or all of the processor set's cache may be located "off-chip". In some computing environments, the processor set 1110 may be designed to operate with qubits and perform quantum computing.
[0113] Computer-readable program instructions are typically loaded into computer 1101, causing the processor set 1110 of computer 1101 to execute a series of operational steps, thereby realizing a computer implementation method, and as a result, the instructions executed in this manner instantiate the methods (collectively referred to as “inventive methods”) that are identified in the flowcharts or descriptive descriptions of computer implementation methods contained herein. These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 1121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 1110 to control and direct the execution of the inventive methods. In the computing environment 1100, at least some of the instructions for executing the inventive methods may be stored in block 1180 in persistent storage 1113.
[0114] The communication fabric 1111 is a signal conduction path that enables various components of the computer 1101 to communicate with one another. Typically, this fabric consists of switches and conductive paths such as switches and conductive paths that constitute buses, bridges, physical input / output ports and similar components. Other types of signal communication paths may be used, such as optical fiber communication paths or wireless communication paths.
[0115] Volatile memory 1112 is any type of volatile memory currently known or to be developed in the future. Examples include dynamic random-access memory (RAM) or static RAM. Typically, volatile memory is characterized by random access, but this is not required unless explicitly stated. In computer 1101, volatile memory 1112 is located in a single package and is internal to computer 1101, but alternatively or additionally, volatile memory may be distributed across multiple packages or located externally with respect to computer 1101.
[0116] Persistent storage 1113 is any form of non-volatile storage for a computer, currently known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is supplied to the computer 1101 and / or directly to the persistent storage 1113. Persistent storage 1113 may be read-only memory (ROM), but typically, at least a portion of the persistent storage allows for writing, deleting, and rewriting of data. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. The operating system 1122 can take multiple forms, such as various known proprietary operating systems or open-source portable operating system interface operating systems employing a kernel. The code contained in block 1180 typically includes at least a portion of computer code involved in performing the inventive method.
[0117] The peripheral device set 1114 includes a set of peripheral devices for the computer 1101. Data communication connections between the peripheral devices and other components of the computer 1101 can be implemented in various ways, including, for example, Bluetooth® connections, near-field communication (NFC) connections, connections made by cables (e.g., Universal Serial Bus (USB) type cables), insertable connections (e.g., Secure Digital (SD) cards), connections made via local area communication networks, and even connections made via wide area networks such as the Internet. In various embodiments, the UI device set 1123 may include components such as a display screen, speaker, microphone, wearable devices (e.g., goggles and smartwatches), keyboard, mouse, printer, touchpad, game controller, and haptic devices. Storage 1124 is external storage such as an external hard drive, or insertable storage such as an SD card. Storage 1124 may be persistent or volatile. In some embodiments, storage 1124 may take the form of a quantum computing memory device for storing data in the form of qubits. In embodiments where computer 1101 is required to have a large amount of storage (for example, when computer 1101 locally stores and manages a large database), this storage may be provided by peripheral storage devices designed to store large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 1125 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another may be a motion detector.
[0118] The network module 1115 is a collection of computer software, hardware, and firmware that enables computer 1101 to communicate with other computers via the WAN 1102. The network module 1115 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing or depackaging data for communication network transmission, or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of the network module 1115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control and forwarding functions of the network module 1115 are performed on physically separate devices, resulting in the control function managing multiple different network hardware devices. Computer-readable program instructions for performing the inventive method can typically be downloaded from an external computer or external storage device to computer 1101 over the network via a network adapter card or network interface included in the network module 1115.
[0119] WAN1102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any currently known or future-developed technology for transmitting computer data. In some embodiments, a WAN may be replaced or complemented by a local area network (LAN), such as a Wi-Fi network, designed to transmit data between devices located in a local area. A WAN or LAN typically includes computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.
[0120] The end-user device (EUD) 1103 is any computer system used and controlled by an end-user (e.g., a customer of the company operating computer 1101) and can take any of the forms discussed above in relation to computer 1101. Typically, EUD 1103 receives useful and valuable data from the operation of computer 1101. For example, in a hypothetical case where computer 1101 is designed to provide recommendations to an end-user, these recommendations would typically be communicated from computer 1101's network module 1115 to EUD 1103 via WAN 1102. In this way, EUD 1103 can display or otherwise present the recommendations to the end-user. In some embodiments, EUD 1103 may be a client device such as a thin client, heavy client, mainframe computer, or desktop computer.
[0121] The remote server 1104 is any computer system that provides at least some data or functionality to computer 1101. The remote server 1104 may be controlled and used by the same entity that operates computer 1101. The remote server 1104 represents a machine that collects and stores useful and valuable data for use by other computers, such as computer 1101. For example, in a hypothetical case where computer 1101 is designed and programmed to provide recommendations based on historical data, such historical data may be provided to computer 1101 from the remote database 1130 of the remote server 1104.
[0122] The public cloud 1105 is any computer system available for use by multiple entities, providing on-demand availability of computer system resources or other computing capabilities, particularly data storage (cloud storage) and computing power, without direct active management at scale. Direct and active management of the computing resources of the public cloud 1105 is performed by the computer hardware or software of the cloud orchestration module 1141. The computing resources provided by the public cloud 1105 are typically implemented by virtual computing environments running on various computers that make up the host physical machine set 1142, which is a universe of physical computers that are in or available to the public cloud 1105. The virtual computing environment (VCE) typically takes the form of virtual machines from the virtual machine set 1143 or containers from the container set 1144. It is understood that these VCEs may be stored as images and may be transferred between various physical machine hosts, either as images or after the VCEs have been instantiated. The cloud orchestration module 1141 manages image transfer and storage, deploys new VCE instances, and manages active instances of VCE deployments. The gateway 1140 is a collection of computer software, hardware, and firmware that enables the public cloud 1105 to communicate over the WAN 1102.
[0123] Some further explanation of virtualized computing environments (VCEs) is provided here. A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from that image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows for the existence of multiple isolated user-space instances called containers. These isolated user-space instances typically behave like actual computers from the perspective of the programs running within them. Computer programs running on a normal operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running within a container can only use the contents of the container and the devices allocated to the container; this feature is known as containerization.
[0124] The private cloud 1106 is similar to the public cloud 1105, except that its computing resources are available for use by a single enterprise only. While the private cloud 1106 is illustrated as communicating with the WAN 1102, in other embodiments, the private cloud may be completely isolated from the internet and accessible only via a local / private network. A hybrid cloud is a configuration of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. Each of the multiple clouds remains a separate and independent entity, but the larger hybrid cloud architecture is coupled together by standardized or proprietary technologies that enable orchestration, management, or data / application portability between the multiple configuration clouds. In this embodiment, both the public cloud 1105 and the private cloud 1106 are part of a larger hybrid cloud.
[0125] The embodiments described herein, at any possible level of integration of technical detail, cover one or more systems, methods, apparatus, or computer program products. A computer program product may include a computer-readable storage medium (or medium) on which computer-readable program instructions are written to cause a processor to execute aspects of one or more embodiments described herein. The computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a superconducting storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, or mechanically encoded devices such as punch cards or raised structures in grooves on which instructions are recorded, or any suitable combination thereof. Computer-readable storage media as used herein should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through optical fiber cables), or electrical signals transmitted through wires.
[0126] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device, via a network such as the Internet, a local area network, a wide area network, or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers such instructions for storage on a computer-readable storage medium within each computing / processing device. The computer-readable program instructions for performing the operation of one or more embodiments described herein may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk®, C++, or similar, or procedural programming languages such as the "C" programming language or similar programming languages. Computer-readable program instructions may be fully executed on a computer, partially executed on a computer, executed as a standalone software package, partially executed on a computer or on a remote computer, or fully executed on a remote computer or server. In the latter scenario, the remote computer may be connected to the computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (for example, via the Internet using an Internet service provider).In one or more embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by personalizing the electronic circuit using state information of computer-readable program instructions in order to perform an aspect of one or more embodiments described herein.
[0127] Aspects of one or more embodiments described herein are described with reference to flowcharts or block diagrams of methods, apparatus (systems), and computer program products according to one or more embodiments described herein. It will be understood that each block in a flowchart or block diagram, and combinations of blocks in a flowchart or block diagram, can be implemented by computer-readable program instructions. These computer-readable program instructions may be provided to a processor of a general-purpose computer, a dedicated computer, or other programmable data processing device to generate a machine, thereby enabling instructions executed via the processor of the computer or other programmable data processing device to generate means for implementing the functions / operations specified in the blocks or groups of blocks of the flowchart or block diagram. These computer-readable program instructions may be stored in a computer-readable storage medium that can instruct a computer, a programmable data processing device, or other device to function in a particular manner, thereby enabling the computer-readable storage medium in which the instructions are stored to constitute a product containing instructions that can implement the functions / operations specified in the blocks or groups of blocks of the flowchart or block diagram. Computer-readable program instructions can also be loaded into a computer, other programmable data processing device, or other device to perform a series of actions on the computer, other programmable device, or other device, thereby generating a computer implementation process in which the instructions executed on the computer, other programmable device, or other device implement the functions / actions specified in blocks or groups of blocks in a flowchart or block diagram.
[0128] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, or operation of a possible implementation of a system, computer-implementable method, or computer program product according to one or more embodiments described herein. In this regard, each block in a flowchart or block diagram may represent a module, segment, or part of an instruction containing one or more executable instructions for implementing a specified logical function. In one or more alternative implementations, the functions described in a block may occur in an order different from the order shown in the figure. For example, two consecutively shown blocks may be executed substantially simultaneously, or, depending on the functionality involved, blocks may be executed in reverse order. It should also be noted that each block in a block diagram or flowchart, or any combination of blocks in a block diagram or flowchart, may be implemented by a dedicated hardware-based system capable of performing a specified function or operation, or by dedicated hardware or a combination of computer instructions.
[0129] Although the subject matter is described above in the general context of computer executable instructions for computers or computer program products running on computers, those skilled in the art will recognize that one or more embodiments described herein can also be implemented at least partially in parallel with one or more other program modules. Generally, a program module includes routines, programs, components, or data structures that perform a particular task or implement a particular abstract data type. Furthermore, the computer implementation methods described above can be implemented in single-processor or multi-processor computer systems, minicomputing devices, mainframe computers, and other computer system configurations including computers, portable computing devices (e.g., PDAs®, telephones), or microprocessor-based or programmable consumer or industrial electronic devices. The illustrated embodiments can also be implemented in a distributed computing environment in which tasks are performed by remote processing devices linked via a communication network. However, at least some, if not all, of one or more embodiments of the one or more embodiments described herein can be implemented on a standalone computer. In a distributed computing environment, program modules can be located on both local and remote memory storage devices.
[0130] As used herein, the terms “component,” “system,” “platform,” or “interface” may refer to, or include, one or more computer-related entities or entities relating to operating machines having a particular functionality. Entities described herein may be hardware, a combination of hardware and software, software, or running software. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. Exemplarily, an application running on a server and both the application and the server may be components. One or more components may reside within a process or an execution thread, and components may be located on one computer or distributed across two or more computers. In another example, each component may run from various computer-readable media on which various data structures are stored. Components may communicate via local or remote processes, such as following signals with one or more data packets (for example, data from one component interacting via signals with another component, such as a local system, a distributed system, or a network such as the Internet). As another example, a component may be a device having a specific functionality provided by mechanical parts operated by electrical or electronic circuits, and such electrical or electronic circuits are operated by software or firmware applications executed by a processor. In such a case, the processor may be internal or external to the device and may execute at least a portion of the software or firmware application. As yet another example, a component may be a device that provides a specific functionality through electronic components without mechanical parts, and such electronic components may include a processor or other means for executing software or firmware that at least partially imparts the functionality of the electronic components.In one embodiment, the component can emulate an electronic component via a virtual machine, for example, within a cloud computing system.
[0131] Furthermore, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or it is clear from the context, “X adopts A or B” is intended to mean any of the natural inclusive permutations. That is, if X adopts A; X adopts B; or X adopts both A and B, “X adopts A or B” is satisfied under any of the above cases. Where used herein, the term “and / or” is intended to have the same meaning as “or.” Furthermore, the articles “a” and “an” used in the specification and accompanying drawings of this subject matter should generally be interpreted as “one or more” unless otherwise specified or it is clear from the context that they refer to a singular form. Where used herein, the terms “example” or “exemplary” are used to mean serving as an example, case, or illustration. To avoid doubt, the subject matter described herein is not limited by such examples. Furthermore, no embodiment or design described herein as “example” or “exemplary” should necessarily be construed as being preferable or advantageous to any other embodiment or design, nor is it intended to exclude equivalent exemplary structures and techniques known to those skilled in the art.
[0132] The disclosure herein describes non-limiting examples of various embodiments. For the sake of convenience of explanation or commentary, various parts of the disclosure herein use the terms “each,” “every,” or “all” when discussing various embodiments. Such uses of the terms “each,” “every,” or “all” are non-limiting examples. In other words, whereever the disclosure herein provides a description that applies to “each,” “every,” or “all” of any particular object or component, it should be understood that it is a non-limiting example of various embodiments, and furthermore, it should be understood that in various other embodiments, such a description may apply to fewer than “each,” “every,” or “all” of that particular object or component.
[0133] As adopted in this specification, the term “processor” can mean substantially any computational processing unit or device, including but not limited to single-core processors; single processors with software multithreading capability; multi-core processors; multi-core processors with software multithreading capability; multi-core processors with hardware multithreading technology; parallel platforms; or parallel platforms with distributed shared memory. In addition, a processor can mean integrated circuits, application-specific integrated circuits (ASICs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic controllers (PLCs), complex programmable logic devices (CPLDs), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Furthermore, a processor may leverage nanoscale architectures, including but not limited to molecular and quantum dot-based transistors, switches, or gates, to optimize space use or improve the performance of associated equipment. A processor can be implemented as a combination of computational processing units.
[0134] In this specification, terms such as “store,” “storage,” “data store,” “data storage,” “database,” and substantially any other information storage component relating to the operation and functionality of a component are used to refer to an entity embodied in a “memory component,” “memory,” or a component containing memory. The memory or memory component described herein may be either volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. To the extent of exemplary, not limited to, non-volatile memory includes read-only memory (ROM), programmable memory (PROM), electrically programmable memory (EPROM), electrically erasable memory (EEPROM), flash memory, or non-volatile random-access memory (RAM) (for example, ferroelectric RAM (FeRAM)). Volatile memory includes RAM that can operate as, for example, external cache memory. To the extent of exemplary, not limited to, RAM may be available in many forms, such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data-rate SDRAM (DDR SDRAM), extended SDRAM (ESDRAM), sync-link DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), or Rambus dynamic RAM (RDRAM). The memory components of the systems or computer implementations described herein are intended to include, but are not limited to, these or any other suitable types of memory.
[0135] The above descriptions include only examples of systems and computer implementations. While it is naturally impossible to describe every conceivable combination of components or computer implementations for the purpose of illustrating one or more embodiments, those skilled in the art will recognize that many more combinations or permutations of one or more embodiments are possible. Furthermore, to the extent that “include,” “has,” “possess,” and similar terms are used in the detailed description, claims, appendices, or drawings, such terms are intended to be as comprehensive as the term “comprising,” in the same manner that “comprising” is interpreted when adopted as a transitional term in a claim.
[0136] The descriptions of various embodiments are presented for illustrative purposes only and are not intended to be exhaustive or limitful to the embodiments described herein. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the embodiments described. The terms used herein have been selected to best describe the principles of the embodiments, their practical applications to the art found in the market, or technical improvements, or to enable other persons skilled in the art to understand the embodiments described herein.
Claims
1. A system comprising a processor that executes computer executable components stored in non-temporary computer-readable memory, wherein the computer executable components are: An access component that accesses the first text passage and the neural sentence encoder; and An execution component that generates a first embedding for the first text passage based on independently executing the neural sentence encoder on multiple fragments of the first text passage. A system that has
2. The system according to claim 1, wherein independently executing the neural sentence encoder on the plurality of fragments results in a plurality of fragment embeddings, and the first embedding for the first text passage is based on fusing the plurality of fragment embeddings.
3. The system according to claim 2, wherein the fusion includes linking or adding together the plurality of fragment embeddings.
4. The system according to claim 2, wherein the fusion includes projecting each of the plurality of fragment embeddings into a dimensionally reduced form, thereby generating a plurality of compressed fragment embeddings, and concatenating the plurality of compressed fragment embeddings together.
5. The access component accesses the second text passage, and the execution component: A second embedding for the second text passage is generated based on executing the neural sentence encoder on the second text passage, and not based on independently executing the neural sentence encoder on multiple fragments of the second text passage; Calculate a similarity score between the first embedding and the second embedding; and In response to the similarity score exceeding a threshold, it is determined that the first text passage is semantically related to the second text passage. The system according to claim 2.
6. The system according to claim 5, wherein the similarity score is based on the cosine similarity or Euclidean distance between the first embedding and the second embedding.
7. In response to determining that the first text passage is semantically related to the second text passage, the execution component: Calculate a second similarity score between the second embedding and each of the plurality of fragment embeddings, thereby generating a plurality of second similarity scores; and Based on identifying which of the plurality of second similarity scores exceeds the threshold, the fragment with the highest semantic relevance among the plurality of fragments is located. The system according to claim 5.
8. The system according to claim 1, wherein, in response to receiving an electronic command requesting the execution of the neural sentence encoder for the first text passage, the execution component independently executes the neural sentence encoder for the plurality of fragments.
9. The system according to claim 1, wherein the execution component decomposes the first text passage into the plurality of fragments through text analysis.
10. The step of accessing a first text passage and a neural sentence encoder by a device operably coupled to the processor; and The device generates a first embedding for the first text passage by independently executing the neural sentence encoder on multiple fragments of the first text passage. A computer implementation method comprising the following:
11. The computer implementation method according to claim 10, wherein independently executing the neural sentence encoder on the plurality of fragments results in a plurality of fragment embeddings, and the first embedding for the first text passage is based on fusing the plurality of fragment embeddings.
12. The computer implementation method according to claim 11, wherein the fusion includes the device linking or adding together the plurality of fragment embeddings.
13. The aforementioned fusion is: The steps include: projecting each of the plurality of fragment embeddings into a dimensionally reduced form using the device, thereby generating a plurality of compressed fragment embeddings; and The device then performs the step of concatenating the multiple compressed fragment embeds together. The computer implementation method according to claim 11, including the method described in claim 11.
14. The step of accessing the second text passage using the aforementioned device; The step of generating a second embedding for the second text passage by the device based on executing the neural sentence encoder on the second text passage, and not based on independently executing the neural sentence encoder on multiple fragments of the second text passage; The device calculates a similarity score between the first embedding and the second embedding; and In response to the similarity score exceeding a threshold, the device determines that the first text passage is semantically related to the second text passage. The computer implementation method according to claim 11, further comprising:
15. The computer implementation method according to claim 14, wherein the similarity score is based on the cosine similarity or Euclidean distance between the first embedding and the second embedding.
16. The steps of: calculating a second similarity score between the second embedding and each of the plurality of fragment embeddings, in response to the device determining that the first text passage is semantically related to the second text passage, thereby generating a plurality of second similarity scores; and The device identifies which of the plurality of second similarity scores exceeds the threshold, and based on this, the step of locating the fragment that has the highest semantic relevance among the plurality of fragments. The computer implementation method according to claim 14, further comprising:
17. The computer implementation method according to claim 10, wherein, in response to receiving an electronic command requesting the execution of the neural sentence encoder on the first text passage, the device independently executes the neural sentence encoder on the plurality of fragments.
18. The device performs the step of decomposing the first text passage into the plurality of fragments through text analysis. The computer implementation method according to claim 10, further comprising:
19. A computer program for facilitating asymmetric relationships through the execution of a fragment-based neural sentence encoder, wherein the processor: Procedure for accessing the first text passage and neural sentence encoder; and A procedure for generating a first embedding for a first text passage based on independently executing the neural sentence encoder on multiple fragments of the first text passage, A computer program designed to execute something.
20. The computer program according to claim 19, wherein independently executing the neural sentence encoder on the plurality of fragments results in a plurality of fragment embeddings, and the first embedding for the first text passage is based on fusing the plurality of fragment embeddings.
