Method for processing natural language using multiple models based on context level domain
Patent Information
- Application Number
- EP2022840307
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-12-30
- Filing Date
- 2022-12-13
- Publication Date
- 2024-11-06
AI Technical Summary
Current Natural Language Understanding (NLU) and Natural Language Processing (NLP) models face challenges in maintaining long-term viability and accuracy as they become increasingly complex, failing to effectively absorb new information without degrading performance, especially in conversational flows with parallel dialogue sequences where interactions are not connected.
The method involves partitioning models into conversational flow context sub-models, allowing new information to be added only to relevant sub-models, reducing the complexity of the original model and using a pipeline of heuristic tests to select the most accurate intent based on dialogue context.
This approach maintains model accuracy by isolating the impact of new information to specific sub-models related to the context, reducing ambiguity and ensuring long-term viability of NLP systems by activating only relevant models for each conversational flow context.
Smart Images

Figure 1.1
Abstract
Description
[0001] DESCRIPTION
[0002] METHOD FOR PROCESSING NATURAL LANGUAGE USING MULTIPLE
[0003] MODELS BASED ON CONTEXT LEVEL DOMAIN
[0004] FIELD OF THE INVENTION
[0005] The present invention is enclosed in the technical area of Natural Language Processing . Speci fically, it belongs to conversational arti ficial intelligence field and relates to Natural Language Understanding engines and the process of choosing the most accurate model to apply at each moment to a conversation flow .
[0006] PRIOR ART
[0007] The digital trans formation of industries and companies implies the use of intelligent systems . They deal with massive amounts of information and interact with customers ( the users ) .
[0008] Due to the daily interactions between the systems and the users , the arti ficial intelligence models need to be fed with more information every day to expand and increase the knowledge , resulting in a more intelligent virtual assistant . This leads to increasingly high complex models that will no longer be viable in the long run because the model complexity most often influences its accuracy .
[0009] On the other hand, current models do not consider information from conversational flows , particular in conversational flows with several parallel dialogue sequences where the interactions of those dialogue sequences are not all connected to each other .
[0010] PROBLEM TO BE SOLVED
[0011] The AT models used in Natural Languages Understanding (NLU) and in Natural Language Processing (NLP ) technologies face a new chal lenge to ensure the long-term viability of models . This requires models to maintain predictability to continue to absorb more information without decreasing their accuracy .
[0012] The present invention aims to overcome such issues by implementing a method to partition the model ' s domain into sub-models based on its conversational flow . This partitioning methodology will create sub-models where adding new information will influence only the sub-models that have context relation with the new information, rather than influencing the original and unique model .
[0013] SUMMARY OF THE INVENTION
[0014] The present invention takes in consideration Portuguese patent application No . 116967 and adds a method that allows the Natural Language Processing o f a user utterance or sentence through several conversational flow context models available . The di f ferent models may be individually activated or assigned given a speci fic flow context . Only the intents that belong to the assigned models will be selected and the most suitable intent is chosen in order to give the best answer, given the dialogue context .
[0015] The user utterances are tested against all activated conversational flow context models , determining the most accurate answer, by running a pipeline of tests that either eliminate or reduce ambiguity, before identi fying the proper and final answer .
[0016] DESCRIPTION OF FIGURES
[0017] Figure 1 - diagram pipeline process representation of the data set preparation and the heuristic model used in the present invention, wherein the reference numbers represent : 100- definition of the data sets with segmented data of the conversational flow context Al models, including the intents and their associated training phrases; 101- heuristic for intent selection.
[0018] Figure 2 - diagram pipeline representation of the heuristic and suitable model answer selection used in the present invention, wherein the reference numbers represent:
[0019] 200- get session context information to identify the active contexts in the conversation session;
[0020] 201 - conversational flow context model consul;
[0021] 202 - applying minimum confidence level of intents;
[0022] 203 - validation of intents input context based on session context information;
[0023] 204 - applying entities match heuristic;
[0024] 205 - evaluation of confidence value of the intent;
[0025] 206 - selection of the correct answer.
[0026] Figure 3 - example of a conversational flow context.
[0027] 300- shape for the conversational flow beginning;
[0028] 301, 306, 307, 314, 315 - visual representation of conversational contexts;
[0029] 302, 303, 308, 309, 310, 316, 317, 318 - represents an intent ;
[0030] 304, 305, 311, 312, 313, 319, 320 - shape with the answer to the user.
[0031] DETAILED DESCRIPTION
[0032] The process that supports the present innovation described in the Summary of the invention will be detailed bellow in accordance with other advantageous and / or preferred embodiments of implementation of the present invention . The present invention provides a method to improve NLP techniques , it proposes : 1 ) splitting the Programs models into conversational flow context models sub-models ; 2 ) extending of the heuristic to integrate the conversational flow context models .
[0033] The purpose of the present invention is to build a conversational AT able to talk about a set of subj ects and their supporting data set , where it is considered, a method composed by the following stages described in Figure 1 : 1 ) corresponds to the data set segmentation ( 100 ) , 2 ) corresponds to the heuristic tests pipeline that are used to perform intent identi fication ( 101 ) .
[0034] Stage 1 :
[0035] The first stage of the proposed method impacts the way data is categori zed and maintained and the present invention add the possibility to partition the sub-sets of intents , designated as Programs , into even smaller sub-sets of intents , designated as Conversational Flow Context models .
[0036] More particularly, data is split into identi fiable di f ferent conversational subj ects , designated as Programs . Each Program holds a sub-set of intents which are related with each other in a linguistic understanding domain and these intents may include certain logical links between them .
[0037] One can define the set of intents of each Program as : where k denotes each intent program .
[0038] The innovative aspect of the present invention is to create a smaller intent grouping model , designated as Conversational Flow Context model that is a partition of an intents Program model . Let one define the intents set of each Conversational Flow Context model as a partition of the intents set of a Program model as :
[0039] (Eq . 2 ) where m denotes each Conversational Flow Context .
[0040] It is possible to note that the space ykof a Program is , in itsel f , composed by the union of all emsubspaces of all Conversational Flow Contexts of that Program . The union of all Conversational Flow Context intents subsets Emof a Program is the intents set of the Program itsel f . You can formulate it as :
[0041] (Eq . 3 ) where m denotes the number of Conversational Flow Context .
[0042] The intents set of a Program and the respective logical links between them represent the conversational flow of a Program, where the logical links represent interactions flows of dialogues sequences . The conversational flow is used to define all interactions , of all dialogue sequences of the Program, as presented in Figure 3 for a generic example of a Program conversational flow . A conversational flow can have defined several parallel dialogue sequences being each one represented by a column vector ds, which can be represented as a sequence of intents ( some intents may occur more than once in the dialogue ) , that represents the dialogue context of the entire dialogue sequence . The column vector dscontains the entire sequence of intents ordered as defined in the interactions flow of the dialogue sequence . Let one define a column vector dsas the entire ordered sequence of intents used to define a particular dialogue sequence :
[0043] (Eq . 4 ) where r represents the number of interactions in the dialogue sequence .
[0044] In this way, a Program conversational flow can be represented in mathematical terms as an s-dimensional vector space being s the number of existing dialog sequences in the Program . Thus , a conversational flow can be represented as a matrix of intents fkwhere each column is a column vector dsas defined in Equation 4 and the rows represent the position in the sequence order . One can define the conversational flow matrix of intents of each Program as :
[0045] (Eq . 5 ) where r represents the number of interactions in the longest dialogue sequence and s represents the number of dialogue sequences . It is important to note the cardinality of each column vector ds, from Equation 5 , could be di f ferent , on several dialogue sequences of the Program, because dialogue sequences could have a di f ferent number of interactions . One can define a Program conversational flow matrix as a multiset intents :
[0046] (Eq . 6 )
[0047] From the conversation flow definition and the Equation 6 one can formulate that Fkis a multiset of which means by definition that FkA Pkis itself. The universe of intents for a conversational flow is, from the conversational flow definition, the Program's set of intents itself, therefore, a conversational flow matrix intent set of a Program can be formulated as:
[0048] Ek=FknPk
[0049] (Eq.7)
[0050] Thus, one can formulate that Ek= Pkwhich means that a conversational flow matrix intent set of a Program is the Program intent set itself:
[0051] When creating a data set exists the liberty to define a splitting logic, subjective and relying on his linguistic understanding of themes, to split the entire data set, into data sub-sets that represent different conversational subjects, designated as Programs. However, the splitting of sub-set Programs into sub-sets of Conversational Flow Context requires an objective approach that should be based in the information of the Program conversational flow. The approach consists into select rows or the columns, but never both simultaneously, of a conversational flow matrix of intents as rule to split the intents set of a Program into Conversational Flow Context intents sub-sets. The splitting procedure allows to define which matrix rows or matrix columns are assigned to which Conversational Flow Context sub-sets.
[0052] You can define the sub-set of intents of each conversational flow matrix row as:
[0053] Rr= {yL-,ysr}
[0054] (Eq.8) where r denotes each row of the conversational flow matrix of intents. One can also define the sub-set of intents of each conversational flow matrix column as :
[0055] (Eq . 9 ) where s denotes each row of the conversational flow matrix of intents .
[0056] From the equation 7 one can formulate in Equation 10 the conversational flow matrix intent set of a Program is either the union of the intents sub-sets of each conversational flow matrix row or the union of the intents sub-sets of each conversational flow matrix column .
[0057] (Eq . 10 )
[0058] As formulated in Equation 10 , the intents set of a Program Pkis either the union of the intents sub-sets of each conversational flow matrix row or the union of the intents sub-sets of each conversational flow matrix column . Thus , by applying the rule for spl itting the intents set of a Program into Conversational Flow Context intents sub-sets , one can choose between rows or columns , which implies that each Conversational Flow Context intents sub-set Cmcould be either a union of sub-sets Rror a union of sub-sets Ss.
[0059] Therefore , let one define the sub-intents set of each Conversational Flow as :
[0060] 1 ) - for rows selection
[0061] (Eq . 11 )
[0062] 2 ) - for columns selection (Eq . 12 ) where m denotes each Conversational Flow Context .
[0063] A Program being partitioned into a set of Conversational Flow Context models reduces the co-dependence between the set intents from a program . Changes ( addition of new training phrases ) in one intent will not influence the confidence of all program intents but only intents from the conversational flow context model where this intent belong . Stage 1 details the process of data set preparation and the model ' s generation, how these models will be used and the detail about the pipeline process used to perform intent identi fication ( classi fication) is detailed in stage 2 .
[0064] Stage 2 :
[0065] The second stage identi fies the detailed pipeline process starting from the user utterance arrival at the system until it returns the most suitable response for the user utterance . This pipeline process contains heuristic based tests that eliminate incorrect intents , albeit possessing high confidence results , thus reducing ambiguity in intent identi fication .
[0066] To make use of the Conversation Flow Context models , a new heuristic was developed, which is composed of 7 steps , as represented in Figure 2 , and takes advantage of the existing NLP engine that supports the Conversational Flow Context Models mentioned .
[0067] A description of the heuristic steps is provided below :
[0068] At step 200 , as represented in Figure 2 , the arrival of the user utterance starts the pipeline process , and the session context information is used to directly influence the selection and activation of the models to use . For example , as represent in Figure 3 , in the beginning of the conversational flow the context start ( 301 ) is the only one that is active in the session context . Session context information has the list of active contexts in current moment of the conversation . The conversational flow context models that will be active are the models that contains intents with the context start as input context . It is applied a process to choose the conversational flow context models based on the contexts present in session context information . When the utterance arrives the intent 302 and 303 will be the candidates and have the context start as an input context . I f the intent 302 is chosen, the response defined in shape 304 wil l be given to the user and the context 306 will be activated because is the output context of the intent selected and intent 308 will be the next candidate . After this point the same process is applied to intent 308 .
[0069] At step 201 , the knowledge of natural language processing is extracted from the conversational flow context models selected in the step 200 .
[0070] At step 202 , the confidence thresholds per program are used to improve the accuracy . A set of problems exists associated with identi fications of intents from a single data set , therefore, with a segmentation approach these problems are controlled . Each Program will have the most suitable confidence threshold which can be calculated based on accuracy values , for example a higher threshold should be set when the intent or question expected to be identi fied is very speci fic and it is necessary to really be certain of the subj ect being talked about . The confidence threshold eliminates intents candidates selected in stage 1 that have confidence below the threshold value . As formulated in patent application No . 116967 , the intent candidates can be defined as the following : where akis the threshold for a speci fic program Pkwhere the intent yt is defined .
[0071] At step 203 , although the maj ority of intents are discarded as intent candidates through the use of the Program confidence threshold criteria, some may still remain as candidates . The identi fied context is used to reduce the number of non-context related intents which are present in the set returned by the NLU Engine in the previous step 201 . As represented in Figure 3 , a context corresponds to a point in a conversational flow that is reached in an interactions sequence of a dialogue . From an intent point o f view a context can be viewed as : an input context when it precedes the intent or an output context when it follows the intent .
[0072] In a more formal way, an intent (y of a Conversational Flow Context may contain a variable number (nc ) of input contexts ( t^) :
[0073] One can define the set Gkof input contexts that includes all the input contexts of each Program as : where k denotes each Program . Thus , one can def ine the set Gmof input contexts that includes all the input contexts of each Conversation Flow Context model as : z Gm= 4 i=l where m denotes each Conversation Flow Context model .
[0074] The conversation context ( dialog history) represents the sequence of intents and input contexts , from the start of the conversation up until the current input context (next intent ) . It is likely that the next input context in a conversation is related to the current input context though the next intent . The sequence of input contexts crossed during a conversation corresponds to the sequence of input contexts activated throughout the conversation being the sequence of the recently input contexts activated Okstored in the session context information of a Program :
[0075] One can say with higher degree of certainty that the intents candidates will be the ones which have input context recently activated .
[0076] At step 204 , entities are used in the pipeline process to resolve some ambiguity that still remain . Entities may be nouns or contents of actions that are performed . As an example , in an utterance : " I like to play football" , football is a noun . One could easily replace " football" by "basketball" , " snooker" , "hockey" changing the obj ect while retaining the idea that " I like to play <sport>" . Therefore , entities are the nouns or the content for the action that needs to be performed . For example , London, Madrid, and Paris are different words, but they all refer to a city, so city should be an entity. Utterances may include several entities in them, and the model that identifies intents from utterances is also capable of extracting the entities from the utterance. For each utterance, the results of intent recognition must also include the recognized entities and the intent selection heuristic, when faced with ambiguity between certain intent candidates, will choose the intent that possesses a higher number of recognized entities.
[0077] In a more formal way, an intent (y£) of a Conversational Flow Context may contain a variable number (ne) of entities (vin);
[0078] The number of entities of an intent is the dimension of the non-zero elements of the array: numberEntitieSy. = dim(vi)
[0079] The resulting intent candidates from the previous pipeline heuristic is: y candidates [yi, yn}
[0080] Thus, to choose the candidate with more recognized entities:
[0081] However, if both intent candidates have the same number of recognized entities other criteria will be used as the next step.
[0082] At step 205, if the ambiguity remains in the previous step, a heuristic that selects the intent with the highest confidence will be used: At step 206 , the last step in the pipeline process , the answer associated to the chosen intent , for a received user utterance , is returned to the user .
[0083] Each sub-model , hereby designated as
[0084] Conversational Flow Context , is comprised of a data set of utterances , or training phrases , that define intents , which are a subsets of a Program intents set . These intents could be logically linked to form conversational flows used in Program models .
[0085] In order for the method developed be able to select the best knowledge sub-model for the provided utterance , it shall process the results of a NLU engine , along a pipeline of heuristic-based tests that el iminate incorrect intents , albeit possessing high confidence results , thus reducing ambiguity in intent classi fication . The procedure of the described pipeline process uses previously recogni zed intents in an established session, as a way to define a context of the current dialogue .
[0086] DESCRIPTION OF THE EMBODIMENTS
[0087] In the following an embodiment of the present invention being related to a method for natural language processing based on conversational flow context models will be described . Said method is comprised of the following steps :
[0088] — it is performed the partitioning of an intents data set into multiple sub-sets designated as Conversational Flow Context models , according to a defined criterion, to reduce the high coupling between intents . Adding new training phrases or any type of change to a Program intent will no longer influence the accuracy of the other Program intents, except those belonging to the Conversation Flow Context model whose intent has changed .
[0089] — the session context information (200) is used to find the conversational flow context models.
[0090] — conversational flow context models request is made (201) to perform an intent recognition and extraction of entities.
[0091] — it's applied another pipeline step on intents based on a pre-set confidence threshold (202) that is previously defined .
[0092] — session context information it is used to refine the intents candidates from previous step (203) . The same conversational flow context model can have intents with different input contexts, as represented in conversational flow context in Figure 3, that when are active can be used to reduce the list of intents candidate .
[0093] — applies a procedure to calculate the number of matched entities for each intent and select the one(s) with the highest number (204) .
[0094] — selects the intent with the highest confidence (205)
[0095] — finally, the answer associated to the intent selection in the previous step is returned (206) .
Claims
CLAIMS1 . Method for processing natural language using multiple models based on context level domain, comprising the steps of :— Partitioning of an intents data set into multiple subsets of conversational flow context models ;— implementation of a pipeline of heuristic-based tests for intent identi fication, wherein, conversation flow context models are sub-sets of intents resulting from partition of sub-sets of intents , designated as Programs ; each program holding a subset of intents which are related with each other in a linguistic understanding domain .2 . Method according to claim x, wherein the partition of an intents data set into multiple sub-sets of conversational flow context models include :— definition of data sub-sets with segmented data by splitting the entire data set of intents into Programs , each representing di f ferent conversational subj ects ;— generation of a program conversation flow matrix,wherein, each column of fkhas the entire ordered sequence of intents y used to define a particular dialogue sequence , each row of fkrepresents the position in the sequence order, r represents the number of interactions in the longest dialogue sequence ,s represents the number of dialogue sequences.— splitting programs into sub-sets of conversational flow context models, by selection at least on row or at least one column of fk.
3. Method according to any of the previous claims, wherein the pipeline of heuristic-based tests includes the following steps:— extracting session context information to identify active conversational flow context models in a conversation session; (200)— extracting knowledge of natural language processing from the conversational flow context models activated;(201)— applying a pre-set confidence threshold level on intents of active conversational flow context models;(202)— validation of intents input context based on session context information; (203)— applying entities match heuristic to select intents with the highest number of matched entities; (204)— evaluation of confidence value of each selected intent; (205)— identification of the intent with the highest confidence .
4. Method according to claim 3, wherein the identification of conversational flow context models is performed based on a user utterance and on session context information .
5. Method according to claim 4, wherein the session context information comprises the list of active18 conversation flow contexts models in current moment of conversation; the active conversation flow context models containing at least one intent with a context start as input context .
6. Method according to any of the previous claims 3 to 5 , wherein an entity is noun or a content of action .7 . Method according to any of the previous claims 3 to 6 , wherein the step of extracting knowledge from the activated conversational flow context models includes performing intent recognition and extraction of entities .8 . Method according to any of the previous claims 3 to 7 , wherein intents of an activated conversational flow context model having confidence below the pre-set threshold are eliminated .
9. Method according to claim 8 , wherein the step of validating intents input context based on ses sion context information is applied to intents having a confidence higher than the pre-set threshold .10 . Method according to any of the previous claims 3 to 9 , wherein the step of applying entities match heuristic includes :— calculate the number of matched entities for each intent ; and— select the intent with the highest number of matched entities .
