System and method for automatic topic detection within text.

Abstract

A method and system for automatic topic detection in text may include receiving text documents of a corpus of documents and extracting one or more phrases from the documents based on one or more syntactic patterns. For each phrase, an embodiment of the invention may apply a word embedding neural network to one or more words of the phrase to obtain one or more respective word embedding vectors, calculate a weighted phrase embedding vector, and calculate a phrase salience score based on the weighted phrase embedding vector. An embodiment of the invention may then generate one or more topic labels representing one or more respective topics in the document based on the calculated phrase salience scores, and select the one or more topic labels according to their relevance to the business domain of the corpus.

Description

[Technical Field]

[0001] This invention generally relates to the field of natural language processing (NLP). More specifically, it relates to a method and system for analyzing text.

[0002] (Cross-referencing of related applications and claim of priority) This application claims priority to U.S. Patent Application No. 17 / 318,524, titled "SYSTEM AND METHOD OF AUTOMATIC TOPIC DETECTION IN TEXT," filed on 12 May 2021. [Background technology]

[0003] Contact centers engage in numerous interactions with customers. These customer interactions can take many forms, such as phone calls, chat rooms, emails, text documents, and web pages. The topics of these interactions can also vary widely, including complaints, requests for assistance, and general questions.

[0004] Analyzing large volumes of dialogue (e.g., several thousand per day) can be done more efficiently using automated systems, which are generally called topic detection systems. Topic detection systems can identify recurring topics, measure their frequency, and group related topics and subtopics according to their subject matter.

[0005] Currently available topic detection systems, which find recurring topics or concepts across a corpus of customer conversations, are prone to generating noise, as is commonly known in the art, leading to the detection of topics that are not necessarily of interest to the end-user or analyst of the topic detection system.

[0006] Examples of such noise in an organization's contact center may include repeated phrases that may not be related to the organization's business line or may not be related to the central theme of the conversation. A common way to filter such terms or phrases is by maintaining a "permitted list" and / or "denied list" that can be manually predefined and / or updated. However, such a strict solution may not handle new or unexpected terms or phrases well.

Summary of the Invention

[0007] The following table, Table 1, may be used herein as a reference to terms or expressions used herein.

[0008]

Table 1

[0009] Therefore, there is a need for methods and systems for the automatic evaluation and filtering of topic labels detected by a topic detection system.

[0010] As detailed herein, embodiments of the present invention can improve the quality or relevance of concepts or topics found by a topic detection system by (a) evaluating the salience or quality of phrases included in a conversation and filtering out low-quality phrases, and (b) evaluating the relevance of the remaining phrases in the context of a particular relevant business domain.

[0011] According to some embodiments, in a first stage, embodiments of the present invention can utilize the dynamics of a conversation to mark or select phrases that are likely to be prominent with respect to the conversation, in the sense that they may be most relevant to the central theme of the conversation. For example, in this first stage, embodiments of the present invention can filter out phrases such as "friend of mine", "feel free", "bear with me just a moment", and "name and telephone number" that may lack salience in the context of the conversation.

[0012] In addition, in a second stage, embodiments of the present invention can utilize the settings of a multi-tenant cloud-based infrastructure to find which concepts described within the conversation are more likely to be relevant to a particular tenant's business line. For example, in this second stage, embodiments of the present invention can filter out topics or concepts such as "collect information", "manage resources", and "answer questions" that may be irrelevant or too general and of little informational value in the context of a particular tenant.

[0013] As detailed herein, embodiments of the present invention can integrate phrase filtering according to salience and concept filtering according to relevance into a system that can analyze tenant conversations, facilitating the discovery of recurring topics of interest to a tenant (such as a call center user or analyst).

[0014] Embodiments of the present invention may include a method for automatic topic detection in text by at least one processor. Embodiments of the method include receiving at least one first text document of a first plurality of text documents, and extracting, based on one or more syntactic patterns, one or more phrases from the first text document, each phrase including one or more words.

[0015] For each phrase, embodiments of the present invention may apply a word embedding neural network (NN) to one or more words of the phrase to obtain one or more word embedding vectors. The one or more word embedding vectors may be adapted to encode the meaning of one or more corresponding words of the phrase. Embodiments may then compute a weighted phrase embedding vector based on the one or more word embedding vectors, and compute a phrase splendor score based on the weighted phrase embedding vector. Embodiments of the present invention may then generate one or more topic labels based on the computed phrase splendor score, each representing one or more topics of at least one first text document.

[0016] Embodiments of the present invention can calculate a weighted phrase embedding vector by calculating a word weight value for each word in a phrase based on the frequency of occurrence of the word in a first set of text documents, and then calculating a weighted phrase embedding vector as a weighted average of the related word embedding vectors based on the related word weight values.

[0017] Embodiments of the present invention involve arranging one or more phrases as vertices in an undirected phrase graph, wherein each pair of vertices can be connected by an edge; calculating similarity weights for one or more edges that represent the semantic similarity between the two phrases of the corresponding connected vertices; and calculating a phrase splendor score for each phrase based on the similarity weights of the connected edges.

[0018] Embodiments of the present invention can assign similarity weights representing semantic similarity between two phrases by calculating cosine similarity values ​​between the word embedding vectors of the two phrases and assigning the cosine similarity values ​​as similarity weights to the edges connecting the vertices of the two phrases.

[0019] Embodiments of the present invention may involve an iterative process for calculating phrase splendor scores, each iteration of which may include: calculating a phrase splendor score for a first phrase based on (a) similarity weights of edges connected to each vertex and (b) phrase splendor scores for at least one second phrase; and updating the phrase splendor scores for at least one second phrase based on the calculated phrase splendor scores for the first phrase.

[0020] Embodiments of the present invention can generate one or more topic labels by obtaining a plurality of phrases and a plurality of phrase splendor scores from a first plurality of documents, selecting a first subset of the plurality of phrases corresponding to the top phrase splendor score, and clustering the subset of phrases into clusters based on the calculated cosine similarity values, wherein each cluster represents a unique topic label.

[0021] According to some embodiments of the present invention, a first set of documents relates to a first business domain.

[0022] Embodiments of the present invention may obtain a second set of documents related to at least one second business domain, calculate a foreground probability value for each topic label, which represents the probability that the topic label represents a topic included in the first set of documents, calculate a background probability value, which represents the probability that the topic label represents a topic included in one of the first set of documents and one of the second set of documents, and calculate a relevance score, which represents the relevance of the topic label to the first business domain, based on the foreground and background probability values.

[0023] Embodiments of the present invention can calculate the foreground probability value by performing a fuzzy search of topic labels to count the number of documents N1 that contain the relevant topic among a first set of documents, and by calculating the foreground probability value as the ratio of N1 to the number of documents included in the first business domain.

[0024] Embodiments of the present invention can calculate the background probability value by performing a fuzzy search of topic labels to count the number of documents N2 that contain the relevant topic among a first plurality of documents and a second plurality of documents, and by calculating the background probability value as the ratio of N2 to the number of documents included in the first business domain and at least one second business domain.

[0025] Embodiments of the present invention may receive at least one query regarding topics present in a first set of documents via a user interface (UI) and generate a response to the query based on one or more relevant topic labels. In addition, or alternatively, embodiments of the present invention may generate a response to a query based on one or more topic labels and corresponding relevance scores.

[0026] Embodiments of the present invention may include a system for automatic topic detection within text. Embodiments of the system may include a non-temporary memory device in which modules of instruction code can be stored, and at least one processor associated with the memory device and configured to execute the modules of instruction code.

[0027] When executing the module of the above instruction code, at least one processor may be configured to receive a first text document from a first plurality of text documents, and to extract one or more phrases from the first text document based on one or more syntactic patterns, wherein each phrase may contain one or more words.

[0028] For each phrase, an embodiment of this system may apply a word embedding neural network (NN) to one or more words in the phrase to obtain one or more word embedding vectors, calculate a weighted phrase embedding vector based on the one or more word embedding vectors, and calculate a phrase splendor score based on the weighted phrase embedding vector.

[0029] Embodiments of this system may subsequently generate one or more topic labels representing one or more respective topics of at least one first text document, based on the calculated phrase splendor score of each phrase. [Brief explanation of the drawing]

[0030] The subject matter considered to be part of the present invention is specifically pointed out and explicitly asserted in the concluding section of this specification. However, the present invention, with regard to both its configuration and method of operation, along with its object, features and advantages, can be best understood by reference to the following detailed description when read in conjunction with the accompanying drawings. [Figure 1] This is a block diagram showing computing devices that may be included in a system for automatic topic detection within text, according to several embodiments of the present invention. [Figure 2] This is a block diagram illustrating an overview of a system for automatic topic detection within text, according to several embodiments of the present invention. [Figure 3A] This is a block diagram showing a spleness calculation module that may be included in a system for automatic topic detection within text, according to several embodiments of the present invention. [Figure 3B] This is a block diagram showing a spleness calculation module that may be included in a system for automatic topic detection within text, according to several embodiments of the present invention. [Figure 4] This is a schematic diagram showing a phrase similarity graph that may be included in a system for automatic topic detection within text according to several embodiments of the present invention. [Figure 5] This is a block diagram showing a relevance calculation module that may be included in a system for automatic topic detection within text, according to several embodiments of the present invention. [Figure 6] This graph shows an example of improved related topic identification according to embodiments of the present invention. [Figure 7]This is a block diagram showing an example of an application module that may be included in or associated with a system for automatic topic detection within text, according to several embodiments of the present invention. [Figure 8] This is a flowchart illustrating a method for automatic topic detection within text according to several embodiments of the present invention.

[0031] For the sake of simplicity and clarity in the illustrations, it should be understood that the elements shown in the diagrams are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Furthermore, where deemed appropriate, reference numbers may be repeated between drawings to indicate corresponding or similar elements. [Modes for carrying out the invention]

[0032] Those skilled in the art will understand that the present invention can be implemented in other specific forms without departing from its spirit or essential features. Therefore, the embodiments described herein should be considered illustrative in all respects and not limiting the invention as described herein. Thus, the scope of the invention is indicated not by the foregoing description but by the appended claims, and all modifications that fall within the meaning and scope of the equivalents of the claims are intended to be encompassed therein.

[0033] The following detailed description includes numerous specific details to provide a complete understanding of the invention. However, it will be understood by those skilled in the art that the invention can be carried out without these specific details. In other examples, well-known methods, procedures, and components are not described in detail so as not to obscure the invention. Some features or elements described in relation to one embodiment may be combined with features or elements described in relation to other embodiments. For clarity, descriptions of the same or similar features or elements may not be repeated.

[0034] Embodiments of the present invention are not limited in this respect, but discussions using terms such as “process,” “calculate,” “calculate,” “determine,” “establish,” “analyze,” and “check” may refer to the operation and / or process of a computer, computing platform, computing system, or other electronic computing device that manipulates and / or converts data represented as physical (e.g., electronic) quantities in the computer’s registers and / or memory into other data similarly represented as physical quantities in the computer’s registers and / or memory, or in other non-temporary information storage medium capable of storing instructions for performing operations and / or processes.

[0035] Embodiments of the present invention are not limited in this respect, but as used herein, the terms “plurality” and “a plurality” may include, for example, “multiple” or “two or more.” The terms “plurality” or “a plurality” may be used throughout this specification to describe two or more components, devices, elements, units, parameters, etc. The term “set,” as used herein, may include one or more items.

[0036] Unless expressly stated otherwise, embodiments of the methods described herein are not restricted to any particular order or sequence. In addition, some embodiments or elements of the methods described herein may occur or be performed simultaneously, at the same time, or in parallel.

[0037] Referring now to Figure 1, this figure is a block diagram showing a computing device that may be included in one embodiment of a system for performing text analysis, according to several embodiments.

[0038] Computing device 1 may include, for example, a processor or controller 2 which may be a central processing unit (CPU) processor, a chip, or any suitable computing device or computing device, an operating system 3, memory 4, executable code 5, a storage system 6, an input device 7, and an output device 8. Processor 2 (or, optionally, one or more controllers or processors across multiple units or devices) may be configured to perform the methods described herein and / or to run or operate as various modules, units, etc. Two or more computing devices 1 may be included, and one or more computing devices 1 may operate as components of a system according to embodiments of the present invention.

[0039] Operating System 3 may be or include any code segment (for example, similar to executable code 5 described herein) designed and / or configured to perform tasks including coordinating, scheduling, mediating, supervising, controlling, or otherwise managing the operation of computing device 1, such as scheduling the execution of a software program or task, or enabling a software program or other module or unit to communicate. Operating System 3 may be a commercial operating system. It should be noted that Operating System 3 may be an optional component, and in some embodiments, for example, the system may include computing devices that do not require or include Operating System 3.

[0040] Memory 4 may be, or include, for example, random access memory (RAM), read-only memory (ROM), dynamic RAM (DRAM), synchronous DRAM (SD-RAM), double data rate (DDR) memory chip, flash memory, volatile memory, non-volatile memory, cache memory, buffer, short-term memory unit, long-term memory unit, or other suitable memory unit or storage unit. Memory 4 may be, or include, a plurality of different possible memory units. Memory 4 may be a computer or processor non-temporary readable medium, or a computer non-temporary storage medium, such as RAM. In one embodiment, a non-temporary storage medium such as memory 4, a hard disk drive, or another storage device may store instructions or code that, when executed by the processor, cause the processor to perform the methods described herein.

[0041] Executable code 5 can be any executable code, such as an application, program, process, task, or script. Executable code 5 may be executed by a processor or controller 2, possibly under the control of the operating system 3. For example, executable code 5 may be an application capable of performing text analysis, as further described herein. For clarity, a single item of executable code 5 is shown in Figure 1, but systems according to some embodiments of the present invention may include multiple executable code segments similar to executable code 5, which are loaded into memory 4 and cause the processor 2 to execute the methods described herein.

[0042] The storage system 6 may be, for example, a flash memory as known in the art, a memory located inside or embedded in a microcontroller or chip as known in the art, a hard disk drive, a CD-Recordable (CD-R) drive, a Blu-ray disc (BD), a Universal Serial Bus (USB) device, or other suitable removable and / or fixed storage units, or may include them. Data related to text analysis may be stored in the storage system 6, loaded from the storage system 6 into memory 4, and processed by the processor or controller 2. In some embodiments, some of the components shown in Figure 1 may be omitted. For example, memory 4 may be a non-volatile memory having the storage capacity of the storage system 6. Thus, although shown as a separate component, the storage system 6 may be embedded in or included in memory 4.

[0043] Input device 7 may be or include any suitable input device, component, or system, such as a detachable keyboard or keypad, a mouse, etc. Output device 8 may include one or more (potentially detachable) displays or monitors, speakers, and / or any other suitable output devices. Any applicable input / output (I / O) devices may be connected to computing device 1 as shown by blocks 7 and 8. For example, a wired or wireless network interface card (NIC), a Universal Serial Bus (USB) device, or an external hard drive may be included in input device 7 and / or output device 8. It will be recognized that any number of suitable input devices 7 and output devices 8 may be operably connected to computing device 1 as shown by blocks 7 and 8.

[0044] Systems according to some embodiments of the present invention may include, but are not limited to, a plurality of central processing units (CPUs) or any other suitable multipurpose or specific processor or controller (similar to, for example, element 2), a plurality of input units, a plurality of output units, a plurality of memory units, and a plurality of storage units.

[0045] A neural network (NN) or artificial neural network (ANN), such as a neural network implementing machine learning (ML) or artificial intelligence (AI) functions, can refer to an information processing paradigm that may include nodes, called neurons, organized into layers, with links between them. These links can transfer signals between neurons and may be associated with weights. An NN can be configured or trained for a specific task, such as pattern recognition or classification. Training an NN for a specific task may involve adjusting these weights based on examples. Each neuron in an intermediate or final layer may receive an input signal, such as a weighted sum of output signals from other neurons, and may process the input signal using a linear or nonlinear function (e.g., an activation function). The results of the input and intermediate layers may be transferred to other neurons, and the results of the output layer may be provided as the output of the NN. Typically, neurons and links within an NN are represented by mathematical constructs such as activation functions and matrices of data elements and weights. A processor, such as a CPU or graphics processing unit (GPU), or dedicated hardware device may perform the relevant calculations.

[0046] Referring now to Figure 2, this figure shows an overview of a system for automatic topic detection within text according to several embodiments of the present invention.

[0047] According to some embodiments of the present invention, system 100 may be implemented as a software module, a hardware module, or any combination thereof. For example, system 100 may be or include a computing device such as element 1 in Figure 1, which may be adapted to perform text analysis by executing one or more modules of executable code (e.g., element 5 in Figure 1), as further described herein.

[0048] As shown in Figure 2, the arrows may represent the flow of one or more data elements to and from system 100, and / or between modules or elements of system 100. Some arrows are omitted in Figure 2 for clarity.

[0049] As shown in Figure 2, the system 100 may include a splendor calculation module 200 configured to receive at least one text document data element 21A. The text document 21A may be associated with or included in multiple groups or corpora 21, as commonly referred to in the art, relating to a corpus 21 associated with the domain 20' of a particular organization or tenant 20 (e.g., a particular car rental agency) and / or similar organizations or tenants (e.g., multiple car rental agencies).

[0050] According to some embodiments, as detailed herein, the spleness calculation module 200 may extract one or more phrases or fragments from a text document 21A based on one or more syntactic patterns. Each fragment may contain one or more words, such as a group of adjacent or related words in the text document 21A. The spleness calculation module 200 may then calculate a spleness score for one or more extracted phrases or fragments.

[0051] According to some embodiments, the splendor calculation module 200 may select a portion or a certain percentage of phrases corresponding to the highest-ranked splendor scores, as detailed herein. In addition, or alternatively, the splendor calculation module 200 may select a subset of phrases having scores above a predetermined threshold, as detailed herein. These selected phrases are shown as “Splendid Phrases 200A” in Figure 2.

[0052] As shown in Figure 2, the system 100 may include a relevance calculation module 300 adapted to receive one or more selected prominent phrases 200A, as detailed herein, and to generate one or more initial labels or indicators from the received prominent phrases for topics discussed in or contained within at least one document 21A. The relevance calculation module 300 may then analyze the initial labels to determine the relevance score for one or more (e.g., each) candidate topics in the context of a particular tenant 20 and / or domain 20'. The relevance calculation module 300 may then mark or select a subset or portion of topic labels corresponding to the highest relevance score (e.g., a score above a predetermined threshold) to represent at least one topic contained in document 21A. These selected topic labels are shown as “Relevant Topic Labels 300A” in Figure 2.

[0053] According to some embodiments, the system 100 may include or store prominent phrases 200A and / or related topic labels 300A as topic data structures 400B (e.g., tables, linked lists, etc.) on a database or storage or memory device (e.g., storage system 6 in Figure 1). For example, each entry in the topic data structure 400B may associate a particular document 21A with one or more prominent phrases 200A and / or related topic labels 300A.

[0054] System 100 may use the topic data structure 400B to provide, for example, a user or an application running on a computing device (e.g., computing device 1 in Figure 1) with one or more topic metadata 400A data elements related to a document 21A. For example, a user may present a database query to System 100 regarding the content of a topic in several documents 21A, such as "which documents discuss the topic of a particular technical issue." System 100 may respond to the query with a query response containing topic metadata 400A. In this example, System 100 may include in this query response 400A a list of titles or a pointer to a document 21A that discusses a particular technical issue.

[0055] As is known in the art, currently available systems for topic detection may attempt to find recurring topics within a corpus of documents by clustering similar phrases together with the help of some form of similarity metric. The clustering method and similarity metric may vary between different implementations. The output of such a process is a set of labels representing a particular topic. The set of topic labels should be formed to maximize diversity (or minimum similarity) between labels, and each label will be understood to be a topic of interest to the end user. In the example of contact center conversations, this means that the user is interested in finding specific conversations that contain this topic by looking at metadata about the relevant topic, such as the amount of conversations containing that topic and how they change over time.

[0056] The systems currently available for topic detection can employ simple methods in which the input for the topic detection process can be the complete text of each dialogue (e.g., in the form of N-grams or syntactic patterns).

[0057] Embodiments of the present invention may include improvements to currently available text analysis techniques. Embodiments may improve the resulting set of topic labels, as detailed herein, by (a) filtering out less prominent text from the input document to obtain prominent phrases 200A, and (b) performing relevance analysis based on the most prominent phrases to obtain relevant topic labels 300A. For example, embodiments of the present invention may minimize the similarity between topic labels in the output set of topic labels. In addition, embodiments of the present invention may better select and present (e.g., to the analyst) the topic labels most relevant to a particular tenant 20 and / or domain 20', as detailed herein.

[0058] Referring now to Figure 3A, this figure is a block diagram showing a spleness calculation module 200 that may be included in a system 100 for automatic topic detection in text according to several embodiments of the present invention.

[0059] As shown in Figure 3A, the sampling calculation module 200 (or simply module 200) may include a phrase extraction module 210. The phrase extraction module 210 may be adapted to receive (a) at least one text document 21A, and (b) one or more syntactic patterns or rules 31A. One or more syntactic patterns 31A may be or include a data structure such as a table, which may be held or stored in a database (e.g., storage device 6 in Figure 1). As is known in the art, one or more syntactic patterns 31A may associate one or more words with grammatical roles in sentences of human languages ​​(e.g., English, Spanish, etc.), and thus can decipher the grammar of human languages. The phrase extraction module 210 may apply one or more syntactic patterns 31A to the text document 21A to extract one or more fragments or phrases 210A from the text document 21A, each fragment or phrase 210A containing one or more words 210B.

[0060] According to some embodiments, the phrase extraction module 210 may include a part-of-speech (PoS) tagger, a tool commonly used for natural language processing (NLP) applications. The PoS tagger may be configured to analyze the grammar of an incoming sentence and may annotate each word in the sentence with an appropriate (POS) tag, such as noun, verb, or adjective. According to some embodiments, the phrase extraction module 210 may use the PoS tagger to annotate words in document 21A. The phrase extraction module 210 may then select a sequence of words (e.g., phrase 210A) in which each PoS tag matches one of the desired predefined syntactic patterns 31A. For example, for the predefined syntactic patterns 31A: ("verb->determiner->noun", "verb->determiner->adjective->noun"), the sequences "book a flight" and "book a quick flight" match and are extracted as phrase 210A. On the other hand, the sequence "flight booking" does not match and is not extracted as phrase 210A. Therefore, the phrase extraction module 210 can receive entire text conversations, dialogues, and / or documents 21A as input and generate from them all phrases or fragments 210A that match one of the predefined syntactic patterns 31A.

[0061] For example, the first phrase 210A may include the text "what I asked for", the second phrase 210A may include the text "request a replacement", and the third phrase 210A may include the text "something similar". These examples of phrase 210A are used herein to further elaborate on aspects of the present invention.

[0062] As is known in the field of artificial intelligence (AI)-based natural language processing (NLP), the vocabulary of human language consists of discrete elements such as words. An NLP system can use a neural network architecture to map each discrete element (e.g., each word) to vectors commonly called "embedding vectors" in a vector space commonly called the "embedding vector space." Using this embedding vector space representation of words can allow an NLP system to have a continuous distributed representation of the vocabulary of human language. The advantage of this continuous embedding vector representation is that an NLP system can map semantically similar words to similar regions in the embedding vector space (e.g., represented by similar embedding vectors). For example, the embedding vector representation of the word "cat" may be more similar (according to some predetermined metric) to the embedding vector representation of the word "feline" than to the embedding vector representation of the word "piano."

[0063] As shown in Figure 3A, module 200 may include a word embedding vector computation module 220. In some embodiments, the embedding vector computation module 220 may be or include a word embedding neural network (NN) adapted to receive words of human language and generate an embedding vector representation 220A of the received words from those words. The embedding vector computation module 220 may also be referred to herein by abbreviation as “module 220” or “embedding NN220”.

[0064] According to some embodiments, module 200 may apply an embedding NN220 to one or more words 210B of the extracted fragment 210A to obtain one or more word embedding vectors 220A for each word. As described above, the embedding vectors 220A may be adapted to encode the meaning of one or more corresponding words 210B of the fragment or phrase 210A.

[0065] With respect to the example phrase 210A, "something similar," the embedding NN220 can generate a first embedding vector 220A representing the word 210B "something" and a second embedding vector 220A representing the word 210B "similar" from phrase 210A.

[0066] As is well known in the field of NLP, the word frequency-inverse document frequency (TF-IDF) score is a numerical statistic that can indicate how important a word is to a particular document within a collection or corpus of documents. The TF-IDF score can be used, for example, as a weighting coefficient in automated retrieval for text information retrieval and text mining. The TF-IDF score value can increase proportionally to the number of times a word appears in a particular document, and is offset by the number of documents containing that word within the corpus. This offset can compensate for the fact that some words generally appear more frequently.

[0067] As shown in Figure 3A, module 200 may include a word weight calculation module 230 that can receive at least one word 210B (e.g., the word "something") from an extracted phrase 210A (e.g., the phrase "something similar") and calculate a weight 230A corresponding to the received word 210B.

[0068] According to some embodiments, the weight calculation module 230 may be a TF-IDF score calculator, and the weight 230A may be a TF-IDF score value. For example, the weight calculation module 230 may calculate the weight 230A of a particular word with respect to a particular document 21A as a TF-IDF function of (a) the number of occurrences of the particular word 210B in the particular document 21A, and (b) the number of occurrences of the particular word 210B in the corpus or multiple 21 of document 21A related to a particular tenant 20 or domain 20'.

[0069] In addition, or alternatively, the word weight calculation module 230 may calculate the weight 230A according to the following formula 1.

[0070]

number

[0071] As shown in the exemplary formula 1, a person skilled in the art would know that the word weight 230A (for example, α[w i It will be understood that this can be calculated, for example, as the inverse document frequency function (e.g., IDF) of the TF-IDF function.

[0072] Referring now to Figure 3B, this figure is a block diagram showing alternative implementations of the spleness calculation module 200, which may be included in a system 100 for automatic topic detection in text according to several embodiments of the present invention. In Figure 3B, the implementation of the spleness calculation module 200 is indicated by 200' and may include two parts or modules, indicated by 200'A and 200'B.

[0073] It will be understood that modules 200'A and 200'B may include modules (e.g., 210, 220, 230, 240, 250, 260, and 270) that are similar to or identical to those of the splendor calculation module 200 in Figure 3A. It will be understood that modules 200'A and 200'B may be implemented on the same computing device (e.g., computing device 1 in Figure 1) or may be distributed across multiple computing devices, such as a cloud computing device.

[0074] As shown in Figure 3A, module 200'A may receive multiple documents or dialogues 21A and may calculate word embedding vectors 220A corresponding to words in the multiple received documents 21A using the word embedding vector calculation module 220, as detailed herein (for example, with respect to Figure 3A).

[0075] In addition, or alternatively, module 200'A may use the word weight calculation module 230, as detailed herein (for example with respect to Figure 3A), to calculate word weights 230A corresponding to words in multiple received documents 21A.

[0076] According to some embodiments, module 200'A may generate or compute word embedding vectors 220A and / or word weights 230A in an offline process and store the word embedding vectors 220A and / or word weights 230A in a repository storage device 280 (e.g., storage element 6 in Figure 1). The term “offline” may be used in this context to indicate a preliminary process in which word embedding vectors 220A and / or word weights 230A are computed or prepared for words contained in a general corpus 21 of documents, which may or may not be related to the analysis of a particular document under consideration, as detailed herein. According to some embodiments, the storage repository 280 may be included in system 200'. Alternatively, as shown in Figure 3B, system 200' may be associated with or communicably connected to a storage repository 280, which may be implemented, for example, on a cloud storage service.

[0077] In a subsequent "online" text analysis process, it will be understood that module 200' (e.g., 200'B) can retrieve the prepared word embedding vector 220A and / or word weight 230A from the repository storage device 280. Then, module 200'B can input the retrieved word embedding vector 220A and / or word weight 230A into the phrase embedding vector calculation module 240 to further analyze document 21A as detailed herein.

[0078] As shown in FIG. 3A, module 200 can include a phrase embedding vector calculation module 240 (or simply module 240) adapted to calculate a weighted phrase embedding vector 240A associated with one or more (e.g., each) extracted phrases 210A based on one or more word embedding vectors 220A.

[0079] According to some embodiments, module 240 can calculate the weighted phrase embedding vector 240A as a weighted average of the word embedding vectors 220A corresponding to the words 210B of the extracted phrase 210A. The weights used in this calculation can be the respective weight values 230A of each word, such as TF-IDF score values.

[0080] For example, module 240 can calculate the weighted phrase embedding vector 240A according to Algorithm 1 below.

[0081]

Number

[0082] As shown in Algorithm 1, in the first row, the weighted phrase embedding vector 240A can be initialized as a 0 vector.

[0083] According to some embodiments, the "for each" loop is executed for each word in fragment s(210A) w i (210B) can be traversed, weighted phrase embedding vector v s (240A) can be gradually constructed or aggregated using each word 210B. In addition, or alternatively, module 200 can be implemented by one or more computing devices having multiple processing cores (e.g., computing device 1 in Figure 1). In such embodiments, module 200 is a word w of fragments s(210A). i Perform parallel processing of (210B) to create weighted phrase embedding vector v s Parallel processing of algorithm 1 can be implemented using multiple processing cores to calculate (240A).

[0084] As shown in Algorithm 1, for each word in a phrase, the word weight calculation module 230 can calculate a word weight value based on the frequency of occurrence of the word in the first set of text documents. In addition, for each word in a phrase, the phrase embedding vector calculation module 240 can calculate a weighted phrase embedding vector v s (240A) is the associated word weight value α[w i Based on 230A, the related word embedding vector v i It can be calculated as a weighted average of 220A.

[0085] In the last line of Algorithm 1, the weighted phrase embedding vector v s (240A) can be normalized according to the number of words m in the phrase s.

[0086] As shown in Figure 3A, module 200 may include a phrase splendor calculation module 260, which is adapted to calculate a phrase splendor score 260A for one or more (e.g., each) extracted phrases 210A based on a weighted phrase embedding vector 240A, as detailed herein.

[0087] According to some embodiments, module 200 may rank phrases 210A of document 21A based on how prominent they are in relation to document 21A. This prominentness may indicate the degree to which a particular phrase 210A is consistent with the main subject or topic of document 21A. Based on the assumption that prominent phrases are semantically similar to more phrases than non-prominent phrases, embodiments of the present invention may use a graph algorithm to rank phrase prominentness according to their similarity to other phrases.

[0088] As shown in Figure 3A, the phrase splendor calculation module 260 may include or maintain a phrase similarity graph 260B, which may be implemented as a data structure such as a table or linked graph, and may represent the similarity between different phrases 210A as described herein.

[0089] According to some embodiments, the phrase splendor calculation module 260 may arrange one or more phrases 210A as vertices or nodes in an undirected phrase similarity graph 260B, where each pair of vertices is connected by an edge. For one or more edges, the phrase splendor calculation module 260 may assign a similarity weight value 260C that can represent the semantic similarity between the two phrases 210A of the corresponding connected vertices in the phrase similarity graph 260B. The phrase splendor calculation module 260 may then calculate a phrase splendor score 260A for each phrase based on the similarity weight values ​​260C of the connected edges, as detailed herein.

[0090] Referring again to Figure 4, this figure is a schematic diagram showing an example of a phrase similarity graph 260B that may be included in a system for automatic topic detection within text according to several embodiments of the present invention.

[0091] As shown in Figure 4, each phrase 210A may be represented by a node or vertex in the phrase similarity graph 260B. The thickness of the edges connecting pairs of nodes may represent a similarity value or similarity weight 260C, indicating the semantic similarity (e.g., semantic similarity) between the phrases 210A of the connected nodes.

[0092] For example, as shown in the example in Figure 4, the two most similar phrases 210A are "request a replacement" and "something similar," while the two least similar phrases 210A are "what I asked for" and "something similar."

[0093] According to some embodiments, module 200 may include a phrase distance metric calculation module 250 that can calculate the similarity 260C (e.g., cosine similarity) between related phrases in a pair of nodes of a phrase similarity graph 260B according to a predetermined similarity metric, such as a cosine similarity metric, based on each phrase embedding vector 240A. For example, the phrase distance metric calculation module 250 may calculate a similarity metric value such as a cosine similarity value 260C between the word embedding vectors 220A of two phrases 210A, and the phrase splendor calculation module 260 may attribute the edges connecting the vertices of each of the two phrases 210A in the phrase similarity graph 260B as similarity weight values ​​using the cosine similarity value 260C.

[0094] In relation to the example in Figure 4, the phrase splendor calculation module 260 may receive a first phrase embedding vector 240A corresponding to the "request a replacement" phrase 210, and a second phrase embedding vector 240A corresponding to the "something similar" phrase 210. The phrase splendor calculation module 260 may work with the phrase distance metric calculation module 250 to calculate a distance metric (e.g., a cosine similarity function) on these first and second phrase embedding vectors 240A. Thus, the phrase splendor calculation module 260 may obtain a similarity value or similarity weight 260C representing the semantic similarity of each phrase.

[0095] The cosine similarity function can be defined according to Equation 2 below.

[0096]

number

[0097] According to some embodiments, the phrase splendor calculation module 260 may calculate a phrase splendor score 260A for a particular phrase 210A as a function of the similarity weights 260C of the edges connected to the vertices representing that particular phrase.

[0098] For example, the phrase splendor calculation module 260 may calculate the phrase splendor score 260A for a particular phrase 210A as the sum of the similarity weights 260C of the edges connected to the vertices representing that particular phrase. In relation to the example in Figure 4, the similarity weights 260C of the edges connected to the "request a replacement" phrase 210 could be 0.5 and 0.7, and therefore the phrase splendor calculation module 260 could assign a phrase splendor score 260A of 0.5 + 0.7 = 1.2 to the "request a replacement" phrase 210. Similarly, the similarity weights 260C of the edges connected to the "what I asked for" phrase 210 could be 5 and 2, and therefore the phrase splendor calculation module 260 could assign a phrase splendor score 260A of 0.5 + 0.2 = 0.7 to the "what I asked for" phrase 210.

[0099] In addition, or alternatively, the phrase splendor calculation module 260 may calculate the phrase splendor score 260A of one or more specific phrases 210A in an iterative process according to the following formula 3.

[0100]

number

[0101] As shown in Equation 3, a specific phrase (phrase embedding vector v i The calculation of a specific phrase splendor score WS(260A), associated with (represented by), may depend on the calculation of one or more other phrase splendor scores WS(260A).

[0102] For example, the iterative process may begin by initializing the phrase splendor score WS(260A) of each node to a value of "1". In the first iteration, the value of at least one phrase splendor score WS(260A) may be updated to be the sum of the edges (e.g., similarity weights 260C) connected to each node. Each subsequent iteration of the iterative process may include (a) calculating the phrase splendor score WS 260A of a first fragment or phrase 210A according to Equation 3 (e.g., based on the similarity weights 260C of the edges connected to each vertex, and further based on the phrase splendor score 260A of at least one second fragment 210A), and (b) updating the phrase splendor score 260A of at least one second fragment 210A based on the calculated phrase splendor score 260A of the first fragment 210A.

[0103] Referring again to the example in Figure 4, we can see that the phrase "request a replacement" 210A is the most prominent phrase because it is more central and has a significant similarity to the other two phrases or fragments, while the other phrases or fragments 210A have a significant similarity to only one of the other phrases.

[0104] According to some embodiments, the phrase splendor calculation module 260 may continue the iterative process of calculating the phrase splendor score 260A, as detailed herein (for example, in relation to Equation 3), until the convergence condition is met.

[0105] For example, convergence in the calculation of the repetition phrase splendor score can be defined as the point at which the sum of the changes in the values ​​mapped by WS between the first repetition and the second subsequent repetition falls below a predetermined threshold ε, as detailed in Equation 4 below.

[0106]

number

[0107] As shown in Figure 3, module 200 may include a phrase selection module 270 adapted to select one or more phrases 210A as prominent phrases 200A according to their respective phrase splendor scores 260A and based on predetermined selection criteria. In other words, the phrase selection module 270 may filter out phrases 210A corresponding to lower phrase splendor scores 260A based on predetermined filtering criteria.

[0108] For example, the phrase selection module 270 may select a predetermined number, a predetermined subset, or a predetermined percentage of phrases 210A corresponding to the top phrase splendor scores as splendor phrases 200A. In another example, the phrase selection module 270 may select phrases with a phrase splendor score 260A above a predetermined threshold as splendor phrases 200A. The phrase selection module 270 may discard phrases 210A that do not meet the relevant selection criteria (for example, phrases 210A corresponding to the lowest phrase splendor score 260A).

[0109] Referring now to Figure 5, this figure is a block diagram showing a relevance calculation module 300 (or simply module 300) that may be included in a system for automatic topic detection within text according to several embodiments of the present invention.

[0110] As shown in Figure 5, module 300 may include a phrase clustering module 310 adapted to cluster multiple prominent phrases 200A from multiple documents 21A and / or multiple corpora 21 into groups or clusters representing topic labels 310A according to a predetermined similarity metric.

[0111] As detailed herein (for example with respect to Figure 3A), the phrase splendor calculation module 260 obtains a plurality of phrases 210A and a plurality of phrase splendor scores for each, and may select a subset of the plurality of phrases 210A corresponding to the top phrase splendor scores, which are referred to herein as the splendor phrase 210A.

[0112] According to some embodiments, the clustering module 310 may be adapted to cluster a subset of phrases (outstanding phrases 210A) into clusters 310A based on a calculated cosine similarity value 260C, where each cluster 310A represents a unique topic label, as detailed herein.

[0113] According to some embodiments, the clustering module 310 may generate one or more topic labels (indicated as candidate topic labels 310A in Figure 5). These topic labels 310A may represent one or more respective topics of at least one text document 21A. The clustering module 310 may generate one or more topic labels 310A based on selected prominent phrases 200A (selected based on the calculated phrase prominentness score 260 as described above).

[0114] For example, one or more first documents 21A may yield a first set of prominent phrases 200A, including the prominent phrase 200A "technical assistance," and one or more second documents 21A may yield a second set of prominent phrases 200A, including the prominent phrase 200A "technical help." The phrase clustering module 310 may calculate a cosine similarity metric value 260C between pairs of prominent phrases 200A from the first set and the second set, as detailed herein (e.g., with respect to Equation 2). The phrase clustering module 310 may use the calculated similarity metric value (e.g., cosine similarity value) to cluster the prominent phrases 200A into groups or clusters, each cluster may represent a topic label. In this example, the cosine similarity metric value 260C can be expected to be high for the prominent phrases 200A "technical assistance" and "technical help," since their meanings are nearly identical. Therefore, the phrase clustering module 310 can be expected to combine these prominent phrases 200A into a single cluster.

[0115] The phrase clustering module 310 may assign or generate an appropriate topic label 310A to each cluster based on the uneven distribution of prominent phrases 200A within that cluster. In the same example, if the prominent phrase 200A "technical assistance" appears more frequently in the corpus 21 than the prominent phrase 200A "technical help", the phrase clustering module 310 may assign a topic label 310A identical to the more unevenly distributed prominent phrase 200A "technical assistance" to the common cluster.

[0116] As is known in the art, a cloud-based service providing a contact center solution may have various tenants that can be grouped into domains 20' (e.g., business domains 20') at some selected level of granularity. For example, a first domain 20' may include tenant 20 that is an insurance company, a second domain 20' may include tenant 20 that is a telecommunications company, and a third domain 20' may include tenant 20 that is a car rental agency. Embodiments of the present invention may distinguish between phrases that may be associated with a particular domain 20' and general phrases. As detailed herein, a phrase clustering module 310 may aggregate or cluster prominent phrases 200A into a plurality of candidate topic labels 310A, which may represent topics that are repeatedly mentioned in various conversations or documents 21A. As detailed herein, the relevance calculation module 300 can filter out topic labels from a plurality of candidate topic labels 310A that are either (a) not sufficiently specific or sufficiently relevant to a particular tenant 20, or (b) not correlated with a global topic or subject of interest (e.g., a topic relevant to multiple tenants 20).

[0117] The systems currently available for topic detection can estimate the relevance, or "informationality," of text within a single document under analysis by (a) calculating an n-gram language model for the background corpus, (b) calculating an n-gram language model for the foreground corpus, and (c) prioritizing n-grams that have a significantly higher probability on the foreground corpus compared to the background corpus as contributing information to the document under analysis. The term "foreground" may be used in this context to refer to a corpus of documents related to a particular tenant 20 or domain 20' of interest. Conversely, the term "background" may be used in this context to refer to a corpus of documents that may be related to a global or general tenant 20 or domain 20'.

[0118] The n-gram model can consist of multiple probability values ​​that represent the probability of a particular sequence of words appearing. These probability values ​​can be calculated, for example, based on equation 5 below.

[0119]

number

[0120] Given a preceding sequence of N-1 words, what is the word W? n The probability can be calculated by counting the number of occurrences of the subsequence (indicated by c) within the corpus, as shown in Equation 6 below.

[0121]

number

[0122] According to some embodiments, the corpus 21 of the present invention may include a diverse collection of many documents or dialogues 21A, each having its own writer, speaker, style, and jargon. Furthermore, embodiments of the present invention may aim to determine the relevance or informationality of a particular topic within multiple documents, and with respect to a particular tenant and / or business domain 20'. Embodiments of the present invention may include several improvements to currently available systems for topic discovery to address these objectives, as detailed herein.

[0123] The first such improvement may involve the use of phrases or fragments, rather than n-grams, as building blocks for topic detection, as detailed herein (for example, with respect to Figures 2 and 3A). This approach has been shown to significantly reduce the number of computations required for currently available systems by avoiding the need to compute n-gram models that include a large number of sequence probabilities (for example, as detailed in Equations 5 and 6).

[0124] Another improvement of the present invention to currently available systems for topic detection may include the extraction of prominent phrases 200A to improve the quality of phrases within documents 21A input to the relevance calculation module 300, as detailed herein (for example, with respect to Figure 2).

[0125] Another improvement of the present invention to currently available systems for topic detection may involve using a document frequency (DC) metric, e.g., the number of dialogues or documents 21A containing word sequences, instead of the number of occurrences of sequences in the corpus. Thus, embodiments of the present invention can avoid the need to calculate the probabilities of subsequences as detailed in Equations 5 and 6. In other words, embodiments of the present invention may use Equation 7 below, which is a reduced version of Equation 5, and thus can simplify the calculation of topic relevance without impairing the results of the calculation, as demonstrated herein (for example, with respect to Figure 6 below). It will be understood that such a reduction in calculation by embodiments of the present invention can facilitate practical application and improve the performance of at least one computing device (e.g., computing device 1 in Figure 1) in the natural language processing process.

[0126]

number

[0127] Another improvement of the present invention to currently available techniques for topic detection may include normalizing the effect of tenants in the background corpus so that each tenant and each domain 20' has a similar effect on the selected topic, as detailed herein.

[0128] Another improvement of the present invention to currently available systems for topic detection may include using a fuzzy search for topics within a relevant corpus of documents, rather than a strict n-gram search, as detailed herein.

[0129] According to some embodiments, the relevance calculation module 300 can calculate the relevance of candidate topic labels 310A with respect to a specific set of documents in a corpus. For example, the relevance calculation module 300 may receive documents 21A from a first set of documents in a corpus 21 relating to a first business domain 20' (e.g., car rental agencies) and documents 21A from a second set of documents in a corpus 21 relating to at least one second business domain 20' (e.g., hospitals).

[0130] The relevance calculation module 300 may calculate a foreground probability value 320A for one or more (e.g., each) candidate topic labels 310A, representing the probability that the topic label represents a topic included in the first multiple or corpus 21. In addition, the relevance calculation module 300 may calculate a background probability value for one or more (e.g., each) candidate topic labels 310A, representing the probability that the topic label represents a topic included in at least one of the first corpus 21 and the second corpus (e.g., any one of the domains). The relevance calculation module 300 may then calculate a relevance score 340A representing the relevance of the candidate topic labels to the first business domain, based on the foreground and background probability values, as detailed herein.

[0131] As shown in Figure 5, the relevance calculation module 300 may include a foreground probability calculation module 320 (or simply module 320).

[0132] According to some embodiments, module 320 may receive metadata 21B associated with one or more documents 21A (for example, via input device 7 in Figure 1). The metadata 21B may include, for example, annotations for tenants 20 and / or domains 20' associated with one or more (for example, each) documents 21A. For example, metadata 21B may be or include a data structure (e.g., a table) that can associate a particular document 21A with the respective identification of tenants 20 and / or domains 20'. In addition, module 320 may also receive from phrase clustering module 310 one or more candidate topic labels 310A associated with a particular document 21A (and, via metadata 21B, a particular tenant 20 and / or domain 20').

[0133] According to some embodiments, module 320 may use a fuzzy search module 321 adapted to perform a fuzzy word search of candidate topic labels 310A in document 21A, as is known in the art. The term “fuzzy” in this context may be used to refer to a search that allows a configurable amount of editing operations (insertion, deletion, substitution) between a desired sequence and a found sequence, with the aim of finding lexical similar word sequences, rather than strictly finding the exact occurrence of a word sequence in a document (as is done, for example, in the n-gram model). For example, a fuzzy search may ignore up to two “insertion” words to match the phrase “book me two tickets” with the desired sequence “book tickets”. In addition, a fuzzy search may ignore function words such as “the” and “is”, and may apply headwordization to words, thereby retrieving variations such as “book ticket” and “booking tickets”.

[0134] According to some embodiments, module 320 may use equation 7 to calculate the probability that one or more candidate topic labels 310A appear in documents 21A of a foreground corpus of documents related to a particular tenant 20 of interest, based on a fuzzy word search. In addition, module 320 may calculate the probability that one or more candidate topic labels 310A appear in documents 21A of a foreground corpus.

[0135] According to some embodiments, module 320 may retrieve data related to multiple tenants 20 in the same domain 20' according to the following formula 8A. Module 320 may use the fuzzy word search results of module 321 to calculate the probability that a candidate topic label 310A appears in document 21A of a corpus of documents 21 related to the foreground domain 20' (e.g., a specific domain 20' of interest). In this calculation, different amounts of documents 21 across tenants and / or time periods may be normalized accordingly.

[0136]

number

[0137] According to some embodiments, as detailed by Equation 8A, module 320 can calculate the foreground probability value P(s;FG)320A for a particular candidate topic label 310A by (a) applying a fuzzy search 321 for topic labels to a corpus 21 associated with a foreground (FG) domain 20' of interest, (b) counting the number of documents 21A in the corpus 21 that contain the relevant topic DCt(s), and (c) calculating the foreground probability value P(s;FG)320A as the ratio of DCt(s) to |t|, for example, the number of documents associated with tenant t 21. In addition, or alternatively, the foreground probability value P(s;FG)320A may be calculated as the ratio of DCt(s) to the number of documents in the foreground (FG) domain 20' of interest. P(s;FG)320A can be further normalized by Num_t_in_FG, for example, the number of tenants in the foreground domain 20' of interest.

[0138] According to some embodiments, the relevance calculation module 300 may include a background probability calculation module 330 (or simply module 330). According to some embodiments, module 330 may use a fuzzy word search module to perform a fuzzy word search on document 21A of a corpus of documents 21 related to a background domain 20' (e.g., a general domain 20' which is not limited to including only tenants 20 or domains 20' of interest).

[0139] Module 330 may use the fuzzy word search results of Module 331 to calculate the probability that candidate topic label 310A appears in document 21A of the corpus of documents 21 related to the background domain 20', according to Equation 8B below. It will be understood that this calculation may allow Module 300 to evaluate the probability that candidate topic label 310A is commonly used across various domains 20'.

[0140]

number

[0141] According to some embodiments, module 320 can calculate the background probability value P(s;BG)330A of a particular candidate topic label 310A by (a) applying a fuzzy search 331 of topic labels to a corpus 21 associated with a background (BG) domain 20', (b) counting the number of documents 21A in the corpus 21 that contain the relevant topic DCt(s), and (c) calculating the background probability value P(s;BG)330A as the ratio of DCt(s) to |t|, for example, the number of documents associated with tenant t 21. In addition, or alternatively, the background probability value P(s;BG)330A may be calculated as the ratio of DCt(s) to the number of documents in the BG domain 20'. P(s;BG) may be further normalized by Num_t_in_BG, for example, the number of tenants in the background domain 20'.

[0142] According to some embodiments, module 300 may include a global topic module 350 adapted to handle a group or list of topic labels predefined as globally important topics (indicated as 350A), separate from candidate topic labels 310A. For example, globally important topic labels 350A may refer to a list of topics that may be of interest to most or all tenants or contact centers using system 100. The list of globally important topic labels 350A may be received, for example, from a user via the input device 7 in Figure 1, and may include topics such as "supervisor" and "lawsuit," and may be modified over time (for example, by the user).

[0143] According to some embodiments, module 300 may include a domain relevance estimation module 340 adapted to estimate the relevance or informationality of one or more topic labels (e.g., 310A, 350A) to a tenant or domain of interest and to generate one or more related topic labels 300A from them.

[0144] As detailed above, for example with respect to Equation 8A, Module 320 can calculate the probability P(s;FG)320A that a candidate topic label s310A appears in a document 21A related to the foreground corpus 21. In addition, as detailed above, for example with respect to Equation 8B, Module 330 can calculate the probability P(s;BG)330A that a candidate topic label 310A appears in a document 21A related to the background corpus 21.

[0145] According to some embodiments, the domain relevance estimation module 340 may calculate a relevance score 340A for one or more (e.g., each) candidate topic labels s 310A as the distribution ratio of two calculated probabilities. For example, the relevance score 340A may be calculated according to the following equation 9.

[0146] formula 9 Relevance score 340A = P(s;FG)320A / P(s;BG)330A

[0147] The term "distribution ratio" may be used herein to mean the ratio of (a) the probability or distribution of occurrence of candidate topic label 310A in corpus 21 related to the foreground domain 20' to (b) the probability or distribution of occurrence of candidate topic label 310A in corpus 21 related to the background domain 20'.

[0148] According to some embodiments, the domain relevance estimation module 340 may compare the relevance score 340A with a predetermined threshold. If the relevance score 340A exceeds the predetermined threshold, the candidate relevance topic label 310A may be considered relevant or important in the context of the foreground domain. The candidate topic label 310A may then be included in the output relevance topic label 300A.

[0149] Alternatively, if the relevance score 340A does not exceed a predetermined threshold, the related candidate topic label 310A may be flagged as a candidate for removal. According to some embodiments, under this condition, the global topic processing module 350 may use the phrase distance metric (e.g., cosine similarity) calculation module 250 to determine the proximity of the flagged candidate topic label 310A to one or more global important topic labels 350A, as detailed herein (e.g., with respect to Figure 3A). For each of the flagged labels, if the calculated distance to the global important topic exceeds a predetermined threshold (e.g., the cosine similarity 260C is less than the predetermined threshold), the related candidate topic label 310A may be discarded as irrelevant or of low importance in the context of the foreground domain 20'.

[0150] Referring now to Figure 6, this is a graph illustrating an example of improved relevant topic identification according to an embodiment of the present invention, as shown in the example in Figure 6. The orange plots represent the performance of a topic detection system according to an embodiment of the present invention, which (a) uses phrase filtering by module 200 to obtain prominent phrases 200A as detailed herein (for example, with respect to Figure 3A), and (b) uses domain relevance estimation module 340 to estimate the relevance or informationality of one or more candidate topic labels 310A to a tenant 20 or domain 20' of interest. The blue plots represent the performance of a topic detection system according to an embodiment of the present invention, which may use phrase filtering by module 200 but does not use domain relevance estimation module 340. The green plots represent the performance of a topic detection system according to an embodiment of the present invention, which does not use phrase filtering by module 200 and does not use domain relevance estimation module 340.

[0151] As illustrated in the example in Figure 6, “Yield” may represent the number of relevant topic labels (e.g., element 300A in Figure 5) that can be obtained by embodiments of the present invention. “Accuracy” may represent the percentage of truly positive topic labels from the total number of generated topic labels 300A, as illustrated in the example in Figure 6.

[0152] As shown by the dashed line in the example in Figure 6, the three plots represent the level of accuracy for a given number of related topic labels.

[0153] It will be understood (by comparing the blue plot and the green plot) that filtering phrases by module 200 to identify prominent phrases 200A as a preliminary step before calculating topic relevance can improve the quality (e.g., accuracy) of the identified topic labels 300A. In other words, embodiments of the present invention may include improvements to currently available systems for topic identification by identifying prominent phrases 200A and using prominent phrases 200A (e.g., instead of n-grams and / or general phrases) for topic detection.

[0154] In addition, it will be understood (by comparing the blue plot and the orange plot) that filtering candidate topic labels according to their relevance in the context of a particular domain (for example, according to relevance score 340A) can improve the quality (e.g., accuracy) of the identified topic labels 300A. In other words, embodiments of the present invention may include improvements to currently available systems for topic identification using tenant 20-specific and / or domain 20'-specific filtering of candidate topic labels, as detailed herein (for example, in relation to Figure 5).

[0155] Referring now to Figure 7, which is a block diagram, shows an example of an application module 400 that may be included in or associated with a system for automatic topic detection in text (e.g., system 100 in Figure 2) according to several embodiments of the present invention. According to some embodiments, the application module 400 may be implemented on the same computing device as system 100 in Figure 2 (e.g., element 1 in Figure 1). In addition, or alternatively, the application module 400 may be implemented on a computing device separate from system 100 and may be communicably connected to system 100 via a computer network (e.g., the Internet).

[0156] According to some embodiments, the application module 400 may receive one or more documents 21A and corresponding metadata 21B from, for example, the input device 7 in Figure 1. In addition, the application module 400 may receive one or more related topic labels 300A associated with the documents 21A and metadata 21B (for example, from the system 100), as detailed herein. The application module 400 may store the received data as a topic data structure 400B (for example, a list, a table in a database, etc.) (for example, on the storage device 6 in Figure 1).

[0157] According to some embodiments, the application module 400 may receive at least one query 31B via a user interface (UI), such as the input device 7 in Figure 1. The query 31B may, for example, relate to a topic that may or may not be present in one or more received documents 21A.

[0158] According to some embodiments, the application module 400 may subsequently generate a response 400A to a query based on one or more relevant topic labels 300A.

[0159] In other words, as detailed herein (for example with respect to Figure 5), module 300 may generate a related topic label 300A by selecting one or more candidate topic labels 310A as relevant based on the relevance score 340A. Thus, application module 400 may generate a response to a query that may be relevant to one or more topics contained in a document or dialogue 21A, based on one or more candidate topic labels 310A and the corresponding relevance score 340A.

[0160] For example, query 31B might inquire whether a particular topic is included in a particular document 21A, and query response 400A might contain the answer to that query 31B (e.g., yes / no). In another example, query 31B might inquire which of the documents 21A contain one or more topics from a list of topics, and query response 400A might contain the answer to query 31B (e.g., a list of corresponding documents).

[0161] Referring now to Figure 8, this figure is a flowchart illustrating a method for automatic topic detection within text using at least one processor (for example, processor 2 in Figure 1) according to several embodiments of the present invention.

[0162] As shown in step S1005, at least one processor 2 may receive at least one text document (e.g., document 21A in Figure 2) from among multiple or a corpus (e.g., corpus 21 in Figure 2).

[0163] As shown in step S1010, at least one processor 2 may work with a phrase extraction module (e.g., the phrase extraction module 210 in Figure 3A) to extract one or more phrases (e.g., phrase 210A in Figure 3A) from at least one text document. As detailed herein (e.g. with respect to Figure 3A), the phrase extraction module 210 may extract phrases 210A based on one or more syntactic patterns 31A, each phrase 210A may contain one or more words 210B.

[0164] As shown in step S1015, for each phrase 210A, at least one processor 2 may apply a word embedding NN (e.g., the word embedding vector calculation module 220 in Figure 3A) to one or more words 210B of one or more phrases 210A to obtain one or more word embedding vectors (e.g., elements 220A in Figure 3A).

[0165] As shown in steps S1020 and S1025, for each phrase 210A, at least one processor 2 may subsequently calculate a weighted phrase embedding vector (e.g., element 240A in Figure 3A) based on one or more word embedding vectors 220A, as detailed herein (e.g., with respect to Figure 3A), and calculate a phrase splendor score 260A based on the weighted phrase embedding vector 240A.

[0166] As shown in step S1030, at least one processor 2 may generate one or more topic labels (e.g., related topic labels 300A in Figure 5) representing one or more respective topics of at least one first text document, based on the calculated phrase splendor score 260A, as detailed herein (e.g., with respect to Figures 3A and 5).

[0167] As detailed herein, embodiments of the present invention may include practical applications that perform automated text analysis, for example, including determining or identifying relevant topics within documents and / or text dialogues. For example, embodiments of the present invention may be configured to receive at least one query (e.g., query 31B in Figure 7) (e.g., via a UI) with respect to a topic presenting a plurality of documents, as detailed herein (e.g., with respect to Figure 7), and to generate a response to the query based on one or more relevant topic labels 300A.

[0168] Embodiments of the present invention include several improvements to the technology of currently available systems and methods of text analysis, which can facilitate automated identification of topics within text documents in a faster and more accurate manner than currently available systems, as detailed herein (for example, with respect to Figure 6).

[0169] Unless expressly stated otherwise, the embodiments of the methods described herein are not restricted to any particular order or sequence. Furthermore, all formulas described herein are intended only as examples, and other or different formulas may be used. In addition, some of the embodiments or elements thereof of the methods described herein may occur or be performed at the same time.

[0170] While certain features of the present invention have been illustrated and described herein, many modifications, substitutions, alterations, and equivalents can be conceived by those skilled in the art. It should be understood that the appended claims are intended to encompass all such modifications and alterations that fall within the true spirit of the invention.

[0171] Various embodiments have been presented. Each of these embodiments may, of course, include features from other embodiments presented, and embodiments not specifically described may include various features described herein.

Claims

1. A method for automatic topic detection in text using at least one processor, Receiving at least one first text document from a set of first text documents relating to a first business domain, Extracting one or more phrases from the first text document based on one or more syntactic patterns, wherein each phrase contains one or more words, For each phrase, A word embedding neural network (NN) is applied to one or more words in the aforementioned phrase to obtain one or more word embedding vectors for each word. Based on the one or more word embedding vectors mentioned above, a weighted phrase embedding vector is calculated. The calculation of the phrase splendor score based on the weighted phrase embedding vector, wherein the calculation of the phrase splendor score is The arrangement of one or more phrases as vertices in an undirected phrase graph, wherein each pair of vertices is connected by an edge, For one or more edges, calculate similarity weights that represent the semantic similarity between two phrases at corresponding connected vertices. The semantic similarity between two phrases is represented by a similarity weight calculated by determining the cosine similarity value between the word embedding vectors of the two phrases, and assigning the cosine similarity value as a similarity weight to the edge connecting the vertices of the two phrases, and This includes, for each phrase, calculating a phrase splendor score based on the similarity weights of the connected edges, Based on the calculated phrase splendor score, generate one or more topic labels representing one or more topics of the at least one first text document, wherein generating the one or more topic labels is Obtaining multiple phrases and their respective phrase splendor scores from the first multiple text documents, Selecting a first subset of the multiple phrases corresponding to the top phrase splendor score, and This includes clustering a subset of the phrases into clusters based on the calculated cosine similarity values, wherein each cluster represents a unique topic label. Obtaining a second set of documents related to at least one second business domain, For each topic label, A foreground probability value is calculated that represents the probability that the topic label represents a topic included in the first plurality of text documents. A background probability value is calculated that represents the probability that the topic label represents a topic included in one of the first plurality of text documents and the second plurality of documents. A method comprising calculating a relevance score that represents the relevance of the topic label to the first business domain, based on the foreground probability value and the background probability value.

2. The method according to claim 1, wherein the one or more word embedding vectors are adapted to encode the meaning of the one or more corresponding words in the phrase.

3. Calculating weighted phrase embedding vectors is possible. For each word in the aforementioned phrase, a word weight value is calculated based on the frequency of occurrence of the word in the first plurality of text documents. The method according to claim 1, comprising calculating the weighted phrase embedding vector as a weighted average of the related word embedding vectors based on the related word weight values.

4. The phrase spleness score is calculated iteratively, and each iteration is performed as follows: (a) Calculating a phrase splendor score for the first phrase based on the similarity weights of the edges connected to each of the vertices and (b) the phrase splendor score of at least one second phrase, The method according to claim 1, further comprising updating the phrase splendor score of at least one second phrase based on the calculated phrase splendor score of the first phrase.

5. Calculating the foreground probability value is Perform a fuzzy search of the topic labels to count the number N1 of documents among the first set of text documents that contain the relevant topic, The method according to claim 1, comprising calculating the foreground probability value as the ratio of N1 to the number of documents included in the first business domain.

6. Calculating background probability values ​​is Perform a fuzzy search of the topic labels to count the number N2 of documents among the first and second sets of text documents that contain the relevant topic, The method according to claim 1, comprising calculating the background probability value as the ratio of N2 to the number of documents included in the first business domain and the at least one second business domain.

7. The system receives at least one query regarding a topic present in the first plurality of text documents via a user interface (UI), The method according to claim 1, further comprising generating a response to the query based on one or more topic labels.

8. The system receives at least one query regarding a topic present in the first plurality of text documents via a user interface (UI), The method according to claim 1, further comprising generating a response to the query based on one or more topic labels and corresponding relevance scores.

9. A system for automatic topic detection within text, comprising: a non-temporary memory device storing instruction code modules; and at least one processor associated with the memory device and configured to execute the instruction code modules, wherein when the instruction code modules are executed, the at least one processor: Receiving the first text document among a first set of text documents related to the first business domain, Extracting one or more phrases from the first text document based on one or more syntactic patterns, wherein each phrase contains one or more words, For each phrase, A word embedding neural network (NN) is applied to one or more words in the aforementioned phrase to obtain one or more word embedding vectors for each word. Based on the one or more word embedding vectors mentioned above, a weighted phrase embedding vector is calculated. Based on the aforementioned weighted phrase embedding vector, calculate the phrase splendor score, Based on the calculated phrase splendor score for each phrase, one or more topic labels are generated representing one or more topics in the first text document. Obtaining a second set of documents related to at least one second business domain, For each topic label, A foreground probability value is calculated that represents the probability that the topic label represents a topic included in the first plurality of text documents. A background probability value is calculated that represents the probability that the topic label represents a topic included in one of the first plurality of text documents and the second plurality of documents. Based on the foreground probability value and the background probability value, a relevance score is calculated that represents the relevance of the topic label to the first business domain. A system configured to perform the following actions.

10. The aforementioned at least one processor processes the weighted phrase embedding vector, For each word in the aforementioned phrase, a word weight value is calculated based on the frequency of occurrence of the word in the first plurality of text documents. The system according to claim 9, configured to calculate the weighted phrase embedding vector as a weighted average of the related word embedding vectors based on the related word weight values.

11. The at least one processor determines the phrase splendor score, The arrangement of one or more phrases as vertices in an undirected phrase graph, wherein each pair of vertices is connected by an edge, For one or more edges, calculate similarity weights that represent the semantic similarity between two phrases at corresponding connected vertices, The system according to claim 9, configured to calculate a phrase splendor score for each phrase based on the similarity weights of the connected edges.

12. The at least one processor determines a similarity weight representing the semantic similarity between two phrases, The cosine similarity value between the word embedding vectors of the two phrases mentioned above is calculated, The system according to claim 11, configured to assign the cosine similarity value as a similarity weight to the edge connecting the vertices of the two phrases.

13. The aforementioned at least one processor determines the foreground probability value Perform a fuzzy search of the topic labels to count the number N1 of documents among the first set of text documents that contain the relevant topic, The method according to claim 9, wherein the foreground probability value is calculated by the ratio of N1 to the number of documents included in the first business domain.

14. The aforementioned at least one processor determines the background probability value Perform a fuzzy search of the topic labels to count the number N2 of documents among the first and second sets of text documents that contain the relevant topic, The method according to claim 9, wherein the background probability value is calculated by the ratio of N2 to the number of documents included in the first business domain and the at least one second business domain.

15. The aforementioned at least one processor, The system receives at least one query regarding a topic present in the first set of text documents via a UI, The method according to claim 9, configured to generate a response to the query based on one or more topic labels and corresponding relevance scores.

Images