Method and system for searching and retrieving information

Abstract

Methods and systems for searching and retrieving information. In one aspect, there is a method of retrieving information using a knowledge base. The method includes receiving a search query input by a user and identifying, using a first model, a category corresponding to the received search query. The method also includes identifying, based on the received search query, a loss function of the first model, and an objective function of a second model, T topics corresponding to the received search query and performing a search for the received search query only on a portion of the knowledge base associated with the identified category and / or the identified topics. The method also includes retrieving one or more files associated with the identified category and / or the identified topics.

Description

TECHNICAL FIELD

[0001] Embodiments related to methods and systems for searching and retrieving information are disclosed. BACKGROUND

[0002] Efficiently handling service engineers (domain experts) time is a huge challenge for managed services. Most service industries are trying to reduce manpower and replace it with intelligent robots. This trend will lead to a decrease in the number of available service engineers. Further, there can be situations where service engineers are far away from the location where a task needs to be performed. In such situations, the service engineer’s time is wasted when the service engineer travels to the location where the task needs to be performed.

[0003] Further, since a field service operator (FSO) typically needs to search and retrieve files required to perform a given task (e.g., by using a search engine), it is desirable to provide the FSO with the most relevant files in a timely manner to perform a given task (e.g., repair and installation) to reduce the time required to perform the given task. Providing the FSO with information that is not relevant to the given task can frustrate the FSO and increase the time required to perform the given task. This delay can also prevent the FSO from performing other tasks that are required at different locations. Therefore, there is a need for improving methods for searching and retrieving information. SUMMARY

[0004] Generally, performing a search using a search engine involves retrieving information and displaying search results that identify the retrieved information. To retrieve relevant information, a knowledge base can be used. However, as the search space increases with the amount of available information, the computational complexity of performing a search using a knowledge base becomes higher. In the related art, to reduce this computational complexity, a particular search method called elastic search is used. However, as the amount of information that needs to be searched further increases, performing a search using an elastic search scheme becomes insufficient to reduce the computational complexity.

[0005] Therefore, in some embodiments, a search across a knowledge base is performed using a combination of information classification and topic modeling, thereby reducing the computational complexity of performing the search.

[0006] For example, after obtaining a collection of files (e.g., a collection of service manuals and / or installation instructions), a classification model (e.g., a machine learning classification model) is used to classify each file based on the content of each file. After classifying the obtained files, a topic model (e.g., a natural language processing (NLP) model) is used to obtain the words and context (i.e., topics) of the files. Both the classification model and the topic model are interrelated to perform operations to speed up the search process. Therefore, embodiments of the present disclosure provide a way to quickly retrieve files required by a FSO to perform a given task in real-time, so that the FSO can efficiently handle the given task.

[0007] As described above, some embodiments of the present disclosure enable FSOs to perform a given task efficiently by allowing the FSOs to obtain information that is required or helpful to perform the given task in an efficient manner. Currently, most search tools for searching information use Elasticsearch as a backend. Elasticsearch is based on keyword matching. However, using a knowledge base can help simplify the search process. A knowledge base adds more semantic information to a file by constructing a topology-based graph. However, employing knowledge graph-based search involves a lot of manual work.

[0008] For example, a user has to extract keywords and / or key phrases from a file and perform part-of-speech (POS) tagging and named entity recognition (NER) on the extracted keywords and / or key phrases. The user then needs to arrange them into a knowledge base structure. The size of the obtained knowledge base depends on the size of the file. As an example, web-based search engines use a large number of files for searching. However, if a knowledge base is created for all the files, such creation would occupy a large amount of memory, and the number of files searched for the desired output can be too large and thus can require a long time to complete the search. Therefore, in some embodiments of the present disclosure, a technique for limiting the time required to perform a search using a knowledge base is provided.

[0009] According to some embodiments, a method of retrieving information using a knowledge base is provided. The method includes receiving a search query input by a user, and identifying, based on the received search query, a category corresponding to the received search query using a first model. One or more files can be assigned to the identified category, and the first model can be a classification model whose function is to map an input to one of M different categories, where M is greater than 1. The method further includes identifying T topics corresponding to the received search query based on (i) the received search query, (ii) a loss function of the first model, and (iii) an objective function of a second model, where T is greater than or equal to 1. The method further includes performing a search for the received search query only on a portion of the knowledge base associated with the identified category and / or the identified topics using the identified category and the identified topics. The method further includes retrieving one or more files associated with the identified category and / or the identified topics based on the performed search.

[0010] According to some embodiments, a method for building a knowledge base is provided. The method includes obtaining a set of N files, where each file included in the set of files is assigned to one of M different categories, where N and M are greater than 1. The method further includes identifying a set of T topics based on (i) the content of the N files, (ii) a loss function of a first model, and (iii) an objective function of a second model, where T is greater than 1 and each topic is a group of one or more keywords. The method further includes generating a knowledge base using the identified topics, and for each of the N files, adding the file to the knowledge base based on the particular category to which the file is assigned and the keywords included in the file. The first model is a classification model whose function is to map an input sentence to one of the M categories.

[0011] In another aspect, an apparatus adapted to perform any of the methods disclosed herein is provided. In some embodiments, the apparatus includes processing circuitry and a memory storing instructions that, when executed by the processing circuitry, cause the apparatus to perform any of the methods disclosed herein. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which are incorporated in and form a part of the specification, illustrate various embodiments.

[0013] Figure 1 An example knowledge base is shown.

[0014] Figure 2 An example knowledge base according to some embodiments is shown.

[0015] Figure 3 is a process according to some embodiments.

[0016] Figure 4 An example knowledge base according to some embodiments is shown.

[0017] Figure 5 is a partial process according to some embodiments.

[0018] Figure 6 is a partial process according to some embodiments.

[0019] Figure 7 is a process according to some embodiments.

[0020] Figure 8 is a process according to some embodiments.

[0021] Figure 9 An apparatus according to some embodiments is shown. DETAILED DESCRIPTION

[0022] Figure 1A portion of an example knowledge base 120 is shown, in the form of a knowledge graph.

[0023] The knowledge graph 120 includes a top node 122 in a first layer of the knowledge graph 120 and middle nodes 124, 126, and 128 in a second layer of the knowledge graph 120. To perform a search on the knowledge graph 120, a search must be performed on the entire knowledge graph 120. However, a search of the entire knowledge graph 120 requires a longer time period.

[0024] Accordingly, in some embodiments, both classification and topic modeling are used so that a search need only be performed on a portion of the knowledge graph rather than the entire knowledge graph.

[0025] For classification, a model based on domain knowledge (e.g., a hierarchy) or artificial intelligence (AI) can be used. For example, a convolutional neural network (CNN) model can be used to classify files based on an input search query. As used herein, a “file” is a collection of data that is processed as a unit.

[0026] For topic modeling, a latent Dirichlet allocation (LDA) model can be used to identify the main topics in a file.

[0027] Figure 2 A portion of an example knowledge graph 220 is shown, in accordance with some embodiments. In comparison to the knowledge graph 120, a search is easier to perform on the knowledge graph 220 because for each keyword (or topic), a category (or context) is given. Accordingly, if the context of a user’s search query can be identified, then a search need only be performed on a portion of the knowledge graph rather than the entire portion of the knowledge graph. This makes the search faster and can result in more effective results. This is different from a named entity recognition (NER) model because the NER model can only identify existing sentences available in, for example, Wikipedia, and new models are needed based on domain words. Moreover, from a search query, it is difficult to identify the NER of the search query because it will be very small. Accordingly, in some embodiments, a classification model constructed from files is used to perform classification of a search query.

[0028] When using the LDA model to identify topics in a file, a loss function of the LDA model is used to find a distribution of words associated with each topic so that the distribution of words is uniform. A problem with using the loss function of the LDA model is that it is unsupervised and, as a result, can produce poor results. Moreover, due to text being noisy, employing a classifier (i.e., categorizer) can result in poor results. Accordingly, in some embodiments, the loss function (i.e., objective function) of the LDA model is modified by adding a loss function of a classifier (i.e., categorizer) to the loss function of the LDA model.

[0029] An example loss function of the LDA model is where d corresponds to a document, N is the total number of available documents, n e N d represents words included in each document, is a probability distribution of document topic distribution, and is a random parameter that affects the distribution of words in each topic. The LDA model is used to find words in each topic such that the distribution is uniform across all topics. However, this process is unsupervised and requires information of the number of topics to be input to the document.

[0030] Accordingly, according to some embodiments, a loss function of the LDA model is modified such that the modified loss function of the LDA model is based on a loss function of the classifier and the loss function of the LDA model. For example, the modified loss function of the LDA model is where y d is an actual class of a document to be input to the classifier (i.e., a predefined class of the document), and is a predicted class determined by the classifier. By considering a two-norm of a difference between the predefined class of the document and the predicted class of the document determined by the classifier, the LDA model can extract more meaningful topics from the document and thus can improve accuracy of the LDA model.

[0031] Figure 3 A process 300 of constructing a knowledge base (e.g., a knowledge graph) according to some embodiments is shown. The process 300 can start from step s302.

[0032] In step s302, all documents in a database that needs to be searched are obtained.

[0033] After obtaining the documents, in step s304, each of the obtained documents is classified and labeled with one or more classes. For example, a document used by a service engineer for managing a wireless network device can be labeled with classes—“installation” and “troubleshooting”. Since sentences included in the document are likely to be related to one or more classes of the document, each sentence included in the document can also be classified according to the one or more classes of the document.

[0034] After classifying and labeling the documents, in step s306, keywords and / or key phrases are extracted from the documents using a character recognition engine (e.g., a Tesseract optical character recognition (OCR) engine), and each document is divided based on sentences included in each document. Each of the extracted key phrases can be identified as a single word by connecting multiple words included in each key phrase with a hyphen, a dash, or an underscore (e.g., solving_no_connection_problem).

[0035] In step s308, a classification model is built. The classification model can be configured to receive one or more sentences as input and output one or more categories associated with the input sentences as output. Since search queries are typically in the form of sentences, the input to the classification model is set to be in the form of sentences (rather than words or paragraphs). In some embodiments, a CNN model can be used as the classification model.

[0036] In step s310, topic modeling is performed on files of the same category and the main keywords that form the topics in the files are identified. In some embodiments, an LDA model can be used to perform the topic modeling.

[0037] After identifying (i) the categories of the files and (ii) the topics associated with each category of the files, a knowledge base is built in step s312. In the knowledge base, each category identified in step s304 can be assigned to a node in the top layer of the knowledge base (hereinafter referred to as “top node”), and the topics associated with each category of the files can be assigned to nodes in the middle layer branching out from the top node (hereinafter referred to as “middle node”). Figure 4 An exemplary knowledge graph 400 built as a result of performing step s312 is shown.

[0038] As shown in Figure 4 The knowledge graph 400 includes top nodes 402 and 404. Each of the top nodes 402 and 404 is associated with a category - “installation” or “troubleshooting”. The knowledge base 400 also includes middle nodes 406, 408, 410, and 412 branching out from the top nodes 402 and 404. Each of the middle nodes 406, 408, 410, and 412 corresponds to a topic associated with at least one of the categories. For example, the middle node 408 corresponds to the topic (or keyword, key phrase) - “no connection” - and is associated with the categories - “installation” and “troubleshooting”.

[0039] After building the knowledge base in step s312, in step s314, nodes corresponding to the names of the files are added to a lower layer of the knowledge base. The nodes in the lower layer (hereinafter referred to as “lower nodes”) are associated with one or more topics in the middle layer of the knowledge base and branch out from the associated topics. For example, in the knowledge graph 400, the node 414 corresponds to the file name - “file 1” - and branches out from the nodes 406 and 410, which correspond to the topics - “low power” and “poor signal” - associated with “file 1”.

[0040] In some embodiments, after performing topic modeling in step s310, two additional steps can be performed before building the knowledge base in step s312. Specifically, as shown inFigure 5 As shown, after performing topic modeling in step s310, part-of-speech (POS) tagging can be performed in step s502. For example, after identifying topics in step s310 in the topic modeling, keywords associated with each of the identified topics can be tagged as a noun or a verb based on the position of the word within the topic.

[0041] After performing POS tagging, in step s504, NER building can be performed. In the NER building step, one or more words included in the obtained file are tagged with what the word represents. For example, the word "London" can be tagged as "capital" and the word "France" can be tagged as "country."

[0042] After performing NER building in step s504, a knowledge base can be built in step s312.

[0043] Figure 6 A process 600 of performing a search on a knowledge base according to some embodiments is shown. The process 600 can start from step s602.

[0044] In step s602, a search query is received at a user interface. The user interface can be any device capable of receiving user input. For example, the user interface can be a mouse, a keyboard, a touch panel, and a touch screen.

[0045] After receiving the search query, in step s604, one or more sentences corresponding to the search query are provided as input to a classification model such that the classification model identifies one or more categories associated with the search query. The classification model used in this step can correspond to the classification model established in step s408.

[0046] After identifying one or more categories associated with the search query, in step s606, a topic model identifies one or more topics associated with the search query based on one or more keywords of the search query. The topic model used in this step can correspond to the entity that performs topic modeling in step s310.

[0047] Based on the identified categories and topics associated with the search query, in step s608, a search is performed on only the portion of the knowledge base that relates to the identified categories and the identified topics, rather than the entire knowledge base. By performing a search on only the portion of the knowledge base that is most likely relevant to the user's search query, the files relevant to the search query can be retrieved more quickly.

[0048] Figure 7 A flowchart showing a process 700 for retrieving information using a knowledge base is shown. The process 700 can start from step s702.

[0049] Step s702 includes receiving a search query input by a user.

[0050] Step s704 includes identifying, using a first model, a category corresponding to the received search query based on the received search query. One or more files can be assigned to the identified category, and the first model can be a classification model whose function is to map an input to one of M different categories, where M is greater than 1.

[0051] Step s706 includes identifying, based on (i) the received search query, (ii) a loss function of the first model, and (iii) an objective function of a second model, T topics corresponding to the received search query, where T is greater than or equal to 1.

[0052] Step s708 includes performing a search for the received search query only against a portion of the knowledge base that is associated with the identified category and / or the identified topics, using the identified category and the identified topics.

[0053] Step s710 includes retrieving, based on the performed search, one or more files that are associated with the identified category and / or the identified topics.

[0054] In some embodiments, process 700 can further include constructing the knowledge base. Constructing the knowledge base can include obtaining a set of N files, each file in the set being assigned to one of M different categories, where N is greater than 1. Constructing the knowledge base can further include identifying a set of topics based on (i) contents of the N files, (ii) a loss function of the first model, and (iii) an objective function of the second model, where each topic is a group of one or more keywords. Constructing the knowledge base can further include generating the knowledge base using the identified topics, and for each of the N files, adding the file to the knowledge base based on a particular category to which the file is assigned and keywords included in the file.

[0055] Figure 8 is a flowchart illustrating a process 800 for constructing a knowledge base. Process 800 can begin with step s802.

[0056] Step s802 includes obtaining a set of N files, each file in the set being assigned to one of M different categories, where N and M are greater than 1.

[0057] Step s804 includes identifying a set of T topics based on (i) contents of the N files, (ii) a loss function of the first model, and (iii) an objective function of the second model, where T is greater than 1, and each topic is a group of one or more keywords.

[0058] Step s806 includes generating a knowledge base using the identified topics.

[0059] Step s808 includes, for each of the N files, adding the file to the knowledge base based on the particular category to which the file is assigned and the keywords included in the file.

[0060] The first model can be a classification model whose function is to map an input sentence to one of M categories.

[0061] In some embodiments, the classification model is a machine learning (ML) model. Process 800 can further train the ML model using the files that are classified as training data.

[0062] In some embodiments, identifying the set of T topics includes identifying the set of one or more keywords for each topic using a sum of a loss function of the first model and an objective function of the second model.

[0063] In some embodiments, the loss function of the first model depends at least on a probability distribution of each topic in the set of T topics and a random parameter that influences a distribution of words in each topic in the set of T topics.

[0064] In some embodiments, the objective function of the second model depends at least on a predetermined category of the file and a predicted output of the first model.

[0065] In some embodiments, the second model is a latent Dirichlet allocation (LDA) model.

[0066] In some embodiments, process 800 includes performing POS tagging on the keywords associated with the identified set of T topics.

[0067] Figure 9 is a block diagram of an apparatus 900 for performing the methods disclosed herein, in accordance with some embodiments. As such, the apparatus 900 can comprise a processor 902, a memory 904, and a transceiver 906. Figure 9As shown, the apparatus 900 can include processing circuitry (PC) 902, which can include one or more processors (P) 955 (e.g., a general -purpose microprocessor, and / or one or more other processors, such as an application specific integrated circuit (ASIC), field programmable gate array (FPGA), and / or the like), which can be collectively located in a single housing or in a single data center, or which can be geographically distributed (i.e., the apparatus 900 can be a distributed computing apparatus); at least one network interface 948, including a transmitter (Tx) 945 and a receiver (Rx) 947, for enabling the apparatus 900 to transmit data to, and receive data from, other nodes connected to a network 110 (e.g., an Internet Protocol (IP) network), the network interface 948 being connected (directly or indirectly) to the network 110 (e.g., the network interface 948 can be wirelessly connected to the network 110, in which case the network interface 948 is connected to an antenna arrangement); and a storage unit (also referred to as a "data storage system") 908, which can include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments in which the PC 902 includes a programmable processor, a computer program product (CPP) 941 can be provided. The CPP 941 includes a computer readable medium (CRM) 942 storing a computer program (CP) 943 comprising computer readable instructions (CRI) 944. The CRM 942 can be a non-transitory computer readable medium, such as magnetic

[0068] While various embodiments have been described herein, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of the present disclosure should not be limited by any of the above described exemplary embodiments, but should be defined in accordance with the following claims and their equivalents.

[0069] Additionally, although the processes are described above and illustrated in the drawings as a series of steps, it is to be understood that not all of the steps are necessarily required to be performed in the particular order described or illustrated. Thus, some steps can be performed in an order other than that which is described or illustrated.

Claims

1. A method (700) of retrieving information using a knowledge base, the method comprising: receiving (s702), through a user interface, a search query entered by a user; identifying (s704), using a first model by processing circuitry, a category corresponding to the received search query based on the received search query, wherein one or more files are assigned to the identified category, and wherein the first model is a classification model whose function is to map an input to one of M different categories, where M is greater than 1; identifying (s706) a set of T topics corresponding to the received search query based on the received search query, a loss function of the first model, and an objective function of a second model, wherein T is greater than or equal to 1, wherein the identified set of T topics is associated with each of the categories of the one or more files, and wherein the one or more files are assigned to the identified set of T topics; performing (s708) a search for the received search query only on a portion of the knowledge base associated with the identified category and / or the identified set of T topics using the identified category and the identified set of T topics; and retrieving (s710) one or more files associated with the identified category and / or the identified set of T topics based on the performed search, wherein the method further comprises building the knowledge base, wherein building the knowledge base comprises: obtaining, by processing circuitry, a set of N files, wherein each file included in the set of files is assigned to one of the M different categories, wherein N is greater than 1; identifying a set of topics based on contents of the N files, a loss function of the first model, and an objective function of the second model, wherein each topic is a group of one or more keywords; generating, by processing circuitry, the knowledge base using the identified set of topics; and for each of the N files, adding the file to the knowledge base based on a particular category to which the file is assigned and keywords included in the file; wherein identifying the set of T topics comprises identifying the group of one or more keywords for each topic using a sum of the loss function of the first model and the objective function of the second model, and wherein the second model is a Latent Dirichlet Allocation, LDA, model.

2. The method of claim 1, wherein: the classification model is a machine learning, ML, model, and the method further comprises training the ML model using the classified files as training data.

3. The method of claim 1, wherein: the loss function of the first model depends at least on a probability distribution of each topic in the set of T topics and a random parameter that affects a distribution of words in each topic in the set of T topics.

4. The method of claim 1, wherein: the objective function of the second model depends at least on a predetermined category of a file and a predicted output of the first model.

5. The method of claim 1, further comprising: performing part-of-speech (POS) tagging on keywords associated with the identified set of T topics.

6. An apparatus (900) for retrieving information using a knowledge base, the apparatus being adapted to: receive (s702), through a user interface, a search query input by a user; Based on the received search query, a first model is used (s704) to identify a category corresponding to the received search query, wherein, one or more files are assigned to the identified categories, and wherein the first model is a classification model, a function of which is to map an input to one of M different categories, where M is greater than 1; identify (s706), based on the received search query, a loss function of the first model, and an objective function of a second model, a set of T topics corresponding to the received search query, where T is greater than or equal to 1, wherein the identified set of T topics is associated with each of the categories of the one or more files, and wherein the one or more files are assigned to the identified set of T topics; perform (s708), using the identified categories and the identified set of T topics, a search for the received search query only on a portion of the knowledge base associated with the identified categories and / or the identified set of T topics; and retrieve (s710), based on the performed search, one or more files associated with the identified categories and / or the identified set of T topics; the apparatus is further adapted to build the knowledge base, wherein building the knowledge base comprises: obtaining, by processing circuitry, a set of N files, wherein each file included in the set of files is assigned to one of the M different categories, where N is greater than 1; identifying, based on contents of the N files, a loss function of the first model, and an objective function of the second model, a set of topics, where each topic is a group of one or more keywords; generating, by processing circuitry, the knowledge base using the identified set of topics; and for each of the N files, adding the file to the knowledge base based on a particular category to which the file is assigned and keywords included in the file; wherein identifying the set of T topics comprises identifying the group of one or more keywords for each topic using a sum of the loss function of the first model and the objective function of the second model, and wherein the second model is a Latent Dirichlet Allocation (LDA) model.

7. The apparatus of claim 6, wherein: the classification model is a machine learning (ML) model, and the apparatus is adapted to train the ML model using the classified files as training data.

8. The apparatus of claim 6, wherein: the loss function of the first model depends at least on a probability distribution of each topic in the set of T topics and a random parameter affecting a distribution of words in each topic in the set of T topics.

9. The apparatus of claim 6, wherein: the objective function of the second model depends at least on a predetermined category of a file and a predicted output of the first model.

10. The apparatus of claim 6, further comprising: performing part-of-speech (POS) tagging on the keywords associated with the identified set of T topics.

11. A non-transitory computer-readable medium having stored thereon computer- executable instructions that, when executed by a processing circuit (902), cause the processing circuit to perform the method of any of claims 1-5.

Images