Signature keyword generation and timeline-based search system for document retrieval

Abstract

The present invention relates to a signature keyword generation and timeline-based search system for document search that improves search efficiency and accuracy by generating a signature keyword composed of words that can represent the content of a document and generating a timeline based on the date of creation during document search.

Description

Technology Field

[0001] The present invention relates to a signature keyword generation and timeline-based search system for document search, and more specifically, to a signature keyword generation and timeline-based search system for document search that generates signature keywords composed of words capable of representing the content of a document and generates a timeline based on the creation date during document search to improve the efficiency and accuracy of the search. Background Technology

[0003] Advancements in morphological analyzers and Natural Language Processing (NLP) algorithms have significantly advanced technologies for extracting nouns and deriving relationships within Korean documents. However, due to the nature of the Korean language, compound nouns are continuously generated, and searching for desired information becomes difficult if the relationships between them cannot be accurately identified.

[0004] Existing document search systems generally operate based on keywords and are implemented by searching for and displaying documents related to specific keywords entered by the user. While such search systems are useful for finding documents containing specific keywords, they have the drawback of making it difficult to track the chronological flow between documents or the evolution of topics because they do not consider the time the documents were written or the relevance of their subjects.

[0005] Meanwhile, the aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot necessarily be considered publicly known technology disclosed to the general public prior to the filing of the present invention. Prior art literature

[0007] Korean Patent Publication No. 10-2023-0062932 The problem to be solved

[0008] One aspect of the present invention provides a signature keyword generation and timeline-based search system for document search that improves search efficiency and accuracy by generating a signature keyword composed of words that can represent the content of a document and generating a timeline based on the date of creation when searching for documents.

[0009] The technical problems of the present invention are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0011] A signature keyword generation and timeline-based search system for document search according to one embodiment of the present invention generates a signature keyword composed of words that can represent the content of a document, generates a timeline based on the creation date when searching for a document, and searches for documents based on the generated signature keyword and the timeline.

[0012] The above knowledge graph-based question and answer generation method and system are,

[0013] A data collection unit that collects document data;

[0014] A preprocessing unit that separates the text included in the collected document data into sentences and extracts each word from each separated sentence;

[0015] A signature keyword generation unit that sets candidate keywords from extracted words, quantifies the association between any one set candidate keyword and another candidate keyword, and generates a signature keyword formed by combining at least two candidate keywords based on the quantified result; and

[0016] It includes a timeline management unit that extracts the creation date of a document as metadata, identifies keywords and date information related to the topic and stores them in a database, and manages the collected documents by sorting them in chronological order using the database.

[0017] The above preprocessing unit is,

[0018] Nodes are connected based on semantic similarity using the sentence embedding results, and nodes are connected based on thematic similarity using keywords extracted by the preprocessing unit.

[0019] The above signature keyword generation unit is,

[0020] Among the set multiple candidate keywords, the number of times the first candidate keyword appears in the document, the number of times the second candidate keyword appears in the document, and the number of times the first candidate keyword and the second candidate keyword appear simultaneously in the same sentence are set as variables, and an association index between the first candidate keyword and the second candidate keyword is calculated based on the set variables.

[0021] The above signature keyword generation unit is,

[0022] It is characterized by extracting candidate keyword combinations in which the calculated relevance index exceeds a preset threshold value, and generating a signature keyword formed by combining the extracted candidate keywords.

[0023] The above correlation index is,

[0024] It is characterized by being calculated in proportion to the ratio of the number of times the first candidate keyword and the second candidate keyword appear simultaneously in the same sentence, relative to the value calculated by combining the number of times the first candidate keyword appears in the document and the number of times the second candidate keyword appears in the document. Effects of the invention

[0026] According to one aspect of the present invention described above, it is possible to perform a search that effectively reflects complex semantic relationships between documents using signature keywords, and by generating a timeline based on the creation date during document search, users can efficiently explore documents of a specific topic in chronological order, thereby reducing search time and improving understanding of the searched documents. Brief explanation of the drawing

[0028] FIG. 1 is a conceptual diagram illustrating the overall flow of a signature keyword generation and timeline-based search system for document search according to an embodiment of the present invention. Specific details for implementing the invention

[0029] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the invention is limited only by the appended claims, including all equivalents to those claimed therein, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0030] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

[0031] FIG. 1 is a conceptual diagram illustrating the overall flow of a knowledge graph-based question-and-answer generation method and system according to an embodiment of the present invention, and FIG. 2 is a block diagram illustrating the schematic configuration of a knowledge graph-based question-and-answer generation method and system according to an embodiment of the present invention.

[0032] The signature keyword generation and timeline-based search system for document search according to the present invention aims to improve the efficiency and accuracy of search by generating a signature keyword composed of words that can represent the content of a document and generating a timeline based on the creation date during document search.

[0033] To this end, a signature keyword generation and timeline-based search system for document search according to one embodiment of the present invention can be implemented in a document search server.

[0034] The server may be a self-contained server for the service according to the present invention, a cloud server, or a peer-to-peer (P2P) set of distributed nodes.

[0035] The server can perform one or more of the computation, storage, reference, input / output, and control functions of a general computer, and may include an artificial neural network described below based on input data.

[0036] The server may include a processor and memory. The processor may perform a smart interior service according to the present invention and may include devices capable of performing such service. The processor may execute a program or control the server. Program code executed by the processor may be stored in memory. The memory may store relevant information for performing the service according to the present invention or a program for implementing a method. The memory may be volatile memory or non-volatile memory.

[0037] The server can use a network to send data to an external device or receive data from an external device.

[0038] The server can train artificial neural networks and can also utilize artificial neural networks once training is complete. The processor can train or execute artificial neural networks stored in memory, and memory can store artificial neural networks once training is complete. The electronic device that trains the artificial neural network and the electronic device that uses it may be the same, but they may also be separate.

[0039] This document management server includes a data collection unit (110), a preprocessing unit (120), a signature keyword generation unit (130), and a timeline management unit (140).

[0040] The data collection unit collects document data. In one embodiment, the data collection unit may collect document data from a legacy system, but is not limited thereto, and it is understood that document data can be collected through various widely known methods and channels.

[0041] The preprocessing unit separates the text contained in the collected document data into sentences. Subsequently, the preprocessing unit performs sentence embedding to vectorize each separated sentence and simultaneously extracts keywords from each separated sentence.

[0042] In other words, the preprocessing unit can divide the collected document data into individual sentences and then split the collected documents into nouns and compound nouns using a morphological analyzer. Additionally, the preprocessing unit can generate a candidate set for the signature keywords described later by extracting candidate keywords using a Large Language Model (LLM).

[0043] The signature keyword generation unit (130) converts text within a document into an embedding vector using an embedding technique (BERT, Word2Vec, etc.) and identifies relationships by analyzing semantic similarity between words that constitute a candidate group for signature keywords.

[0044] To this end, the signature keyword generation unit (130) calculates the correlation index between candidate keywords using the following mathematical formula 1.

[0046] [Mathematical Formula 1]

[0047]

[0048] Here, SKS(Wn, Wm) is the association index between candidate keyword n ​​(hereinafter the first candidate keyword) and candidate keyword m (hereinafter the second candidate keyword), p(Wn) is the total number of times the first candidate keyword appears in the document, p(Wm) is the total number of times the second candidate keyword appears in the document, and p(Wn, Wm) is the number of times the first candidate keyword and the second candidate keyword appear simultaneously in the same sentence.

[0049] In other words, the present invention quantifies and displays information on how frequently two words appear within a document, and a higher correlation index indicates that the two words are related to each other.

[0050] In this case, the present invention does not quantify association based solely on the frequency of a word's appearance within a document, because it is difficult to consider modifiers or articles that appear meaninglessly within a sentence as having a deep relationship with a specific candidate keyword.

[0051] Accordingly, the present invention is characterized by the ability to quantify the association between words more reliably by calculating an association index that considers the frequency of simultaneous occurrence of two words.

[0052] Meanwhile, in the mathematical formulas described above, the variable of each term can be calculated by considering only the magnitude of each variable. That is, for each variable according to the mathematical formulas described above, only the magnitude of the variable is considered, and the consumer propensity index can be set using the magnitude of the result of the left-hand term, which is the result calculated in this way.

[0053] In the mathematical formula described above, the reason for applying a logarithmic function to the calculated value is that the logarithmic function can express large values ​​by relatively reducing them. That is, as the x value increases, the logarithmic function changes into a form that is less affected by units or scales. This allows the formula to respond less sensitively to changes in units, enabling the comparison of data of various scales on the same scale. Furthermore, the logarithmic function expresses changes in x values ​​as linear values ​​when they exhibit exponential differences. This allows for the appropriate reduction of large ranges of change, making it useful when the change in x values ​​has a proportional nature. In the mathematical formula described above according to the present invention, a logarithmic function was applied to the final calculated value to prevent large changes caused solely by the calculated value.

[0054] The timeline generator performs the function of analyzing input documents to extract information such as major topics, keywords, and creation dates. Utilizing Natural Language Processing (NLP) technology, the timeline generator identifies key words and context within the document and derives the document's core topic based on this.

[0055] The timeline generation unit extracts the creation date of a document as metadata during this process, identifies keywords and date information related to the topic, stores them in a database, and uses the stored database to sort the chronological order of documents to form a timeline of documents with similar topics in chronological order. Accordingly, users can navigate the timeline for a specific topic, and documents related to that topic can be intuitively displayed on the timeline.

[0056] In some other embodiments, instead of extracting candidate keywords from individual sentences using existing algorithms such as TF-IDF, TextRank, etc., the preprocessor calculates the importance index of each word included in the sentence using the following mathematical formula 1, and sets the word with the highest calculated importance index as the candidate keyword and extracts it.

[0058] [Mathematical Formula 1]

[0059]

[0060] Here, I_n is the importance index for word n, r_n is the number of times word n ​​appears repeatedly in document data, v_n is the magnitude value of the embedding vector for word n, v_i is the magnitude value of the embedding vector for the i-th most frequently occurring word in document data, and t is a variation constant, which is set to a first constant value if word n ​​is a word included in a preset important word dictionary, and is set to a second constant value smaller than the first constant value if word n ​​is a word not included in a preset important word dictionary.

[0061] In this way, the preprocessing unit calculates an importance index for each word included in a sentence by comprehensively considering factors such as embedding vector similarity, frequency of repetition, and inclusion in a dictionary, and sets the word with the highest calculated importance index as the keyword of the sentence, thereby improving the reliability of the set keyword.

[0062] Meanwhile, in the mathematical formulas described above, the variable of each term can be calculated by considering only the magnitude of each variable. That is, for each variable according to the mathematical formulas described above, only the magnitude of the variable is considered, and the consumer propensity index can be set using the magnitude of the result of the left-hand term, which is the result calculated in this way.

[0063] In the mathematical formula described above, the reason for applying a logarithmic function to the calculated value is that the logarithmic function can express large values ​​by relatively reducing them. That is, as the x value increases, the logarithmic function changes into a form that is less affected by units or scales. This allows the formula to respond less sensitively to changes in units, enabling the comparison of data of various scales on the same scale. Furthermore, the logarithmic function expresses changes in x values ​​as linear values ​​when they exhibit exponential differences. This allows for the appropriate reduction of large ranges of change, making it useful when the change in x values ​​has a proportional nature. In the mathematical formula described above according to the present invention, a logarithmic function was applied to the final calculated value to prevent large changes caused solely by the calculated value.

[0064] In addition, the preprocessor can perform embeddings that vectorize sentences based on a language model.

[0065] The method of representing words or sentences in the form of dense vectors is called word embedding or sentence embedding, and these dense vectors are also called embedding vectors because they are the results obtained through the word embedding process.

[0066] Word embedding methodologies include LSA, Word2Vec, FastText, and Glove. Embedding(), a tool provided by Keras, does not use the aforementioned methods; instead, it converts words into dense vectors with random values ​​and then learns the word vectors in a manner similar to learning the weights of an artificial neural network. Therefore, embedding vectors are represented as real values.

[0067] The preprocessing unit can build a neural network that extracts contextual information from input data by training the data with the Word2Vec algorithm to understand the meaning of the data.

[0068] The Word2Vec algorithm may include a Neural Network Language Model (NNLM). A Neural Network Language Model is fundamentally a neural network composed of an Input Layer, a Projection Layer, a Hidden Layer, and an Output Layer. The Neural Network Language Model is used as a method for vectorizing words. Since the Neural Network Language Model is a well-known technology, a more detailed explanation will be omitted.

[0069] The Word2vec algorithm is designed for text mining and determines proximity based on the preceding and succeeding relationships between words. It is an unsupervised learning algorithm. As its name suggests, Word2vec is a quantitative technique that represents the meaning of words in vector form. The Word2vec algorithm can represent each word as a vector in a space of approximately 200 dimensions. By utilizing the Word2vec algorithm, a vector corresponding to each word can be obtained.

[0070] The Word2vec algorithm can enable a dramatic improvement in precision in the field of natural language processing compared to other conventional algorithms. Word2vec learns the meaning of words by utilizing the relationships between words and adjacent words within sentences of an input corpus. Based on artificial neural networks, the Word2vec algorithm starts from the premise that words with the same context carry similar meanings. The algorithm learns through text documents, training the neural network to identify related words by identifying other words that appear nearby (approximately 5 to 10 words before and after) a given word. Since words with related meanings are highly likely to appear close together within a document, the two words can gradually acquire closer vectors as the learning process is repeated.

[0071] There are two training methods for the Word2vec algorithm: CBOW (Continuous Bag Of Words) and skip-gram. The CBOW method predicts a target word by utilizing the context created by surrounding words. The skip-gram method predicts words that may follow a single word. The skip-gram method is known to be more accurate in large-scale datasets.

[0072] Accordingly, in the embodiments of the present invention, a Word2vec algorithm using the skip-gram method is used. For example, if training is successfully completed through the Word2vec algorithm, similar words can be located nearby in a high-dimensional space. According to the Word2vec algorithm described above, the closer the distribution of surrounding words within a training document, the more similar the calculated vector values ​​can be, and words with similar calculated vector values ​​can be considered similar. Since the Word2vec algorithm is a known technology, a more detailed explanation regarding the calculation of vector values ​​will be omitted.

[0073] The preprocessing unit can input collected data into a neural network to extract evaluation result vector values ​​representing contextual information.

[0074] The preprocessing unit calculates the similarity between the evaluation result vector value and each of the multiple reference vector values, and can extract the reference vector value with the highest similarity to the evaluation result vector value among the multiple reference vector values. At this time, the Euclidean distance, cosine similarity, Tanimoto coefficient, etc., may be adopted as the similarity calculation method.

[0075] The preprocessing unit can extract the word corresponding to the reference vector value with the highest similarity to the evaluation result vector value as the word corresponding to the recognized text.

[0076] Furthermore, the preprocessing unit can train artificial neural networks or utilize artificial neural networks that have completed training. The processor can train or execute artificial neural networks stored in memory, and memory can store artificial neural networks that have completed training. The electronic device that trains the artificial neural network and the electronic device that utilizes it may be the same, but they may also be separate. Artificial intelligence is a computer system that partially implements the functions of the human brain and is capable of learning, speculating, and making judgments on its own. As learning progresses, the probability of extracting the correct answer may increase. Artificial intelligence can be composed of learning and elemental technologies that utilize it. The learning of artificial intelligence is an algorithmic technology that classifies and learns features based on input data, and the elemental technologies may be technologies that partially implement the functions of the human brain by utilizing learning algorithms.

[0077] Artificial intelligence is a technology that facilitates the approach to problems where multiple probabilistic answers are possible, enabling it to logically and probabilistically infer optimal cycles, methods, and plans based on input data. AI inference techniques can include evaluating input data, optimization prediction, knowledge and probability-based reasoning, and preference-based planning.

[0078] Artificial neural networks are learning algorithms in the field of machine learning that programmatically implement the connections between neurons and synapses in the brain. By creating a neural network structure through programming and then training it, artificial neural networks can acquire desired functions. Although errors may exist, they can learn from massive datasets to produce appropriate output data from input data. They have the advantage of being able to obtain output data that has yielded statistically good results and are similar to human reasoning.

[0079] The preprocessing unit can construct query / metric datasets required for learning using artificial intelligence algorithms built based on big data, and for this purpose, it may include multiple pre-trained artificial neural networks.

[0080] The system according to the present invention may include a plurality of pre-trained artificial neural networks for performing machine learning algorithms. Through machine learning, it can output output data based on input data and learn autonomously using the results, thereby improving its data processing capabilities. The artificial neural network can extract features and predict regularities based on input data to output result data, and as this process accumulates, the reliability of the result data increases.

[0081] In this embodiment, the artificial neural network may be an algorithm that outputs text data from at least one feature data among the shape, length, number, and height difference of an object recognized as text. The artificial neural network can infer the best output data by using big data as input data as is, or by using it as input data after undergoing a processing step to clean up unnecessary data.

[0082] Artificial intelligence machine learning models are classified according to the type of learning into Supervisory Learning, Unsupervisory Learning, Semi-supervised Learning, and Reinforcement Learning. In addition, machine learning algorithms include Decision Trees, K-Nearest Neighborhoods, Artificial Neural Networks, Support Vector Machines, Ensemble Learning, Gradient Descent, and Na Bayes Classifier, Hidden Markov Model, K-Means Clustering, etc. can be used.

[0083] Artificial neural networks can be pre-trained on various input values ​​that may be included in the input data. An artificial neural network can be trained using reinforcement learning, a learning method. Reinforcement learning is a method that gradually increases the probability of obtaining the correct result by setting rewards and constraints. Artificial neural networks can also be modeled based on Convectional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs).

[0084] In this way, the preprocessing unit can distinguish each sentence from document data using big data and artificial neural networks, and construct embeddings for each distinguished sentence.

[0085] The technology according to the present invention, as described above, may be implemented in the form of program instructions that can be executed through various computer components or implemented as an application, and may be recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either individually or in combination.

[0086] The program instructions recorded on the above-mentioned computer-readable recording medium are those specifically designed and configured for the present invention, but may also be those known and available to those skilled in the art of computer software.

[0087] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory.

[0088] Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0089] Although the invention has been described above with reference to embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as set forth in the following claims.

Claims

Claim 1 A signature keyword generation and timeline-based search system for document search, which generates a signature keyword composed of words that can represent the content of a document, generates a timeline based on the creation date when searching for documents, and searches for documents based on the generated signature keyword and the timeline. Claim 2 In claim 1, the knowledge graph-based question and answer generation method and system comprises: a data collection unit for collecting document data; a preprocessing unit for separating text included in the collected document data by sentence and extracting words from each separated sentence; a signature keyword generation unit for setting candidate keywords from the extracted words, quantifying the association between any one of the set candidate keywords and another candidate keyword, and generating a signature keyword formed by combining at least two candidate keywords based on the quantified result; and a timeline management unit for extracting the creation date of a document as metadata, identifying keywords and date information related to the topic and creating a database, and managing the collected documents by sorting them in chronological order using the database, thereby forming a signature keyword generation and timeline-based search system for document search. Claim 3 A signature keyword generation and timeline-based search system for document search, wherein the preprocessing unit connects nodes based on semantic similarity using sentence embedding results and connects nodes based on thematic similarity using keywords extracted by the preprocessing unit. Claim 4 In paragraph 2, the signature keyword generation unit sets the number of times a first candidate keyword appears in a document, the number of times a second candidate keyword appears in a document, and the number of times the first candidate keyword and the second candidate keyword appear simultaneously in the same sentence among a plurality of set candidate keywords as variables, and calculates an association index between the first candidate keyword and the second candidate keyword based on the set variables, a signature keyword generation and timeline-based search system for document search. Claim 5 A signature keyword generation and timeline-based search system for document search, wherein, in paragraph 4, the signature keyword generation unit extracts candidate keyword combinations in which the calculated relevance index exceeds a preset threshold value, and generates a signature keyword formed by combining the extracted candidate keywords. Claim 6 A signature keyword generation and timeline-based search system for document search, characterized in that, in paragraph 4, the relevance index is calculated in proportion to the ratio of the number of times the first candidate keyword and the second candidate keyword appear simultaneously in the same sentence compared to the value calculated by multiplying the number of times the first candidate keyword appears in the document and the number of times the second candidate keyword appears in the document.

Images