Method, device and apparatus for generating word-text graph

Abstract

The present application relates to the field of computer technology, and discloses a method for generating a word-text graph. The method comprises: obtaining a text set, wherein the text set comprises two or more texts; extracting keywords from each of the texts; obtaining a first distance between the keyword and the corresponding text, obtaining a second distance between each keyword, and obtaining a third distance between each of the texts; generating a word-text graph for the text and the keyword based on the first distance, the second distance, and the third distance. By extracting keywords from each text, obtaining the distance between the keyword and its corresponding text, the distance between the keywords, and the distance between each text, and generating a word-text graph for the text and the keyword based on the distance, the relationship between the text and the keyword, the relationship between each keyword, and each text can be reflected. The present application also discloses a device and equipment for generating a word-text graph.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and for example, to a method, device and apparatus for generating a word-text graph. Background Art

[0002] With the advent of the big data era, the scale of text storage and processing is getting larger and larger. People are not only limited to obtaining text information, but also pay more attention to the relationship between text and text, keywords and text, and keywords and keywords.

[0003] In the process of implementing the embodiments of the present disclosure, it was found that there are at least the following problems in the related art: there is a lack of a word-text graph that can reflect the relationship between texts and keywords. Summary of the invention

[0004] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0005] The embodiments of the present disclosure provide a method, apparatus and device for generating a word-text graph, so as to reflect the relationship between text and keywords.

[0006] In some embodiments, the method comprises:

[0007] Acquire a text set, wherein the text set includes more than two texts;

[0008] extracting keywords from each of the texts;

[0009] Obtaining a first distance between the keyword and its corresponding text, obtaining a second distance between each keyword, and obtaining a third distance between each text;

[0010] A word-text graph is generated from the text and the keyword according to the first distance, the second distance and the third distance.

[0011] In some embodiments, the device includes: a processor and a memory storing program instructions, and the processor is configured to execute the above-mentioned method for generating a word-text graph when executing the program instructions.

[0012] In some embodiments, the device includes: the above-mentioned device for generating a word-text graph.

[0013] The method, device and equipment for generating a word-text graph provided by the embodiments of the present disclosure can achieve the following technical effects: by extracting keywords from each text, obtaining the distance between the keyword and its corresponding text, the distance between the keywords and the distance between each text, and generating a word-text graph of the text and the keyword based on the distance, which can reflect the relationship between the text and the keyword, and between each keyword and each text.

[0014] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0016] Figure 1 is a schematic diagram of a method for generating a word-text graph provided by an embodiment of the present disclosure;

[0017] Figure 2 It is a schematic diagram of a device for generating a word-text graph provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0019] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0020] Unless otherwise stated, the term "plurality" means two or more.

[0021] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0022] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0023] Combination Figure 1 As shown, the present disclosure provides a method for generating a word-text graph, including:

[0024] Step S101, obtaining a text set, the text set including more than two texts;

[0025] Step S102, extracting keywords from each text;

[0026] Step S103, obtaining a first distance between a keyword and its corresponding text, obtaining a second distance between each keyword, and obtaining a third distance between each text;

[0027] Step S104, generating a word-text graph from the text and the keywords according to the first distance, the second distance and the third distance.

[0028] The method for generating a word-text graph according to the embodiment of the present disclosure is adopted to extract keywords from each text, obtain the distance between the keywords and their corresponding texts, the distance between the keywords and the distance between each text, and generate a word-text graph of the text and keywords based on the distance, which can reflect the relationship between the text and the keywords, and between each keyword and each text.

[0029] Optionally, obtaining the first distance between the keyword and the text corresponding to it includes: obtaining words in the text whose first similarity with the keyword meets a threshold, the number of word samples in the text, the length of each word sample in the text, and the average length of all word samples in the text;

[0030] By calculation

[0031]

[0032] Obtaining a first distance between the keyword and the text corresponding to it;

[0033] Among them, Score(q,d) is the first distance, q is the keyword in the text, d is the text corresponding to keyword q, n(d) is the number of all words in text d, n c (d) is the number of word samples in text d, is a word in the text whose first similarity with the keyword meets the threshold, is the first adjustment factor, υ is the second adjustment factor, k is the third adjustment factor, z is the fourth adjustment factor, θ1 is the first component weight factor, θ2 is the second component weight factor, sim(attr) is the second similarity between the texts in the text set, dl cis the length of each word in the text, avgdl c is the average length of all word samples in the text, N is the number of texts in the text set, and n(·) represents the count.

[0034] Optionally, the second similarity between the texts in the text set is the attribute similarity between the texts, that is, the similarity of the bibliographic information between the texts.

[0035] Optionally, the word samples are all non-repeated words in the text.

[0036] In this way, by adding words that meet the first similarity threshold with the keyword, introducing the probability distribution of word sample length, and introducing the attribute similarity between texts, the calculated first distance between the keyword and its corresponding text is more accurate and can better reflect the relationship between the text and the keyword.

[0037] Optionally, obtaining the second distance between the keywords includes: obtaining the distance between the meanings of the keywords in the text and the number of characters matched between the keywords;

[0038] By calculation

[0039]

[0040] Get the second distance between keywords;

[0041] Among them, Score(q1,q2) is the second distance, L1 is the distance between the meanings of keyword q1 and keyword q2, m is the number of matching characters between keywords, t is the number of unmatched characters between keyword q1 and keyword q2, n(D) is the number of text sets, n D (q1,q2) is the number of texts in which keyword q1 and keyword q2 appear simultaneously in the text set D, n D (q1) is the number of texts containing keyword q1 in the text set D, n D (q2) is the number of texts in which keyword q2 appears in the text set D, len(q1) is the length of keyword q1, len(q2) is the length of keyword q2, γ is the proportional coefficient, w1 is the word meaning weight, and w2 is the word co-occurrence weight.

[0042] Optionally, when q1[o] and q2[l] satisfy |ol|≤γ·min(len(q1),len(q2)), the characters in keywords q1 and q2 meet the matching condition. For example, keyword q1 is "electric vehicle" and keyword q2 is "electric car", and there are three characters "electric, dynamic, car" in keywords q1 and q2 respectively that meet the matching condition, that is, m is 3.

[0043] In this way, by considering the distance between keyword meanings, word sample similarity and word co-occurrence when calculating the second distance between keywords, the calculated second distance, i.e., word spacing, is more accurate and can better reflect the relationship between keywords.

[0044] In some embodiments, there are semantically subordinate relationships, strong or weak meanings, synonyms, near-synonyms, and antonyms between keywords. Such keywords can be semantically distinguished from each other, but such distances are usually difficult to express in digital form. In some embodiments, "Chongqing" and "Yubei" are in a subordinate relationship. Semantically speaking, Yubei is a district under the jurisdiction of Chongqing, and Yubei should have zero distance from Chongqing. On the other hand, Chongqing has more than one district, Yubei, so the distance between such keywords is difficult to measure. The distance between keywords such as synonyms and antonyms cannot be calculated by an unsupervised algorithm model. In the case of synonyms and antonyms, the synonym similarity calculation interface of Harbin Institute of Technology or Tencent is used for calculation.

[0045] The distance between two completely unrelated keywords is obtained according to the number of times and weights that the two completely unrelated keywords appear in the same text. In some embodiments, when the two keywords are neither synonyms nor frequent terms, the two keywords are assigned a maximum distance. Optionally, when the frequency of the two keywords appearing together meets a preset condition, the two keywords are frequent terms, such as husband and wife. Two originally independent individuals often appear together. If you find the husband, you can contact the wife, and if you find the wife, you can contact the husband. Such words are usually closer. Optionally, two completely unrelated keywords in a natural language text also have a closer distance. In some embodiments, "beer" and "diapers" have a closer distance under certain circumstances.

[0046] Construct a link graph based on the distance between the keyword and the text; traverse the text linked by the keyword and combine it with other keywords in turn; calculate the frequency between the keyword and other words; sort by frequency and normalize the keywords; when the keywords are neither synonyms nor related words, maximize the distance between the keywords.

[0047] Optionally, obtaining the third distance between each text includes: obtaining a paragraph with the highest paragraph weight in the text; obtaining a sparse matrix of word spacing of the topic sentence of the paragraph with the highest paragraph weight, and a second similarity between each text in the text set;

[0048] By calculation Get the third distance between texts;

[0049] Among them, Score(v i ,v j) is the third distance, v i For text i The paragraph with the highest weight in the paragraph, v j For text j The paragraph with the highest weight in the middle paragraph, H i v i The word spacing sparse matrix of the topic sentence in H j v j The word spacing sparse matrix of the topic sentence in , is the first weight factor, is the second weight factor, sim(attr) is the second similarity between the texts in the text set, and E(·) is the mean value.

[0050] In some embodiments, the set of terms in the paragraph with the highest weight in text 1 is [AB], and the set of terms in the paragraph with the highest weight in text 2 is [BCD]. Optionally, in the same text, the distance between different words is 0.5, for example, the distance between A and B is 0.5; the distance between the same words is 0; in different texts, the distance between the same words is 0; the distance between different words is 1, for example, the distance between A and C is 1. The word spacing sparse matrix of text 1 is shown in Table 1, which is an example table of the word spacing sparse matrix of text 1:

[0051] word A B C D A 0 0.5 1 1 B 0.5 0 1 1 C 1 1 1 1 D 1 1 1 1

[0052] Table 1

[0053] The word spacing sparse matrix of text 2 is shown in Table 2. Table 2 is an example table of the word spacing sparse matrix of text 2:

[0054] word A B C D A 1 1 1 1 B 1 0 0.5 0.5 C 1 0.5 0 0.5 D 1 0.5 0.5 0

[0055] Table 2

[0056] The absolute values ​​of the matrix differences between the word spacing sparse matrix of text 1 and the word spacing sparse matrix of text 2 are shown in Table 3, which is an example table of the absolute values ​​of the matrix differences:

[0057] word A B C D A 1 0.5 0 0 B 0.5 0 0.5 0.5 C 0 0.5 1 0.5 D 0 0.5 0.5 1

[0058] Table 3

[0059] Calculate according to Table 3 The absolute value of the sparse matrix difference between the word spacing between text 1 and text 2 is 11. There are 4 words in text 1 and text 2, so the matrix is ​​a 4*4 matrix. Take the first weight factor Second weighting factor The second similarity between text 1 and text 2 is sim(attr) = 0.4, then the third distance between text 1 and text 2 is

[0060] Optionally, obtaining the paragraph with the highest paragraph weight in the text includes: obtaining word weights of all words in the text;

[0061] By calculation Get paragraph p ij The paragraph weight of

[0062] Among them, Score(p ij ) is paragraph p ij The paragraph weight, TR c is the word weight, n(c) is the number of words in the paragraph;

[0063] By calculation Get the paragraph with the highest paragraph weight;

[0064] Among them, v is the paragraph with the highest paragraph weight in the text.

[0065] Optionally, computed by the TextRank method Get text d i The weights of all words in For text i Chinese word c f The weight of .

[0066] Optionally, each word in the paragraph v with the highest weight is used to represent the text d. For example, there are two paragraphs A and B in the text, and there are three words {a′, b′, c′} in paragraph A, and two words {b′, d′} in paragraph B. The weights of the four words a′, b′, c′, d′ are 1, 2, 1, 3 respectively. Then the weight of paragraph A is (1+2+1) / 3=4 / 3, and the weight of paragraph B is (2+3) / 2=2.5. The weight of paragraph B is greater than that of paragraph A, so paragraph B is the paragraph with the highest weight. Paragraph B is selected as the central paragraph of the text, and all the words in paragraph B are used to represent the text.

[0067] In this way, by introducing the attribute similarity between texts and the word spacing sparse matrix in the calculation, and representing the text with the paragraph with the highest weight, the calculated third distance between texts is more accurate and can better reflect the relationship between texts.

[0068] Optionally, it also includes: preprocessing the text; the preprocessing of the text includes one or more of garbled code processing, punctuation cleaning, space symbol processing, punctuation replacement, format processing and metadata formatting.

[0069] Optionally, the preprocessed text is stored separately from the original text, which not only preserves the most original data resources but also lays a data foundation for subsequent processing.

[0070] Optionally, a word-text graph is generated for the text and keywords according to the first distance, the second distance and the third distance, including: connecting the keywords and their corresponding texts through the first edge, connecting each text through the second edge, and connecting each keyword through the third edge; using the first distance as the length of the first edge, the second distance as the length of the second edge, and the third distance as the length of the third edge.

[0071] In this way, by obtaining the keywords in the text and obtaining the first distance, the second distance, and the third distance based on the keywords, a word-text graph between the text and the keywords is generated based on the first distance, the second distance, and the third distance, which reflects the relationship between the text and the keywords, between the keywords, and between the texts.

[0072] In some embodiments, the original data files are classified into text in txt (Text File) format, text in pdf (portable document format) and doc (document) formats, text in image formats such as png (portable network graphics) and jpg (joint photographic experts group, an image file format), text in irregular formats, and non-text. Among them, text in txt format does not store pictures, tables, audio, video, and text with formats; text in pdf and doc formats can store pictures, tables, text, and text in various formats.

[0073] Optionally, texts in different forms are converted into a unified, easy-to-read text format. In some embodiments, text in the form of images is converted into text in text format through OCR (Optical Character Recognition) recognition technology; text in the form of audio is converted into text in text format through speech recognition technology, etc. In this way, not only data differences are eliminated, but also the universality of applications is improved.

[0074] Optionally, the characters in the acquired text are converted into characters in a unified encoding format.

[0075] Optionally, the long book text is processed into text granularity based on the chapter division condition, and the long book text is divided into different chapter units. In this way, the influence of extracting keywords on the text with diverse description content and too long length is reduced.

[0076] Optionally, words and phrases are used as the basic units of NLP (Neuro-Linguistic Programming) for word segmentation. Since the expression of continuous text sentences is too diverse, and a single word usually cannot fully express a morpheme, words and phrases are selected as the basic units of NLP.

[0077] Optionally, word segmentation is performed by Jieba word segmentation tool based on bidirectional GRU (Gate Recurrent Unit) network structure, Thulac and Ltp word segmentation tools based on structured perceptron SP, HanLP, NLPIR, Ansj word segmentation tools based on Bigram+HMM (bigram model + hidden Markov model), Jieba word segmentation tool based on Unigram+HMM (unigram model + hidden Markov model), FNLP word segmentation tool based on PA (Passive Aggressive) algorithm, Pkuseg and StanfordCoreNLP word segmentation tool based on CRF (Conditional random fields), IK-analyzer word segmentation tool based on forward iterative finest granularity segmentation, PaddlePaddle word segmentation tool based on deep learning LAC (Lexical Analysis of Chinese), and Jcseg word segmentation tool based on Chinese word segmentation algorithm MMSEG.

[0078] Optionally, keywords are extracted by a statistical method. In some embodiments, keywords are extracted by methods such as TextRank, TF-IDF (Term Frequency / Inverse Document Frequency) based on frequency statistics, TWE, TPR (Topic PageRank, a topic-based ranking algorithm) based on the LDA (Latent Dirichlet Allocation, a text topic generation model) topic model. Optionally, extracting keywords includes: extracting the part of speech of the keywords. Optionally, the part of speech includes: noun, adverb, adjective, verb, etc. In some embodiments, if the tool for extracting keywords does not support the extraction of part of speech, other tools are used to do part of speech tagging.

[0079] Optionally, different keyword extraction tools based on the same method are used to extract keywords. After keyword cleaning, the most original keyword list of each method is obtained. Then the keywords are merged to obtain a list of <words, part of speech, word frequency>, i.e., a candidate keyword list. At the same time, the original list of keyword lists obtained by each method is retained to facilitate TextRank calculation and TWE topic keyword extraction.

[0080] In some embodiments, the keyword extraction is performed by the TF-IDF method, and the first candidate keyword list is obtained after keyword cleaning; the keyword extraction is performed by the TextRank method, and the second candidate keyword list is obtained after keyword cleaning; the keyword extraction is performed by the TWE method, and the third candidate keyword list is obtained after keyword cleaning, and the keywords in the first, second, and third candidate keyword lists are weighted normalized, and then the keywords are merged and sorted from high to low according to the weight, and the top 10 words are selected as keywords. Optionally, the keyword format is <word, word weight, part of speech, word theme, word vector>.

[0081] Optionally, keyword extraction is performed by a TF-IDF method, including: the TF-IDF weight of a word is proportional to the frequency of the word in the text, and inversely proportional to the frequency of the word in all texts: Where N t is the number of times word t appears in the text, N d is the number of all words in the text, D is the number of all texts in the text set, and D t is the number of all texts containing word t. In D t =D, the logarithm is negative, and the formula Optimize according to The optimized formula is:

[0082] In some embodiments, the text candidate words are the summary candidate words in each text keyword set, and the candidate words are calculated by Get ij TF-IDF value of . Among them, γ ij is the frequency of new words, |t ij | is the word length, λ is the word length weight, D i is the i-th text, t ij For text D i The jth candidate word in f ij t ij In some embodiments, D i The number of candidate words in |D i |, the total list of candidate words is <t ij , fij > list, the text set is D i = {<t ij , f ij >| j = 1, 2…|D i |}. In some embodiments, the information contained in large-grained words is greater than that contained in small-grained words. By calculating to obtain the new word frequency of the word, and promoting long words, large-grained words can be correctly distinguished; by calculating to obtain the TF-IDF value of t ij ; sort the obtained TF-IDF values of t ij from high to low, and select the top 10 words in the sorting as keywords.

[0083] Optionally, keyword extraction is performed by a method based on TextRank, including: drawing on the PageRank algorithm for web page ranking, TextRank regards the text as a network of words, and the network is established with undirected relationship links through n-grams word co-occurrence grammar elements. The network link represents the semantic relationship between word t i and word t j . The importance w s (s i ) of a word is determined by the importance of other words that link to it. By calculating to obtain the importance of the word, where w s (t i ) is the weight value of word t i , e is the attenuation factor, and the value range is 0 < e < 1. Optionally, e is taken as 0.85. Adj(s i ) is the set of neighboring vertices, w ji is the weight of word s i and s j , and the weight value w ji is determined by the co-occurrence times within a window w. In the case of using the keyword extraction method based on TextRank, the words in the original order must be used.

[0084] In some embodiments, keyword extraction based on TextRank does not extract the keywords of a certain text by relying on the information of all texts, but separately depends on the text content in a single text to establish an undirected network graph of words, and then calculates the weight formula to obtain the weight value w s (t i ) of each candidate word, and sorts them from high to low according to the weight, and selects the top 10 words in the sorting as keywords.

[0085] Optionally, keyword extraction is performed using the TWE method, including: after determining the word model through the LDA topic model, combining it with Word2Vec to obtain candidate words. Word2Vec includes two models: CBOW (Continuous Bag-of-Words) and Skip-Gram; the essence of Word2Vec is to obtain predicted word vectors through training in combination with contextual semantic words. LDA is a text topic generation model, also called a three-layer Bayesian probability model, which includes a three-layer structure of words, topics, and texts; text to topic follows a multinomial distribution, and topic to word follows a multinomial distribution.

[0086] In some embodiments, topics are extracted from the text in the text set D through the LDA topic model, and a topic is assigned to each word based on probability; then the words and topics in the text set are initialized through Word2Vec word embedding; then the TWE word vectors are trained, and then keywords are extracted based on the weights of the word vectors, and they are ranked in descending order according to the size of the weight values, and the top 10 words are selected as keywords.

[0087] In this way, starting from the text data at the basic structure, the method of multiple word prompting is used to extract text features and improve the recall rate of words. Then, keywords are extracted step by step multiple times according to the original word sequence list, and the keyword vector is calculated using the word embedding Word2Vec (Word to vector, a related model used to generate word vectors). Character keywords are represented in the form of mathematical vectors, and then clustering is performed through the relationship between vectors to find words near the cluster center as keywords. Keywords are then screened through the TWE (Topic word embedding, topic word embedding) model in the composite form of topic-word vector to improve the quality of keywords. Finally, a word-text map of text and keywords is generated through clustering, which reflects the relationship between text and keywords, keywords, and text.

[0088] Optionally, after extracting the keywords, the words are cleaned. In some embodiments, the words that appear Chinese symbols but are not divided are cleaned, and the Chinese symbols are removed; keywords with function words such as "if" and "if" are removed; stop words in the keywords are removed, for example, the keyword "beautiful" is removed; long words are merged according to the part of speech relationship; quantifiers are deleted, etc.

[0089] Combination Figure 2As shown, an embodiment of the present disclosure provides a device for generating a word-text graph, including a processor (processor) 100 and a memory (memory) 101 storing program instructions. Optionally, the device may also include a communication interface (Communication Interface) 102 and a bus 103. Among them, the processor 100, the communication interface 102, and the memory 101 can communicate with each other through the bus 103. The communication interface 102 can be used for information transmission. The processor 100 can call the program instructions in the memory 101 to execute the method for generating a word-text graph of the above embodiment.

[0090] By extracting keywords from each text, obtaining the distance between the keywords and their corresponding texts, the distance between the keywords and the distance between each text, and generating a word-text graph of the text and keywords based on the distance, it can reflect the relationship between the text and keywords, and between each keyword and each text.

[0091] In addition, the program instructions in the above-mentioned memory 101 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0092] The memory 101 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 100 executes the function application and data processing by running the program instructions / modules stored in the memory 101, that is, the method for generating a word-text graph in the above embodiment is implemented.

[0093] The memory 101 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 101 may include a high-speed random access memory and may also include a non-volatile memory.

[0094] By using the device for generating a word-text graph provided by the embodiment of the present disclosure, by extracting keywords from each text, obtaining the distance between the keywords and their corresponding texts, the distance between the keywords and the distance between each text, and generating a word-text graph of the text and keywords based on the distance, the relationship between the text and the keywords, and between each keyword and each text can be reflected.

[0095] The embodiment of the present disclosure provides a device, including the above-mentioned device for generating a word-text graph. The device extracts keywords from each text, obtains the distance between the keyword and its corresponding text, the distance between the keywords and the distance between each text, generates a word-text graph of the text and the keyword based on the distance, and can reflect the relationship between the text and the keyword, and between each keyword and each text.

[0096] Optionally, the device is a computer or a server.

[0097] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned method for generating a word-text graph.

[0098] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for generating a word-text graph.

[0099] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0100] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0101] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0102] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0103] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0104] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for generating a word-text graph, characterized in that: include: Acquire a text set, wherein the text set includes more than two texts; extracting keywords from each of the texts; Obtaining a first distance between the keyword and its corresponding text, obtaining a second distance between each keyword, and obtaining a third distance between each text; Generate a word-text graph from the text and the keyword according to the first distance, the second distance and the third distance; Obtaining the first distance between the keyword and the corresponding text includes: obtaining words in the text whose first similarity with the keyword meets a threshold, the number of word samples in the text, the length of each word sample in the text, and the average length of all word samples in the text; calculating Obtaining a first distance between the keyword and its corresponding text; wherein, is the first distance, q is the keyword in the text, d is the text corresponding to keyword q, n(d) is the number of all words in text d, n c (d) is the number of word samples in text d, is a word in the text whose first similarity with the keyword meets the threshold, is the first adjustment factor, is the second adjustment factor, is the third adjustment factor, is the fourth adjustment factor, is the first component weight factor, is the second component weight factor, is the second similarity between the texts in the text set, is the length of each word in the text, is the average length of all word samples in the text, and N is the number of texts in the text set; Obtaining the second distance between the keywords includes: obtaining the distance between the meanings of the keywords in the text and the number of characters matching between the keywords; calculating Obtaining a second distance between the keywords; wherein, is the second distance, For keywords With keywords The distance between the meanings of the words, m is the number of characters that match between the keywords, and t is the distance between the keywords. With keywords The number of characters that do not match between them, n(D) is the number of text sets, The keywords appearing simultaneously in the text set D and keywords The number of texts, For keywords appearing in text set D The number of texts, For keywords appearing in text set D The number of texts, For keywords Length, For keywords Length, is the proportionality coefficient, is the word meaning weight, is the word co-occurrence weight; Obtaining the third distance between each of the texts includes: obtaining the paragraph with the highest paragraph weight in the text; obtaining the word spacing sparse matrix of the topic sentence of the paragraph with the highest paragraph weight, and the second similarity between each text in the text set; by calculating The third distance between the texts is obtained; wherein, is the third distance, is the paragraph with the highest paragraph weight in text i, is the paragraph with the highest paragraph weight in text j, for The word spacing sparse matrix of the topic sentence in , for The word spacing sparse matrix of the topic sentence in , is the first weight factor, is the second weight factor, is the second similarity between the texts in the text set, To find the mean; Generating a word-text graph from the text and the keyword according to the first distance, the second distance, and the third distance, including: connecting the keyword and its corresponding text through a first edge, connecting each of the texts through a second edge, and connecting each of the keywords through a third edge; The first distance is taken as the length of the first side, the second distance is taken as the length of the second side, and the third distance is taken as the length of the third side.

2. The method according to claim 1, characterized in that Obtain the paragraph with the highest paragraph weight in the text, including: Obtaining word weights of all words in the text; By calculation Get paragraph The paragraph weight of in, For paragraphs The paragraph weight of is the word weight, is the number of words in the paragraph; By calculation Get the paragraph with the highest paragraph weight; in, It is the paragraph with the highest paragraph weight in the text.

3. The method according to claim 1, characterized in that Also includes: Preprocess the text; The preprocessing of the text includes one or more of garbled code processing, punctuation cleaning, space symbol processing, punctuation replacement, format processing and metadata formatting.

4. A device for generating a word-text graph, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to perform the method for generating a word-text graph as described in any one of claims 1 to 3 when executing the program instructions.

5. A device, characterized in that: It comprises a device for generating a word-text graph as described in claim 4.

Images