Information analysis method and device based on high-order text network and electronic equipment

Abstract

The invention discloses an information analysis method, and particularly relates to an information analysis method and device based on a high-order text network and electronic equipment, and the method comprises the steps: constructing a corresponding high-order text network based on to-be-analyzed information data, in a k-order text hypernetwork, nodes are used for representing a group of k words appearing in a sentence, and the nodes are used for representing a group of k words appearing in the sentence; the sentence is a hyperedge which is connected with the plurality of nodes; calculating M structural feature indexes of the high-order text network corresponding to the to-be-analyzed information data to obtain a first index value corresponding to each structural feature index, and calculating M structural feature indexes of the high-order text network corresponding to reference information data to obtain a second index value corresponding to each structural feature index; and performing information analysis based on the first index value and the second index value, and evaluating the correlation between the to-be-analyzed information data and the reference information data. According to the method, the information analysis accuracy of the information data can be improved.

Description

Technical Field

[0001] The present invention relates to an information analysis method, and more particularly to an information analysis method, device and electronic equipment based on a high-order text network. Background Art

[0002] The author identification problem is to infer the most likely author of an anonymous text based on previous works. It has a long history and important applications in various fields such as text classification, information extraction, and forensic investigation. By analyzing the text, we can not only determine the identity of the text author, but also play an important role in application scenarios such as intelligence analysis and false information in cyber attacks.

[0003] In a network framework, text is often represented as a word co-occurrence network, where words act as nodes and edges represent their adjacencies. These adjacencies can be determined by examining the frequency of two words appearing in the same sentence or paragraph, or their statistical correlation. This format is attractive because it structurally represents textual data and effectively captures contextual relationships between words without requiring an understanding of their meaning or language grammar. Furthermore, by employing complex network metrics such as node degree (word frequency), average shortest path, and intermittency, many features that depend on authorship can be more clearly inferred, making the authorship identification process more understandable and interpretable. In particular, methods based on the complex network framework have shown remarkable accuracy when used to identify the author of a text, detect forged text, and identify computer-generated content.

[0004] Although such word co-occurrence networks are helpful in text information analysis, they are limited in several important ways. The key issue is that word co-occurrence networks focus primarily on pairwise interactions between words. However, human expression involves not only the selection of individual words, but also the habitual combination of multiple words into specific phrases. For example, individuals often use specific "buzzwords" and phrasings in their daily lives, which reflects higher-order structures in the language. Therefore, the accuracy of the analysis results of currently commonly used information analysis methods still needs to be improved. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the word co-occurrence network commonly used in the prior art mainly focuses on the pairwise interaction between words, which makes it difficult to accurately and completely characterize the text, resulting in low accuracy of the text analysis results. In order to solve the above problem, the present invention provides an information analysis method, device and electronic device based on a high-order text network.

[0006] The content of the present invention includes:

[0007] In a first aspect, an embodiment of the present invention provides an information analysis method based on a high-order text network, comprising:

[0008] Construct a corresponding high-order text network based on the information data to be analyzed. In a k-order text hypernetwork, a node is used to represent a group of k words appearing in a sentence. A sentence is a hyperedge connecting multiple nodes, and k is a positive integer.

[0009] Calculate M structural feature indicators of the high-order text network corresponding to the information data to be analyzed, and obtain a first indicator value corresponding to each of the structural feature indicators; and calculate M structural feature indicators of the high-order text network corresponding to the reference information data, and obtain a second indicator value corresponding to each of the structural feature indicators, where M is a positive integer;

[0010] Information analysis is performed based on the first indicator value and the second indicator value to evaluate the correlation between the information data to be analyzed and the reference information data.

[0011] Optionally, performing information analysis based on the first indicator value and the second indicator value to evaluate the correlation between the information data to be analyzed and the reference information data includes:

[0012] Determine a correlation threshold corresponding to a target structural feature indicator, wherein the target structural feature indicator is any one of the structural feature indicators;

[0013] Calculating the Pearson correlation coefficient between the information data to be analyzed and the reference information data based on the first index value and the second index value corresponding to the target structure characteristic index;

[0014] When the Pearson correlation coefficient is greater than the correlation threshold, it is determined that the information data to be analyzed is correlated with the reference information data; otherwise, it is determined that the information data to be analyzed is not correlated with the reference information data.

[0015] Optionally, determining a correlation threshold corresponding to the target structure feature indicator includes:

[0016] Constructing an indicator value set, the indicator value set comprising second indicator values ​​corresponding to the target structure feature indicators of a plurality of first reference information data, and second indicator values ​​corresponding to the target structure feature indicators of a plurality of second reference information data, the first reference information data being the reference information data belonging to the first category, and the second reference information data being the reference information data belonging to the second category;

[0017] Based on the indicator value set, calculating the Pearson correlation coefficient of the i-th first reference information data and the j-th second reference information data, where i and j are positive integers;

[0018] A correlation threshold corresponding to the target structure feature index is calculated based on the Pearson correlation coefficient.

[0019] Optionally, the correlation threshold k corresponding to the target structure feature index F for:

[0020]

[0021] Among them, a is used to represent the first category, c is used to represent the second category, a i The i-th first reference information data used to characterize the first category a, c j The j-th second reference information data used to characterize the second category c, A set of data used to represent the first reference information, A set of data used to represent the second reference information, To characterize a i and c j The Pearson correlation coefficient between .

[0022] Optionally, performing information analysis based on the first indicator value and the second indicator value to evaluate the correlation between the information data to be analyzed and the reference information data includes:

[0023] For each of the reference information data, respectively calculating the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators;

[0024] Based on the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators, the global correlation between the reference information data and the information data to be analyzed is determined, and the global correlation is used to characterize the degree of correlation between the reference information data and the information data to be analyzed.

[0025] Optionally, the global correlation between the information data to be analyzed and the reference information data is:

[0026]

[0027] Among them, R j It is used to characterize the correlation between the information data to be analyzed and the j-th reference information data, R j for:

[0028]

[0029] Among them, M is used to characterize the number of structural characteristic indicators, ρ ij Used to characterize the correlation between the information data to be analyzed and the jth reference information data under the i-th structural feature index.

[0030] Optionally, after determining the global correlation between the reference information data and the information data to be analyzed based on the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators, the method further includes:

[0031] The author of the reference information data having the greatest global correlation with the information data to be analyzed is determined as the author of the information data to be analyzed.

[0032] In a second aspect, an embodiment of the present invention provides an information analysis device based on a high-order text network, comprising:

[0033] A construction module is used to construct a corresponding high-order text network based on the information data to be analyzed. In a k-order text hypernetwork, a node is used to represent a group of k words appearing in a sentence. A sentence is a hyperedge connecting multiple nodes, and k is a positive integer.

[0034] A calculation module, used to calculate M structural feature indicators of the high-order text network corresponding to the information data to be analyzed, and obtain a first indicator value corresponding to each of the structural feature indicators, and calculate M structural feature indicators of the high-order text network corresponding to the reference information data, and obtain a second indicator value corresponding to each of the structural feature indicators, where M is a positive integer;

[0035] An analysis module is used to perform information analysis based on the first indicator value and the second indicator value, and evaluate the correlation between the information data to be analyzed and the reference information data.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps in the information analysis method based on a high-order text network as described in the first aspect.

[0037] In a fourth aspect, an embodiment of the present invention provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the information analysis method based on a high-order text network as described in the first aspect.

[0038] The beneficial effect of the present invention is that, in the embodiment of the present invention, a corresponding first hypernetwork is constructed based on the information data to be analyzed, and the high-order features between multiple words, phrases or sentences in the language structure of the information data to be analyzed can be better characterized through the first hypernetwork. The information analysis of the structure of the first hypernetwork based on the structural feature index can replaceably mine the structural features of the first hypernetwork, significantly improving the accuracy and precision of the correlation analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Attached Figure 1A flowchart of an information analysis method based on a high-order text network provided by an embodiment of the present invention;

[0040] Attached Figure 2 An example diagram of constructing different structure networks for information data provided by an embodiment of the present invention;

[0041] Attached Figure 3a The word frequency distribution of two example information data in the first-order text hypernetwork provided for the embodiment of the present invention;

[0042] Attached Figure 3b The word frequency distribution of two example information data in the second-order text hypernetwork provided by the embodiment of the present invention;

[0043] Attached Figure 4a Schematic diagram of the changes in structural feature indicators in the text hypernetwork of different authors;

[0044] Attached Figure 4b It is a schematic diagram of the Pearson correlation coefficient of the structural feature indicators in the first-order text hypernetwork;

[0045] Attached Figure 4c It is a schematic diagram of the Pearson correlation coefficient of the structural feature indicators in the second-order text hypernetwork;

[0046] Attached Figure 5a A schematic diagram of a correlation characteristic spectrum between the information data to be identified and the reference information data;

[0047] Attached Figure 5b It is a schematic diagram of global correlation;

[0048] Attached Figure 5c A diagram showing the relationship between global correlation and the number of indicators with the same structural features in two text hypernetworks;

[0049] Attached Figure 6a It is a schematic diagram of the accuracy comparison between the first-order text hypernetwork and the second-order text hypernetwork;

[0050] Attached Figure 6b The precision, recall and F1 value of author identification for a sample of 28 authors;

[0051] Attached Figure 7a A schematic diagram of the stylistic correlations between the Brontë sisters;

[0052] Attached Figure 7b Schematic diagram of the effect of the number of training texts on global relevance;

[0053] Attached Figure 7c A similarity network of Mark Twain's styles in different periods;

[0054] Attached Figure 7dSchematic diagram of the impact of reference text selection on global relevance;

[0055] Attached Figure 8 A schematic diagram of an information analysis device based on a high-order text network provided by an embodiment of the present invention;

[0056] Attached Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In the embodiments of the present application, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present application, the term "multiple" refers to two or more, and other quantifiers are similar. The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0060] The embodiments of the present application provide an information analysis method, device and electronic device based on a high-order text network. The method can be applied to the fields of author identification of unknown texts, intelligence text analysis, and false information analysis of network attacks, without specific limitation.

[0061] In some embodiments, information analysis is performed on the information data to be analyzed, and the obtained correlation analysis structure can be used to characterize the category to which the information data to be analyzed belongs. For example, in the field of author identification of unknown information data, by performing information analysis on the information data to be analyzed and obtaining the correlation between the information data to be analyzed and reference information data, the category to which the information data to be analyzed belongs, that is, the author of the information data to be analyzed, can be determined.

[0062] See also Figure 1 , Figure 1 The flowchart of the information analysis method based on the high-order text network provided by the embodiment of the present invention is as follows. Figure 1 As shown, the method specifically comprises the following steps:

[0063] Step 101, construct a corresponding high-order text network based on the information data to be analyzed. In a k-order text hypernetwork, a node is used to represent a group of k words appearing in a sentence, a sentence is a hyperedge connecting multiple nodes, and k is a positive integer.

[0064] Step 102, calculate M structural feature indicators of the high-order text network corresponding to the information data to be analyzed, obtain a first indicator value corresponding to each of the structural feature indicators, and calculate M structural feature indicators of the high-order text network corresponding to the reference information data, obtain a second indicator value corresponding to each of the structural feature indicators, and M is a positive integer.

[0065] Step 103: Perform information analysis based on the first indicator value and the second indicator value to evaluate the correlation between the information data to be analyzed and the reference information data.

[0066] In step 101, a corresponding high-order text network is constructed based on the information data to obtain a first hypernetwork corresponding to the information data to be analyzed, so that the basic elements (words and sentences) of the information data can be expressed through the text hypernetwork. The hypernetwork can be defined as an ordered pair H = (V, E), where V is a set of nodes and E is a series of supernodes. In a hypernetwork, each hyperedge can connect multiple nodes, not limited to two nodes. In a text hypernetwork, words in the information data are represented as nodes, sentences are represented as supernodes, and multiple words are connected to express a certain idea.

[0067] In some optional embodiments, the hypernetwork may be represented in the form of an incidence matrix H. The elements of the matrix H ij describes the interaction between node i and hyperedge j, where i is a node set V = {V1, V2, ..., V n}, j is the hyperedge set E = {E1, E2, ..., E m In some embodiments, an unweighted method is used to identify the jth hyperedge of Hij Encode, that is, if node i is included in hyperedge j, then H ij =1; otherwise H ij =0.

[0068] In the embodiment of the present application, the text hypernetwork constructed based on the information data can be a hypernetwork of any order. In the first-order text hypernetwork, a single word is a node and a sentence acts as a supernode. In the second-order text hypernetwork, a node represents a pair of words that appear simultaneously in the same sentence. Similarly, in a k-order text hypernetwork, a node depicts a group of k words that appear in a sentence.

[0069] In some embodiments, before step 101, the method further includes preprocessing the information data to be analyzed. Exemplarily, first, punctuation marks and words that convey less information are deleted from the information data to be analyzed (for example, words are excluded using the "stopwords" module in the Python Natural Language Toolkit (NLTK)). Second, all uppercase letters in the remaining words are converted to lowercase to avoid treating them as different words. Finally, the words are converted to standard form.

[0070] It should be understood that the reference information data is pre-acquired, and the parameter values ​​of the text hypernetwork and structural feature indicators corresponding to the reference information data have been pre-acquired. The specific construction and acquisition methods can refer to the relevant processing operations performed on the information data to be analyzed in this embodiment, and will not be repeated here.

[0071] See also Figure 2 , Figure 2 This shows the difference between the hypernetwork constructed in this application and the word co-occurrence network commonly used in the prior art. Figure 2 (a) is an example diagram of information data consisting of three sentences and eleven words. Figure 2 As shown in (b), in the word co-occurrence network ( wo rd co-occu rrenc e In a neural network, each node represents a word, and an edge usually connects two nodes, indicating a binary relationship (i.e., the relationship between two words). Usually, an edge is modeled as connecting two adjacent words that appear in the same sentence.

[0072] like Figure 2 The first-order text network in (c) Figure 2 The 2-order text network in (d) and Figure 2As shown in the third-order text hypernetwork (3-order text network) in (e), the hypernetwork constructed by this method allows nodes to represent not only single words but also word groups, and sentences are represented as hyperedges. Hyperedges can connect multiple nodes to represent higher-order relationships beyond simple binary connections.

[0073] by Figure 2 Taking the information data shown in (a) as an example, the text hypernetwork is classified according to the number of words in its nodes. In the first-order text hypernetwork, each word is represented as a node (V = {word1, word2, ..., word11}); in the second-order text hypernetwork, the nodes are represented as pairs (V = {(word2word3), (word2, word5), ..., (word4, word5)}); in the third-order text hypernetwork, the nodes are composed of triplets (V = {(word2, word3, word5)}).

[0074] Word co-occurrence networks focus primarily on pairwise interactions between words. However, human expression involves not only the selection of individual words, but also the habitual combination of multiple words into specific phrases. For example, individuals often use specific "buzzwords" and phrasings in their daily lives, which reflects the higher-order structure in the language. Compared with word co-occurrence networks, the use of text hypernetworks to describe information data in this method can better capture the higher-order structure of information data. In this way, the first hypernetwork constructed based on the information data to be analyzed can more accurately represent the actual structural features of the information data to be analyzed, so that the calculated structural feature indicators can better characterize the information of the information data to be analyzed.

[0075] In step 102, M structural feature indicators of the high-order text network corresponding to the information data to be analyzed are calculated to obtain a first indicator value corresponding to each structural feature indicator. In addition, M structural feature indicators of the high-order text network corresponding to the reference information data are calculated to obtain a second indicator value corresponding to each of the structural feature indicators. Among them, the structural feature indicator can be one indicator or multiple indicators. By measuring the structure of the text hypernetwork by different measurement methods, different structural feature indicators can be obtained.

[0076] For a text hypernetwork encoded by the correlation matrix H, the statistical distribution of its nodes or hyperedges can be described using a general paradigm as follows:

[0077] X=F(H ij );

[0078] Among them, X = [X1, X2, ...] is the statistical result set of structural characteristic indicators, and the element X iThe statistical result corresponding to the ith node or hyperedge. Exemplarily, the structural feature index may include at least one of the following:

[0079] Hyperdegress, the specific calculation method is as follows:

[0080]

[0081] Size of the hyperdegreee is calculated as follows:

[0082]

[0083] Average shortest path length, calculated as follows:

[0084]

[0085] Where d(i,j) is the shortest path length from node i to node j;

[0086] Intermittency is used to measure the number of words between the reappearance of the same word (or word group) in the information data. The specific calculation method is as follows:

[0087]

[0088] Where f is the number of repetitions of word i, t j is the number of words between the jth and i+1th occurrences of word i.

[0089] In other embodiments, the structural feature index may also include at least one of structural features such as closeness centrality, betweenness centrality and clustering coefficient, which is not specifically limited here.

[0090] As an optional implementation, the information data to be analyzed may be analyzed based on a single structural feature index. Optionally, in some embodiments, step 103 includes:

[0091] Determine a correlation threshold corresponding to a target structural feature indicator, wherein the target structural feature indicator is any one of the structural feature indicators;

[0092] Calculating the Pearson correlation coefficient between the information data to be analyzed and the reference information data based on the first index value and the second index value corresponding to the target structure characteristic index;

[0093] When the Pearson correlation coefficient is greater than the correlation threshold, it is determined that the information data to be analyzed is correlated with the reference information data; otherwise, it is determined that the information data to be analyzed is not correlated with the reference information data.

[0094] In this embodiment, an iterative method is proposed to obtain a statistical threshold value that can distinguish different categories under a specified structural feature index from a correlation matrix. Specifically, the correlation of different information data is calculated based on the statistical result of the specified structural feature index F, and a set of values ​​is identified, which makes the correlation of information data of the same author greater than the value and the correlation of information data of different authors less than the value, and the minimum value in the set of values ​​is the correlation threshold value corresponding to the structural feature index.

[0095] Optionally, determining a correlation threshold corresponding to the target structure feature indicator includes:

[0096] Constructing an indicator value set, the indicator value set comprising second indicator values ​​corresponding to the target structure feature indicators of a plurality of first reference information data, and second indicator values ​​corresponding to the target structure feature indicators of a plurality of second reference information data, the first reference information data being the reference information data belonging to the first category, and the second reference information data being the reference information data belonging to the second category;

[0097] Based on the indicator value set, calculating the Pearson correlation coefficient of the i-th first reference information data and the j-th second reference information data, where i and j are positive integers;

[0098] A correlation threshold corresponding to the target structure feature index is calculated based on the Pearson correlation coefficient.

[0099] Optionally, the correlation threshold k corresponding to the target structure feature index F for:

[0100]

[0101] Among them, a is used to represent the first category, c is used to represent the second category, a i The i-th first reference information data used to characterize the first category a, c j The j-th second reference information data used to characterize the second category c, A set of data used to represent the first reference information, A set of data used to represent the second reference information, To characterize a i and c j The Pearson correlation coefficient between .

[0102] The correlation threshold corresponding to the structural feature index F is denoted as kF , in k F With the help of , the relevance of different text hypernetworks can be evaluated. Specifically, if ρ ab >k F , then text hypernetwork a and text hypernetwork b are related under the statistical characteristics of the structural feature index F, otherwise, text hypernetwork a and text hypernetwork b are not related.

[0103] In specific implementation, the correlation thresholds corresponding to different structural feature indicators can be calculated, and the information data to be analyzed can be calculated based on different correlation thresholds to determine whether the information data to be analyzed is related to any information data under different structural feature indicators. When the information data to be analyzed is related to a certain information data, it can be considered that the information data to be analyzed and the information data belong to the same category.

[0104] The above embodiment provides a method for calculating the correlation between text hypernetworks under a single index. As another optional implementation, the correlation between the information data to be analyzed and the reference information data can be evaluated under M structural feature indexes. Optionally, in some embodiments, step 103 includes:

[0105] For each of the reference information data, respectively calculating the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators;

[0106] Based on the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators, the global correlation between the reference information data and the information data to be analyzed is determined, and the global correlation is used to characterize the degree of correlation between the reference information data and the information data to be analyzed.

[0107] It should be understood that in this embodiment, the categories of the reference information data are all known, and the category of the information data to be analyzed can be determined by determining the global correlation between the information data to be analyzed and the reference information data. For a given information data to be analyzed, the correlation between the information data to be analyzed and all reference information data under all structural feature indicators is calculated, and the results are compiled into a correlation matrix:

[0108]

[0109] Wherein, M is the number of structural feature indicators, and N is the number of reference information data. When N=1, the correlation matrix can be expressed as a correlation vector between two information data spanning M structural feature indicators, that is, the global correlation between the information data to be analyzed and a certain reference information data.

[0110] It should be understood that ρ ijIt is used to characterize the correlation between the information data to be analyzed and the jth reference information data under the i-th structural feature index. ij The calculation method of can refer to the calculation method of the second Pearson correlation coefficient in the above embodiment, and will not be described in detail here.

[0111] In the above embodiments, the correlation matrix ρ exists in an M×N dimensional structure. For better understanding, in some embodiments, this high-dimensional matrix is ​​compressed into a more practical and interpretable form, and an observable value R is introduced. j :

[0112]

[0113] The observable value Rj is a function used to represent the overall correlation between the information data to be analyzed and the jth reference information data. However, relying solely on linear global correlation often leads to predictions that are not sufficient to represent the true state. In order to enhance R j In some embodiments, the Shannon information entropy is used to obtain the following expression:

[0114]

[0115] The above expression can be used to calculate the observable value R j Rewrite it as follows:

[0116]

[0117] Among them, ρ ij Used to characterize the correlation between the information data to be analyzed and the jth reference information data under the i-th structural feature index.

[0118] According to the above content, in some embodiments, the global correlation between the information data to be analyzed and the reference information data can be calculated according to the following method:

[0119]

[0120] In this embodiment, a one-dimensional representation of a high-dimensional correlation matrix is ​​obtained, which encapsulates the global correlation between the information data to be analyzed and the reference information data. The global correlation can better evaluate the style similarity of the information data to be analyzed and the reference information data, and determine the most likely category of the information data to be analyzed based on the correlation score.

[0121] Optionally, in some embodiments, after determining the global correlation between the reference information data and the information data to be analyzed based on the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators, the method further includes:

[0122] The author of the reference information data having the greatest global correlation with the information data to be analyzed is determined as the author of the information data to be analyzed.

[0123] The following will take the application of the information analysis method based on high-order text networks provided by the present application in the scenario of author identification of unknown texts as an example to illustrate the beneficial effects of the information analysis method based on high-order text networks provided by the present application.

[0124] In the scenario of author identification of unknown text, the information data to be analyzed is anonymous text, the reference information data is existing text of known authors (called reference text set), and the category corresponding to the text is the author of the text. The anonymous text is analyzed by the information analysis method based on high-order text network provided by this application, and the obtained correlation analysis result (the category to which the anonymous text belongs) is used to characterize the most likely author of the anonymous text.

[0125] See also Figure 3a and Figure 3b In this embodiment, two text hypernetworks are constructed based on the first two thousand sentences in Charles Dickens' novels Oliver twist and The old curiosity shop. i is the supernode number of node i, which is used to represent the word frequency. Figure 3a The top ten most frequently appearing words in the text hypernetwork are given in Figure 1, and the arrows link the same words that appear in the two text hypernetworks. Figure 3a The difference between the first-order text hypernetwork in Figure 3b A second-order text hypernetwork corresponding to texts is constructed in , where two-word phrases appear in separate sentences. Figure 3a The word frequency distribution of two texts by the same author in the first-order text hypernetwork is shown in Figure 1. It can be observed that there is a significant similarity in the first-order word frequency distribution of the two texts. This is attributed to the consistent writing style of the texts by the same author. Figure 3a As shown, this phenomenon also occurs in second-order text hypernetworks.

[0126] By obtaining a reference text set consisting of 44 novels written by 11 famous writers, we found that in the first-order text hypernetwork, the distribution of degree, intermittency and average shortest path length is more dependent on the author, while other indicators such as centrality and clustering coefficient are less correlated with the author. In the second-order text hypernetwork, the degree distribution and intermittency of words are still strongly dependent on the author, while the average shortest path is no longer dependent on the author.

[0127] In this embodiment, hyperdegree, intermittency and average shortest path length are selected as structural feature indicators. Based on the above three structural feature indicators, the statistical characteristics of nodes and hyperedges in the text hypernetwork are captured and used to evaluate the relevance between texts. Figure 4a As shown in Figure 1, f1 and f2 are sequences of 555 common words and 98 common word pairs extracted from 44 novels by 11 authors, respectively. In the text of the same author, three statistical indicators are used to quantify the frequency and spatial distribution of words in the text: i ), intermittent (i i ) and the average shortest path length (p i )The three indicators show similarities in statistical results, but there are significant differences in the texts of different authors.

[0128] exist Figure 4a (a)-(c) in the figure are the statistical results of these indicators in the first-order text hypernetwork with the same author identity. The text hypernetwork originated from Charles Dickens' novels "Oliver twist" and "The old curiosityshop" and is used to show the situation of the same author. Figure 4a (d)-(f) in the figure are the statistical results of these indicators in the first-order text hypernetwork generated by Charles Dickens' novel Oliver Twist and Mark Twain's novel Rough Treatment, which are used to show the situation of different authors. Figure 4a (h)-(m) in are similar statistical features in the second-order text hypernetwork, and the details are not described here.

[0129] By calculating the Pearson correlation of these indicators in different texts, the correlation matrix of the text can be obtained. Figure 4b and Figure 4c As shown in Figure 2, these matrices can be used to quantitatively analyze the style characteristics of authors in text hypernetworks. Through the above experiments, the feasibility and effectiveness of analyzing the correlation between texts through structural feature indicators have been verified. Figure 4b The degree distribution of words in 44 novels was analyzed by Pearson correlation coefficient, where 11 authors each had 4 novels and the author IDs were in a predefined order. Figure 4c It is a similar statistical feature in the second-order text hypernetwork, and will not be described in detail here.

[0130] According to the above Figure 3a-Figure 4cFrom the content and related analysis shown, it can be seen that when the number of structural feature indicators is 1, a single structural feature indicator can be used to quantify the author's identity, that is, the category of anonymous text is analyzed by the correlation threshold in the aforementioned embodiment.

[0131] The feasibility and effectiveness of the classification of anonymous texts based on global correlation analysis will be verified below. Similar to the above embodiment, a reference text set consisting of 44 novels written by 11 famous writers is obtained, and then Thomas Hardy's novel "Life's Little Ironies" is used as an anonymous text for testing. Figure 5a As shown, in some embodiments, the results of all correlations can be formalized into a characteristic spectrum, where the x-axis represents the 44 novels, the y-axis represents the structural characteristic index in the text hypernetwork, and the cyan bars represent the correlation between the text hypernetworks. However, it is challenging to directly observe the most likely authorship of the test text from this spectrum.

[0132] Therefore, the method provided by this application uses the global correlation R to quantify Figure 5a The information effect of the results. Figure 5b As shown in Figure 2, we can see the overall correlation between the writing style of the anonymous text and the 11 authors in the reference text. Figure 5b The author with the highest R value in the dataset is author number 7, Thomas Hardy, who matches the actual author of the anonymous text “Life’s Little Ironies”.

[0133] Figure 5c The relationship between the range of R and M(n) is shown. M(n) represents the number of structural feature indicators with the same characteristics in two text hypernetworks. If the anonymous text is independent of the reference text in all structural feature indicators (i.e., M(n) = 0), then R is 0. On the contrary, when M(n) = 6, R reaches its maximum value at 2.6. In fact, this method does not pursue the highest R, but identifies the author through R. Figure 5b As shown, although the global correlation R has not yet reached its maximum value, it can still be used to distinguish authors.

[0134] Furthermore, the following is a test of the author identification of 170 novels written by 28 authors to verify the application effect of the method provided in the embodiment of the present application in the field of author identification. Figure 6a As shown in the figure, when only the first-order text hypernetwork features are used (i.e., only the pairwise relationships between words are considered), the accuracy of the method provided by the present application in identifying the author reaches 71%. When the features of the second-order text hypernetwork are included, the accuracy is significantly improved to 81%. It can be seen that combining the high-order features of the text hypernetwork can better capture the author's unique writing style, thereby improving the recognition accuracy.

[0135] In order to further verify the reliability of the method provided by this application in author identification, the precision, recall and F1-value of the test results of 28 authors are calculated below. e ),like Figure 6b These results show that the method provided by the present application is reliable in identifying most of the texts written by these authors. For most authors, the precision, recall and F1 value of the analysis results obtained by the method provided by the present application are relatively high.

[0136] In practice, different authors may have similar writing styles, and an author's style may change over time. Figure 7a As shown in the figure, there is a clear correlation between the novels of the Brontë sisters. Specifically, Charlotte Brontë's Jane Eyre is completely correlated with Emily Brontë's Wuthering Heights and Anne Brontë's Agnes Grey. Then, in this embodiment, Emily Brontë's Wuthering Heights is used as the text to be identified, and Charlotte Brontë's novels are used as reference texts for a more comprehensive analysis. The results are shown in the figure. Figure 7b As shown, in Figure 7b In , N represents the number of texts selected from Charlotte Brontë's texts, and ρ represents the correlation matrix between texts. Although the writing styles of Emily Brontë and Charlotte Brontë are completely correlated in individual novels, the correlation between their writing styles gradually weakens as more novels of Charlotte Brontë are included in the reference text set. This shows that considering a sufficient number of reference texts can improve the accuracy of the method provided by this application in author identification.

[0137] By modeling the correlation network of Mark Twain's novels from his early, middle and late periods, we can get Figure 7c and Figure 7d . Figure 7c The figure shows a similarity network of Mark Twain's styles in different periods, with colors distinguishing between Mark Twain's early, middle, and late texts (orange-red, cyan, and green). Figure 7d In the figure, the x-axis represents the sequence number of the four novels of Mark Twain selected as the test text. Yellow represents R calculated using Mark Twain's early and middle novels as training texts, and blue represents R calculated using Mark Twain's early and late novels as training texts. Figure 7c As shown in the figure, there is no correlation between Mark Twain's early and middle novels, while his late novels show correlation with the early and middle ones. This means that if Mark Twain's middle novels are used as reference texts and his early novels are used as texts to be identified, the global correlation R will be 0, which is not in line with the actual situation. Figure 7dIt is shown that choosing different reference texts will produce different analysis results. Therefore, in practical scenarios, it is crucial to consider the variation of authors' writing styles to improve the effectiveness of the proposed method in author identification.

[0138] According to the above content, it can be seen that in the method provided in this application, the use of hypernetworks to characterize the information data to be analyzed can better characterize more complex dependencies in the text, and use correlation to evaluate the similarity of text hypernetwork statistical features and quantify author identity, which can better capture deeper style patterns than simple network models, thereby more accurately identifying author identity.

[0139] See also Figure 8 The embodiment of the present invention further provides an information analysis device 800 based on a high-order text network, comprising:

[0140] A construction module 801 is used to construct a corresponding high-order text network based on the information data to be analyzed. In a k-order text hypernetwork, a node is used to represent a group of k words appearing in a sentence. A sentence is a hyperedge connecting multiple nodes, and k is a positive integer.

[0141] The calculation module 802 is used to calculate M structural feature indicators of the high-order text network corresponding to the information data to be analyzed, and obtain a first indicator value corresponding to each of the structural feature indicators, and calculate M structural feature indicators of the high-order text network corresponding to the reference information data, and obtain a second indicator value corresponding to each of the structural feature indicators, where M is a positive integer;

[0142] The analysis module 803 is used to perform information analysis based on the first indicator value and the second indicator value, and evaluate the correlation between the information data to be analyzed and the reference information data.

[0143] Optionally, the analysis module 803 includes:

[0144] A first determining unit is used to determine a correlation threshold corresponding to a target structural feature indicator, wherein the target structural feature indicator is any one of the structural feature indicators;

[0145] A first calculation unit, configured to calculate a Pearson correlation coefficient between the information data to be analyzed and the reference information data based on the first index value and the second index value corresponding to the target structure characteristic index;

[0146] A second determining unit is configured to determine that the information data to be analyzed is correlated with the reference information data if the Pearson correlation coefficient is greater than the correlation threshold, and otherwise determine that the information data to be analyzed is not correlated with the reference information data.

[0147] Optionally, the first determining unit is specifically configured to:

[0148] Constructing an indicator value set, the indicator value set comprising second indicator values ​​corresponding to the target structure feature indicators of a plurality of first reference information data, and second indicator values ​​corresponding to the target structure feature indicators of a plurality of second reference information data, the first reference information data being the reference information data belonging to the first category, and the second reference information data being the reference information data belonging to the second category;

[0149] Based on the indicator value set, calculating the Pearson correlation coefficient of the i-th first reference information data and the j-th second reference information data, where i and j are positive integers;

[0150] A correlation threshold corresponding to the target structure feature index is calculated based on the Pearson correlation coefficient.

[0151] Optionally, the correlation threshold k corresponding to the target structure feature index F for:

[0152]

[0153] Among them, a is used to represent the first category, c is used to represent the second category, a i The i-th first reference information data used to characterize the first category a, c j The second reference information data for characterizing the second category c, A set of data used to represent the first reference information, A set of data used to represent the second reference information, To characterize a i and c j The Pearson correlation coefficient between .

[0154] Optionally, the analysis module 803 includes:

[0155] A second calculation unit is used to calculate, for each of the reference information data, the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators;

[0156] A third determination unit is used to determine the global correlation between the reference information data and the information data to be analyzed based on the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators, wherein the global correlation is used to characterize the degree of correlation between the reference information data and the information data to be analyzed.

[0157] Optionally, the global correlation between the information data to be analyzed and the reference information data is:

[0158]

[0159] Among them, R j It is used to characterize the correlation between the information data to be analyzed and the j-th reference information data, R j for:

[0160]

[0161] Among them, M is used to characterize the number of structural characteristic indicators, ρ ij Used to characterize the correlation between the information data to be analyzed and the jth reference information data under the i-th structural feature index.

[0162] Optionally, the information analysis device 800 based on the high-order text network further includes:

[0163] The determination module is used to determine the author of the reference information data having the greatest global correlation with the information data to be analyzed as the author of the information data to be analyzed.

[0164] The information analysis device 800 based on a high-order text network provided in the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be repeated in this embodiment.

[0165] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0167] like Fig. 9 As shown, an embodiment of the present application provides an electronic device 900, including: a memory 902, a processor 901, and a program stored in the memory 902 and executable on the processor 901; the processor 901 is used to read the program in the memory 902 to implement the steps in the information analysis method based on the high-order text network as described above.

[0168] An embodiment of the present application also provides a readable storage medium, on which a program is stored. When the program is executed by a processor, the various processes of the above-mentioned information analysis method embodiment based on a high-order text network are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk ((Magneto-Optical Disr, MO)), etc.), optical storage (such as compact disk (Compact Disr, CD), digital video disk (Digital Versatile Disc, DVD), Blu-ray Disc (Blu-ray Disc, BD), high-definition universal disc (High-Definition Versatile Disc, HVD), etc.), and semiconductor memory (such as read-only memory (Read-Only Memory, ROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable programmable read only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state drive (Solid State Disr or Solid State Drive, SSD)), etc.

[0169] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0171] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An information analysis method based on a high-order text network, characterized in that: include: Construct a corresponding high-order text network based on the information data to be analyzed. In a k-order text hypernetwork, a node is used to represent a group of k words appearing in a sentence. A sentence is a hyperedge connecting multiple nodes, and k is a positive integer. Calculate M structural feature indicators of the high-order text network corresponding to the information data to be analyzed, and obtain a first indicator value corresponding to each of the structural feature indicators; and calculate M structural feature indicators of the high-order text network corresponding to the reference information data, and obtain a second indicator value corresponding to each of the structural feature indicators, where M is a positive integer; Information analysis is performed based on the first indicator value and the second indicator value to evaluate the correlation between the information data to be analyzed and the reference information data.

2. The method according to claim 1, characterized in that: The performing information analysis based on the first indicator value and the second indicator value to evaluate the correlation between the information data to be analyzed and the reference information data includes: Determine a correlation threshold corresponding to a target structural feature indicator, wherein the target structural feature indicator is any one of the structural feature indicators; Calculating the Pearson correlation coefficient between the information data to be analyzed and the reference information data based on the first index value and the second index value corresponding to the target structure characteristic index; When the Pearson correlation coefficient is greater than the correlation threshold, it is determined that the information data to be analyzed is correlated with the reference information data; otherwise, it is determined that the information data to be analyzed is not correlated with the reference information data.

3. The method according to claim 2, characterized in that: The step of determining a correlation threshold corresponding to the target structure characteristic index includes: Constructing an indicator value set, the indicator value set comprising second indicator values ​​corresponding to the target structure feature indicators of a plurality of first reference information data, and second indicator values ​​corresponding to the target structure feature indicators of a plurality of second reference information data, the first reference information data being the reference information data belonging to the first category, and the second reference information data being the reference information data belonging to the second category; Based on the indicator value set, calculating the Pearson correlation coefficient of the i-th first reference information data and the j-th second reference information data, where i and j are positive integers; A correlation threshold corresponding to the target structure feature index is calculated based on the Pearson correlation coefficient.

4. The method according to claim 3, characterized in that: The correlation threshold k corresponding to the target structure feature index F for: Among them, a is used to represent the first category, c is used to represent the second category, a i The i-th first reference information data used to characterize the first category a, c j The j-th second reference information data used to characterize the second category c, A set of data used to represent the first reference information, A set of data used to represent the second reference information, To characterize a i and c j The Pearson correlation coefficient between .

5. The method according to claim 1, characterized in that: The performing information analysis based on the first indicator value and the second indicator value to evaluate the correlation between the information data to be analyzed and the reference information data includes: For each of the reference information data, respectively calculating the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators; Based on the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators, the global correlation between the reference information data and the information data to be analyzed is determined, and the global correlation is used to characterize the degree of correlation between the reference information data and the information data to be analyzed.

6. The method according to claim 5, characterized in that: The global correlation between the information data to be analyzed and the reference information data is: Among them, R j It is used to characterize the correlation between the information data to be analyzed and the j-th reference information data, R j for: Among them, M is used to characterize the number of structural characteristic indicators, ρ ij Used to characterize the correlation between the information data to be analyzed and the jth reference information data under the i-th structural feature index.

7. The method according to claim 5, characterized in that: After determining the global correlation between the reference information data and the information data to be analyzed based on the correlation between the reference information data and the information data to be analyzed under the M structural feature indicators, the method further includes: The author of the reference information data having the greatest global correlation with the information data to be analyzed is determined as the author of the information data to be analyzed.

8. An information analysis device based on a high-order text network, characterized in that: include: A construction module is used to construct a corresponding high-order text network based on the information data to be analyzed. In a k-order text hypernetwork, a node is used to represent a group of k words appearing in a sentence. A sentence is a hyperedge connecting multiple nodes, and k is a positive integer. A calculation module, used to calculate M structural feature indicators of the high-order text network corresponding to the information data to be analyzed, and obtain a first indicator value corresponding to each of the structural feature indicators, and calculate M structural feature indicators of the high-order text network corresponding to the reference information data, and obtain a second indicator value corresponding to each of the structural feature indicators, where M is a positive integer; An analysis module is used to perform information analysis based on the first indicator value and the second indicator value, and evaluate the correlation between the information data to be analyzed and the reference information data.

9. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the information analysis method based on a high-order text network as described in any one of claims 1 to 7.

10. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps in the information analysis method based on a high-order text network as described in any one of claims 1 to 7 are implemented.

Images