Text analysis method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a text analysis method and device, electronic equipment and storage medium, and relate to the technical field of data mining. The method comprises: constructing a text citation network corresponding to a set of texts to be analyzed based on the citation relationship between each text in the set of texts to be analyzed in a target field; selecting a plurality of candidate paths from the text citation network; performing at least one clustering operation on the plurality of candidate paths until a preset ending condition is met, and at least two clustering centers obtained when the preset ending condition is met are taken as knowledge evolution paths corresponding to at least two subfields in the target field respectively. Through analysis of the set of texts to be analyzed in the target field, embodiments of the present application obtain the development context corresponding to each subfield in the target field, and the analysis result is more comprehensive; the text semantic information and network structure information of the candidate paths are considered, the information utilization is more comprehensive, and the analysis result is more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data mining, in particular, the present application relates to a text analysis method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the rapid development of science and technology, the number of research results (such as scientific and technological literature, patents, etc.) is also increasing. Mining potential knowledge evolution patterns from massive scientific research results is an important means to understand the development of science and technology and predict future trends in science and technology.

[0003] Main-path analysis is a common quantitative method for technology evolution analysis. Main-path analysis uses citation networks to represent the diffusion of knowledge between documents, and extracts the skeleton structure between important documents from the citation network to represent the main development dynamics of a specific field.

[0004] Currently, the main-path analysis method usually takes the path with the maximum weight in the citation network as the main path. The main path obtained tends to focus on a subfield in the current subject field, thereby ignoring the development trajectory of other subfields that are equally important. The analysis result is not comprehensive, and the accuracy is low. SUMMARY

[0005] The embodiments of the present application provide a text analysis method, device, electronic equipment and storage medium, which can solve the problem of the existing main-path analysis method that the analysis result is not comprehensive and the accuracy is low.

[0006] The technical solution is as follows:

[0007] According to an aspect of an embodiment of the present application, a text analysis method is provided, which comprises:

[0008] Based on the citation relationship between each text in the target field text set to be analyzed, a text citation network corresponding to the text set to be analyzed is constructed;

[0009] Selecting a plurality of candidate paths from the text citation network;

[0010] Performing at least one clustering operation on the plurality of candidate paths until a preset end condition is met, and obtaining at least two cluster centers when the preset end condition is met, as knowledge evolution paths corresponding to at least two subfields in the target field respectively;

[0011] The clustering operation comprises:

[0012] For each candidate path, a first parameter corresponding to the current clustering operation is determined; the first parameter is positively correlated with the number of clustering operations.

[0013] According to the initial density of the candidate paths, the first topological weight of the candidate paths, and the first parameter, a first density of the candidate paths is calculated;

[0014] Based on the first density corresponding to each candidate path respectively, at least two clustering centers obtained by clustering the candidate paths are determined.

[0015] Optionally, the calculation of the first density of the candidate paths according to the initial density of the candidate paths, the first topological weight of the candidate paths, and the first parameter comprises:

[0016] The product of the first parameter and the first topological weight of the candidate paths is taken as a second parameter;

[0017] The sum of the second parameter and the initial density of the candidate paths is taken as the first density of the candidate paths.

[0018] Optionally, the selection of the candidate paths from the text citation network comprises:

[0019] Each source node in the text citation network is determined;

[0020] For each source node, a maximum weight path corresponding to the source node is determined;

[0021] Based on the maximum weight path corresponding to each source node respectively, the candidate paths are determined.

[0022] For each destination node corresponding to the source node, a path with the maximum path weight in at least one first path from the source node to the destination node is taken as a candidate path.

[0023] Optionally, the determination of the maximum weight path corresponding to each source node comprises:

[0024] Each node in the text citation network except the source node is taken as a target node, and the citation level of each target node to the source node is determined;

[0025] The optimal paths from the source node to each target node are determined in turn according to the order of the citation level in turn increasing;

[0026] The maximum weight path is determined from the optimal paths of each target node;

[0027] For each target node, the determination of the optimal path from the source node to the target node comprises:

[0028] At least one initial path from the source node to the target node is determined.

[0029] The optimal path is determined according to the path weight corresponding to each initial path.

[0030] Optionally, the selecting a plurality of candidate paths from the text citation network comprises:

[0031] determining the path weight corresponding to each path in the text citation network;

[0032] selecting a preset number of paths with the largest path weight as the candidate paths based on the path weight corresponding to each path.

[0033] Optionally, the path weight is determined based on the following steps:

[0034] for each path in the text citation network, determining a first topological weight of the path based on the second topological weight of each edge included in the path;

[0035] determining a first semantic weight of the path based on the text similarity between each node pair in the path; the node pair is any combination of two nodes in the path;

[0036] determining the path weight of the path based on the first topological weight of the path and the first semantic weight of the path.

[0037] Optionally, the constructing a text citation network corresponding to the set of texts to be analyzed based on the citation relationship between each text in the set of texts to be analyzed in the target field comprises:

[0038] taking each text in the set of texts to be analyzed in the target field as a node;

[0039] establishing a connection relationship between each node based on the citation relationship between each text in the set of texts to be analyzed; the direction of the edge in the text citation network is from the cited text to the cited text.

[0040] According to another aspect of the embodiments of the present application, a text analysis device is provided, which comprises:

[0041] a network construction module configured to construct a text citation network corresponding to the set of texts to be analyzed based on the citation relationship between each text in the set of texts to be analyzed in the target field;

[0042] a candidate path determination module configured to select a plurality of candidate paths from the text citation network;

[0043] a clustering module configured to perform at least one clustering operation on the plurality of candidate paths until a preset ending condition is met, and obtain at least two clustering centers as knowledge evolution paths corresponding to at least two sub-fields in the target field respectively when the preset ending condition is met;

[0044] The clustering operation comprises:

[0045] For each candidate path, a first parameter corresponding to the current clustering operation is determined, and the first parameter is positively correlated with the number of the clustering operations;

[0046] According to the initial density of the candidate path, the first topological weight of the candidate path, and the first parameter, a first density of the candidate path is calculated.

[0047] At least two clustering centers obtained by clustering the plurality of candidate paths are determined based on the first densities of the plurality of candidate paths respectively.

[0048] According to another aspect of the embodiments of the present application, an electronic device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of any of the above text analysis methods when executing the program.

[0049] According to still another aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program, and the computer program is executable on a processor to implement the steps of any of the above text analysis methods.

[0050] The technical scheme provided by the embodiments of the present application has the following beneficial effects:

[0051] By constructing a text citation network based on the citation relationships between the texts in the set of texts to be analyzed, and selecting a plurality of candidate paths from the text citation network, at least two clustering centers obtained by clustering the plurality of candidate paths are taken as knowledge evolution paths corresponding to at least two sub-fields respectively, and the development context of each sub-field in the target field is obtained by analyzing the set of texts to be analyzed in the target field, so that the analysis result is more comprehensive.

[0052] Meanwhile, in the process of clustering the plurality of candidate paths, both the text semantic information of the candidate paths and the network structure information of the candidate paths are considered, so that the information utilization is more comprehensive, the accuracy of the clustering result is improved, and the analysis result is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical schemes in the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced.

[0054] Figure 1 A flowchart of a text analysis method provided by an embodiment of the present application is shown in FIG. 1.

[0055] Figure 2 A schematic diagram of a text citation network provided by an embodiment of the present application is shown in FIG. 2.

[0056] Figure 3 A schematic diagram of a candidate path search process provided by an embodiment of the present application is shown in FIG. 3.

[0057] Figure 4 A flowchart of a text analysis method provided by another embodiment of the present application is shown in FIG. 4.

[0058] Figure 5 A structural schematic diagram of a text analysis device provided by an embodiment of the present application is shown in FIG. 5.

[0059] Figure 6 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 6. DETAILED DESCRIPTION

[0060] Embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions of the technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.

[0061] Those skilled in the art can understand that the singular forms "a", "an" and "the" used herein include plural forms unless specifically stated otherwise. It should be further understood that the terms "comprise" and "include" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the present technology. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can mean that the element and the other element are connected through an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The term "and / or" used herein means that at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B".

[0062] To make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0063] Main-path Analysis is a common quantitative method for technology evolution analysis. It was first proposed by Hummon and Doreian in 1982. They believed that different citation relationships in the citation network are not equal in status. Important citation relationships act as the main path, and removing it will change the whole knowledge flow process in the citation network. In contrast, the knowledge flow carried by secondary citation relationships and the impact on the network are much smaller. Therefore, the "main path" represents the backbone of the citation network, containing the key links of knowledge development. Hummon et al. called this main path the main path. Through the main path method, technology evolution research can help researchers solve the problem of data overload in large patent citation networks, focus their attention on citation sequences composed of a small number of documents, and thus more easily obtain some valuable information.

[0064] In 2003, V. Batagelj first applied the main path algorithm to patent citation network analysis, but the reason was that the patent citation network was one of the few large networks available at the time. Until 2007, B. Verspagen et al. truly applied the main path method to technology evolution analysis based on the exploration of the similarity between patent citations and paper citations. He noticed that the original main path search algorithm used a greedy strategy. This method of selecting the main path by solving the local optimum may not be the global optimal solution. Therefore, he proposed a new main path search strategy to ensure global optimization: using the exhaustive method to extract all paths from the source point to the sink point, and selecting the path with the maximum arc weight as the main path. J. S. Liu et al. called the main path obtained based on the greedy strategy the local main path, and called the main path obtained by B. Verspagan the global main path.

[0065] Main path analysis method usually selects the path with the maximum path weight in the citation network as the main path. The main path obtained often focuses on a subfield in the current subject field, thereby ignoring the development trajectory of other subfields of equal importance. The analysis result is not comprehensive and has low accuracy.

[0066] The text analysis method, device, electronic equipment and storage medium provided in the present application aim to solve the above technical problems in the prior art.

[0067] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0068] Figure 1 This is a flowchart illustrating a text analysis method provided in an embodiment of this application, such as... Figure 1 As shown, the method includes:

[0069] Step S101: Based on the reference relationships between the texts in the text set to be analyzed in the target domain, construct the text reference network corresponding to the text set to be analyzed.

[0070] Specifically, the target domain can be the subject area that needs to be analyzed, and the text set to be analyzed can be a text set used to analyze the knowledge evolution of the target domain. The text set to be analyzed can include multiple texts in that domain, and there are citation relationships between the multiple texts. The texts can be scientific papers or patent texts.

[0071] After determining the text set to be analyzed, a corresponding text reference network can be constructed based on the reference relationships between the texts in the text set.

[0072] Alternatively, the above text citation network can be constructed in the following way:

[0073] Each text in the target domain's set of texts to be analyzed is treated as a node;

[0074] Based on the reference relationships between various texts in the text set to be analyzed, the corresponding connection relationships between various nodes are established; the direction of the edges in the text reference network is from the referencing text to the referenced text.

[0075] Specifically, the text citation network can be a directed graph, which may include multiple nodes and directed edges between nodes. Each text in the text set to be analyzed corresponds to a node in the text citation network, and each node in the text citation network can be a vector representation of the corresponding text, used to represent the semantic information of the corresponding text.

[0076] On this basis, the connection relationship between each node corresponding to each text in the text citation network can be established according to the citation relationship between each text in the text set to be analyzed. When there is a citation relationship between two texts, an edge connecting the two nodes corresponding to the two texts can be obtained, and the direction of the edge is from the node corresponding to the citing text to the node corresponding to the cited text. For example, node a and node b correspond to text A and text B respectively, text A cites text B, and the text citation network includes a directed edge from node a to node b.

[0077] Step S102, selecting a plurality of candidate paths from the text citation network.

[0078] Specifically, the text citation network includes a plurality of paths, and each path can be used to represent a knowledge evolution context. The candidate paths can be a plurality of important paths in the text citation network.

[0079] Each path in the text citation network has a corresponding path weight, and the path weight can be used to represent the importance of the corresponding path. The greater the path weight, the more important the corresponding path.

[0080] The plurality of candidate paths can be selected from the plurality of paths in the text citation network according to the path weights corresponding to the plurality of paths. The selection method of the candidate paths will be described in detail below.

[0081] Step S103, performing at least one clustering operation on the plurality of candidate paths until a preset ending condition is met, and obtaining at least two cluster centers when the preset ending condition is met as knowledge evolution paths corresponding to at least two sub-fields in the target field respectively;

[0082] The clustering operation includes:

[0083] For each candidate path, a first parameter corresponding to the current clustering operation is determined. The first parameter is positively correlated with the number of clustering operations;

[0084] According to the initial density of the candidate path, the first topological weight of the candidate path, and the first parameter, the first density of the candidate path is calculated.

[0085] Based on the first density corresponding to each candidate path respectively, at least two cluster centers obtained by clustering each candidate path are determined.

[0086] Specifically, in order to obtain the knowledge evolution context corresponding to each sub-field in the target field, the plurality of candidate paths need to be further screened.

[0087] For each subfield in the target field, the knowledge evolution context in different subfields can be different, that is, the candidate paths corresponding to different subfields differ greatly; the knowledge evolution context in the same subfield is usually consistent, that is, the candidate paths corresponding to the same subfield differ less. According to the distribution characteristics of the candidate paths corresponding to each subfield, each candidate path can be taken as a clustering point, and multiple clustering operations are repeatedly performed on multiple candidate paths to realize clustering of the multiple candidate paths.

[0088] Before performing the clustering operation, the semantic representation corresponding to each candidate path can also be determined, and the semantic representation corresponding to each candidate path is taken as a plurality of clustering points in the semantic space. The semantic representation of the candidate path can be used to represent the semantic information of each text included in the corresponding candidate path, for example, the vector representation of all nodes in the candidate path can be accumulated and normalized as the semantic representation of the candidate path.

[0089] Each clustering operation can include the following steps:

[0090] (1) For each candidate path, a first parameter corresponding to the current clustering operation is determined. The first parameter is positively correlated with the number of clustering operations, that is, the more the number of clustering operations, the larger the first parameter.

[0091] (2) For each candidate path, the initial density of the candidate path is adjusted according to the first topological weight of the candidate path and the first parameter of the current clustering operation to obtain the first density of the candidate path.

[0092] The initial density of the candidate path can be the local density of the clustering point corresponding to the candidate path in the original distribution state before the first clustering operation. The first topological weight of the candidate path can be used to represent the network structure information of the candidate path, and the first topological weight of the candidate path can be the sum of the topological weights of the edges on the candidate path.

[0093] (3) According to the first density corresponding to each candidate path and the relative distance between each candidate path, the multiple candidate paths are clustered to form multiple clusters, at least two target clusters are selected from the multiple clusters, and at least two clustering centers corresponding to at least two target clusters are obtained.

[0094] The relative distance between any two candidate paths can be a cosine distance, which is used to reflect the semantic difference between the two corresponding candidate paths. The target cluster can be the main cluster in the multiple clusters, for example, a cluster whose number of clustering points is greater than a preset threshold can be taken as a target cluster.

[0095] In the embodiments of the present application, clustering of the candidate paths is performed in a semantic space, and the clustering points are semantic representations of the candidate paths, that is, the initial density of the candidate paths can reflect semantic information of the candidate paths in the text; the first topological weight of the candidate paths can reflect information of the candidate paths in the network structure, and the first parameter can be used to adjust the proportion between the initial density of the candidate paths and the first topological weight, that is, the proportion between the semantic information of the candidate paths and the network structure information.

[0096] The first parameter is repeatedly updated by repeatedly performing the above clustering operation, and the proportion between the initial density of the candidate paths and the first topological weight is continuously adjusted until a preset ending condition is met. The preset ending condition can be clustering convergence. For example, a difference between the clustering center obtained in the last clustering operation and the clustering center obtained in the current clustering operation can be calculated. If the difference is less than a preset threshold, it indicates that the clustering center tends to be stable, the clustering converges, and the clustering operation can be ended.

[0097] The at least two clustering centers obtained when the preset ending condition is met are taken as the knowledge evolution paths corresponding to the at least two sub-fields respectively.

[0098] In the process of clustering the plurality of candidate paths, both the text semantic information of the candidate paths and the network structure information of the candidate paths are considered, different aspects of information of the candidate paths are fully utilized, the clustering centers obtained by clustering can better represent the corresponding clusters, and the differences between the clustering centers of different clusters are more obvious, thereby improving the accuracy of the clustering result.

[0099] The method provided in the embodiments of the present application constructs a text citation network based on the citation relationship between each text in the set of texts to be analyzed, selects a plurality of candidate paths from the text citation network, clusters the plurality of candidate paths, takes at least two clustering centers obtained by clustering as knowledge evolution paths corresponding to at least two sub-fields respectively, and obtains the development context of each sub-field in the target field by analyzing the set of texts to be analyzed in the target field. The analysis result is more comprehensive.

[0100] Meanwhile, in the process of clustering the plurality of candidate paths, both the text semantic information of the candidate paths and the network structure information of the candidate paths are considered, information utilization is more comprehensive, the accuracy of the clustering result is improved, and the analysis result is more accurate.

[0101] As an optional embodiment, the first density of the candidate paths is calculated according to the initial density of the candidate paths, the first topological weight of the candidate paths, and the first parameter in step S103, including:

[0102] The product of the first parameter and the first topological weight of the candidate paths is taken as the second parameter;

[0103] The second parameter is summed with the initial density of the candidate path, as the first density of the candidate path. Specifically, the first density rho' of the candidate path can be calculated based on the following formula:

[0104] rho' = rho + γ * weight p,t

[0105] In the formula, rho is the initial density, which can be calculated based on a density peak clustering algorithm (Density Peak Cluster), weight p,t is the first topological weight of the candidate path, and γ is the first parameter, used to adjust the proportion of the first topological weight of the candidate path.

[0106] Optionally, in the process of clustering the plurality of candidate paths, the first parameter γ value can be set to gradually increase from an initial value by a preset step, that is, each time the clustering operation is performed, the first parameter increases by a preset step. For example, the initial value is 1, and the preset step is 1, then the first parameter corresponding to the first clustering operation is the initial value 1; the first parameter corresponding to the second clustering operation is the initial value 1 plus the preset step 1, which is 2; the subsequent clustering operations can update the first parameter in the same way as described above.

[0107] As an optional embodiment, step 102 specifically includes:

[0108] determining each source node in the text citation network;

[0109] for each source node, determining the maximum weight path corresponding to the source node;

[0110] based on the maximum weight paths respectively corresponding to each source node, determining the candidate path.

[0111] Specifically, to select the candidate path from the plurality of paths in the text citation network, each source node in the text citation network can be determined first, and the source node can be a node with an in-degree of zero in the text citation network, that is, the source node does not exist a directed edge pointing to itself. For each source node, a plurality of paths can be derived from the source node as a starting point, and the path with the maximum path weight in the plurality of paths with the source node as a starting point can be selected as the maximum weight path corresponding to the source node.

[0112] The candidate path can be selected from the maximum weight paths respectively corresponding to each source node, for example, the maximum weight paths can be sorted in descending order of path weight, and the top several maximum weight paths in the sorting can be selected as the candidate path.

[0113] In the embodiments of the present application, by determining each source node in the text citation network and the maximum weight path corresponding to each source node respectively, a candidate path is selected from each maximum weight candidate path, which can not only ensure that the selected candidate path contains many nodes and has a large path weight, but also ensure the difference between different candidate paths, which is helpful for further screening the knowledge evolution context corresponding to different subfields.

[0114] As an optional embodiment, for each source node, determining the maximum weight path corresponding to the source node comprises:

[0115] Taking each node in the text citation network except the source node as a target node, determining the citation level of each target node to the source node;

[0116] In order to the optimal path from the source node to each target node in turn according to the order of the citation level in turn increasing;

[0117] Determining the maximum weight path from each optimal path of the target node;

[0118] Wherein, for each target node, determining the optimal path from the source node to the target node comprises:

[0119] Determining at least one initial path from the source node to the target node;

[0120] Determining the optimal path according to the path weight corresponding to each initial path respectively.

[0121] Specifically, for each source node, a plurality of paths leading out of the source node are determined, and the nodes in these paths except the source node are taken as target nodes, which can be nodes having a direct or indirect citation relationship with the source node; the citation level of each target node to the source node is determined, and for a target node, the citation level of the target node to the source node can be the number of edges from the target node backtracking to the source node.

[0122] Figure 2 A schematic diagram of a text citation network provided by the embodiments of the present application is shown in FIG. 1, where node 1 is a source node, and nodes 2-7 are target nodes. Node 2 and node 3 directly cite node 1, i.e., node 2 and node 3 are directly connected to node 1, so the citation level of node 2 and node 3 is 1, the citation level of node 4 and node 5 is 2, and the citation level of node 6 and node 7 is 3. Figure 2

[0123] ​After determining the reference levels of the target nodes to the source node, the target nodes with the same reference level can be grouped as a group, and each group can be sorted in the order of the reference level from small to large, and then the optimal path from the source node to the target nodes in the first group, the optimal path from the source node to the target nodes in the second group, and so on are determined in turn, and finally the optimal paths from the source node to the target nodes are obtained. The optimal path from the source node to the target node can be the path with the maximum path weight in the path from the source node to the target node.

[0124] The following will be described in combination with Figure 2 , Figure 2 The reference levels of the node 2 and the node 3 are both 1, the reference levels of the node 4 and the node 5 are both 2, and the reference levels of the node 6 and the node 7 are both 3, so the two optimal paths from the node 1 to the node 2 and the node 3 can be determined first, then the two optimal paths from the node 1 to the node 4 and the node 5 can be determined, and finally the two optimal paths from the node 1 to the node 6 and the node 7 can be determined. Thus, the six optimal paths from the node 1 to the six target nodes can be obtained.

[0125] For a target node, at least one initial path from the source node to the target node can be determined, and the path weights of the initial paths are determined, and the initial path with the maximum path weight is taken as the optimal path from the source node to the target node. The following will be described in combination with Figure 2 the examples given, Figure 2 The initial path 1: 1→2→4 and the initial path 2: 1→3→4 exist from the node 1 to the node 4, the path weights of the initial path 1 and the initial path 2 are calculated, and the initial path with the greater path weight is taken as the optimal path from the node 1 to the node 4. For example, the path weight of the initial path 1 is 1, and the path weight of the initial path 2 is 0.9, so the initial path 1 is taken as the optimal path from the node 1 to the node 4.

[0126] The optimal path from the source node to each target node is the path with the maximum path weight from the source node to each target node, and then the maximum weight path can be determined from the optimal paths, for example, the optimal path with the maximum path weight can be taken as the maximum weight path.

[0127] The method provided by the embodiments of the present application can determine the reference levels of the target nodes to the source node, and then determine the optimal paths from the source node to the target nodes in turn in the order of the reference levels in turn, and then determine the candidate paths from the optimal paths of the target nodes, which can avoid the problem that the paths output by the local path search algorithm can be non-global optimal, and can also avoid the problem of high time complexity caused by the global path search algorithm by enumerating all paths, and improve the efficiency of path search on the premise that the candidate paths are global optimal paths.

[0128] As an optional embodiment, the candidate path is determined based on the following way:

[0129] (1) Taking the source node s as the first traversal point, taking the adjacent node of the source node in the referenced direction as the current node v visited in the current time, for each visited current node v, taking the node as the key, taking the link s→v from s to v and the path weight weight(s, v) of the link as the value and saving to the dictionary table, i.e. (s→v, weight(s, v))→Dictionary[v];

[0130] (2) Taking the adjacent node of the current node in the referenced direction as the current node and continuing the breadth-first traversal;

[0131] (3) If the current node v already exists as the key in the dictionary table, taking out the path path sv,1 already stored in the dictionary table, and according to the traversal trace to find the current path path sv,2 from the source node s to v, and comparing the path weights of the two paths. If the path weight of path sv,2 is greater than that of path sv,1 , then taking the binary tuple composed of path sv,2 and the corresponding path weight weight'(s, v) as the value and saving to the position corresponding to the key v in the dictionary table, i.e. (path sv,2 , weight'(s, v))→Dictionary[v], otherwise keeping the key v and the corresponding binary tuple unchanged;

[0132] (4) Traversing in this way until the end, at this time the dictionary table saves the path set from the source node s to all reachable target nodes, and each path is the global optimal path from the source node to the corresponding target node.

[0133] The method provided by the embodiment of the present application will be further described below in combination with the text reference network shown in Figure 2 . Table 1 is the text similarity between each node in the text reference network of Figure 2 . In this example, the path weight is calculated by the text similarity between all node pairs in the path, and the node pair is the combination of any two nodes on the path. It should be noted that the above path weight calculation method is for facilitating the description of the search process of the candidate path, and does not constitute a limitation on the calculation method of the path weight, which will be described in detail below.

[0134] Table 1

[0135]

[0136] In Figure 2 and Table 1, Figure 3 A schematic diagram of a candidate path search process provided by the embodiment of the present application is shown in Figure 1. Figure 3 As shown in the figure, the search process of the candidate path includes:

[0137] In step (1), the breadth-first traversal is performed from the source node 1 to obtain its adjacent nodes 2 and 3, and the nodes are respectively put into the dictionary table. Taking the node 2 as an example, the node 2 is taken as the key, and the binary tuple composed of the connection line from the node 1 to the node 2 and the weight thereof is taken as the value and stored in the dictionary table.

[0138] In step (2), the adjacent node 4 of the node 2 is obtained, and the path weight of the path 1→2→4 is determined. The node 4 is taken as the key, and the binary tuple composed of the path 1→2→4 and the path weight thereof is taken as the value and stored in the dictionary table.

[0139] In step (3), the adjacent nodes 4 and 5 of the node 3 are obtained. Since the node 4 already exists as the key value in the dictionary table, the weight of the new path 1→3→4 from the source node 1 to the node 4 needs to be calculated, and a weight comparison is performed with the original path 1→2→4. Since the latter is larger, the value corresponding to the key 4 in the dictionary table remains unchanged, otherwise the original value is replaced by the path 1→3→4 and its weight, and then the node 5 is taken as the key and the binary tuple composed of the path 1→3→5 and its weight is taken as the value and stored in the dictionary table.

[0140] In steps (4) and (5), the process is continued in the same way, and finally the path 1→3→5→6 with the highest weight is obtained, and the maximum weight path corresponding to the source node 1 is obtained.

[0141] As an optional embodiment, step 102 specifically includes:

[0142] determining the path weights respectively corresponding to the paths in the text citation network;

[0143] selecting a preset number of paths with the maximum path weights as the candidate paths based on the path weights respectively corresponding to the paths.

[0144] Specifically, the text citation network includes a plurality of paths, and each path has a corresponding path weight. The path weight can include information of knowledge flow on the corresponding path and / or semantic information of texts on the path. The path weight can be determined based on a semantic weight of the path, or determined based on a topological weight of the path, or determined based on the semantic weight and the topological weight of the path, and the embodiment of the present application does not limit this.

[0145] After obtaining the path weights respectively corresponding to each path in the text citation network, the path weights respectively corresponding to each path can be sorted in descending order, and a preset number of paths with high ranking can be selected as candidate paths.

[0146] As an optional embodiment, determining the path weights respectively corresponding to each path in the text citation network comprises:

[0147] For each path in the text citation network, a first topological weight of the path is determined based on the second topological weights of edges included in the path.

[0148] A first semantic weight of the path is determined based on text similarity between each node pair in the path; the node pair is a combination of any two nodes in the path.

[0149] The path weight of the path is determined based on the first topological weight of the path and the first semantic weight of the path.

[0150] Specifically, for each path in the text citation network, the first topological weight of the path can be calculated based on the second topological weights of edges included in the path. For example, the sum of the second topological weights of edges included in the path can be taken as the first topological weight of the path.

[0151] The topological weight is used to reflect the network structure information of the corresponding edge or path, the first topological weight can be the topological weight corresponding to the path, and the second topological weight can be the topological weight corresponding to the edge. The topological weight of an edge can represent the amount of knowledge passing through the edge.

[0152] The second topological weights of edges in the text citation network can be calculated based on SPC (Search Path Count), SPNP (Search Path Node Pair), SPLC (Search Path Link Count), NPPC (Node Pair Projection Count), and the like.

[0153] For each path in the text citation network, the first semantic weight of the path is determined based on the text similarity between each node pair included in the path; the node pair is a combination of any two nodes in the path. The first semantic weight of the path is used to represent the semantic information of the text on the path. The text similarity between a node pair can be used to represent the similarity between the text information of the two nodes included in the node pair. The text similarity can be cosine similarity, Euclidean distance, or Pearson correlation coefficient, etc.

[0154] For example, Figure 2For example, in the path 1→2→4, the first semantic weight of the path includes the text similarity weight s (1,2) between the node pair (1,2), the text similarity weight s (2,4) between the node pair (2,4), and the text similarity weight s (1,4) between the node pair (1,4).

[0155] In the path 1→2→4, the node 2 is textually similar to the nodes 1 and 4 respectively, but the similar content of the node 2 and the node 1 is different from the similar content of the node 2 and the node 4, which leads to that the node 1 is not similar to the node 4 in the text semantics, and thus the theme of the text actually changes greatly from the node 1 to the node 4. In the embodiment of the present application, when the first semantic weight of the path is calculated, the change trend of the theme of each text on the path is fully considered by increasing the text similarity between the node 1 and the node 4, so as to ensure the consistency of the semantics of each text on the path and prevent the theme of the text from deviating in the path search process.

[0156] For each path in the text citation network, after the first topological weight and the first semantic weight of the path are determined, the path weight of the path can be calculated in combination with the first topological weight and the first semantic weight.

[0157] Optionally, a topological weight coefficient corresponding to the first topological weight and a semantic weight coefficient corresponding to the first semantic weight can be determined, and the path weight is calculated based on the first topological weight and the topological weight coefficient corresponding thereto, and the first semantic weight and the semantic weight coefficient corresponding thereto. The topological weight coefficient and the semantic weight coefficient can be analyzed and adjusted according to the application scenario and the actual result.

[0158] The path weight W path can be calculated by the following formula:

[0159]

[0160] In the formula, W path is the weight of the path path, i and j are the serial numbers of the nodes on the path, weight s (node i ,node j ) is the text similarity between the nodes node i and node j , weight t (edge i,j ) is the network structure weight of the edge edge i,j , and α is the semantic weight coefficient and β is the topological weight coefficient.

[0161] Taking path 1→2→4 as an example, and α, β being 1, 2, the path weight is 1*[weight s (1,2)+weight s (2,4)+weight s (1,4)]+2*[weight t (1,2)+

[0162] weight t (2,4)].

[0163] The method provided by the embodiments of the present application determines the first semantic weight of the path based on the text similarity between each node pair included in the path, fully considers the change trend of the theme of each text on the path, guarantees the consistency of each text on the path in terms of semantics, and prevents the theme of the text from deviating in the path search process; determines the path weight of the path based on the first topological weight of the path and the first semantic weight of the path, so that the path weight not only contains the semantic information of the text on the path, but also includes the network structure information of the path, so that the main path selected according to the path weight not only carries the main traffic of knowledge propagation in the text citation network, but also enables each node included in the main path to focus on a clear technical direction and accurately reflect the development context of knowledge evolution.

[0164] Figure 4 A flowchart of a text analysis method provided by the embodiments of the present application is shown in the figure. The text of the method is a patent text, as shown in the figure, the method comprises the following steps: Figure 4

[0165] (1) Constructing a patent citation network

[0166] Firstly, according to a preset search strategy, search is performed in a patent database, and the search results are preprocessed such as data cleaning, and the search results after preprocessing are taken as a patent data set. Then, a patent citation network can be extracted from the patent data set. Among them, the original citation network corresponding to the patent data set can be obtained, and the maximum connected subgraph in the original citation network is preprocessed to remove isolated nodes, network fragments and citation loops, and the maximum connected subgraph after preprocessing is taken as the patent citation network; secondly, the vector representation of each node in the patent citation network can be obtained.

[0167] The vector representation corresponding to each patent text can be obtained in the following manner:

[0168] ​a. Text preprocessing: A series of preprocessing is performed on all patent texts, including word segmentation (for Chinese texts), case conversion (for English texts), stemming (for English texts), lemmatization (for English texts), removing punctuation, removing stop words, removing low-frequency words, etc. All preprocessed patent texts are converted into a vocabulary set, and the words appearing in all patent texts are merged and de-duplicated to generate a dictionary corresponding to the patent texts.

[0169] b. Text vectorization: The patent texts are converted into vector form through a spatial vector model. Specifically, for a patent text, a vector with a length equal to the size of the dictionary is first created. For each word in the dictionary, if it does not appear in the patent text, the corresponding position of the vector is set to zero. Otherwise, the weight of the word is calculated using the TF-IDF method and used as the numerical value of the corresponding position of the vector to obtain the spatial vector representation of the patent text.

[0170] c. Vector dimensionality reduction: The spatial vector representation of the patent text is reduced in dimension through models such as LSI (Latent Semantic Analysis) and LDA (Latent Dirichelet Allocation). The high-dimensional sparse vector is converted into a low-dimensional dense vector to save storage space and computation time. The spatial vector representation after dimensionality reduction is used as the vector representation of the patent text.

[0171] (2) Determine the path weight of each path in the patent citation network.

[0172] For each path in the patent citation network, the topological weight of the path can be determined based on the network structure of the path, the semantic weight of the path can be determined based on the semantic information of the text of the path, and the path weight of the path can be obtained by combining the topological weight and the semantic weight of the path. The specific calculation process of the path weight can be referred to the description in the foregoing, which will not be repeated here.

[0173] (3) Search for candidate paths according to the path weight.

[0174] Any one of the greedy method, the exhaustive method or the dynamic programming method can be used to obtain the candidate paths. The specific process of the dynamic programming method can be referred to the description in the foregoing, which will not be repeated here.

[0175] (4) Output the analysis result according to the candidate paths.

[0176] The candidate path with the maximum path weight can be taken as the optimal path, and the optimal path can be taken as the analysis result. The N candidate paths with the maximum path weight, i.e., Top N paths, can be taken as the analysis result and output. The method based on text clustering can be used on the candidate paths, a plurality of clustering centers obtained by clustering are taken as a plurality of main paths corresponding to different subfields, and the plurality of main paths corresponding to different subfields are taken as the analysis result and output.

[0177] The method provided by the embodiment of the application considers the semantic information of the text on the path and the network structure information of the path when calculating the path weight, so that the main path selected according to the path weight not only carries the main traffic of knowledge propagation in the text citation network, but also enables each node included in the main path to focus on a clear technical direction and accurately reflect the development context of knowledge evolution. The global optimal path is searched as the candidate path by using the dynamic programming method, which can not only avoid the problem that the path output by the local path search algorithm may be non-global optimal, but also avoid the problem of high time complexity caused by the global path search algorithm by enumerating all paths. On the premise that the candidate path is the global optimal path, the efficiency of path search is improved.

[0178] In addition, the method provided by the embodiment of the application can be applied to the potential knowledge evolution mode mined from massive text data, is suitable for analyzing the technical evolution process in the industrial field, the knowledge flow, differentiation and fusion context in the discipline field, and the like, and is helpful for grasping the evolution context of field knowledge and revealing the growth process and inheritance law of an emerging discipline or technical field.

[0179] Figure 5 A structural schematic diagram of a text analysis device provided by the embodiment of the application is shown in FIG. 1. Figure 5 As shown in FIG. 1, the device of the embodiment can include:

[0180] A network construction module 201 is configured to construct a text citation network corresponding to a set of texts to be analyzed based on the citation relationship between each text in the set of texts to be analyzed in a target field.

[0181] A candidate path determination module 202 is configured to select a plurality of candidate paths from the text citation network.

[0182] A clustering module 203 is configured to perform at least one clustering operation on the plurality of candidate paths until a preset ending condition is met, and obtain at least two clustering centers when the preset ending condition is met, as knowledge evolution paths corresponding to at least two subfields in the target field, respectively.

[0183] The clustering operation includes:

[0184] For each candidate path, determine a first parameter corresponding to the current clustering operation; the first parameter is positively correlated with the number of the clustering operation;

[0185] According to the initial density of the candidate path, the first topological weight of the candidate path, and the first parameter, calculate a first density of the candidate path;

[0186] Based on the first density respectively corresponding to each candidate path, determine at least two cluster centers obtained by clustering the candidate paths.

[0187] As an optional embodiment, the clustering module 203 is specifically configured to:

[0188] Take the product of the first parameter and the first topological weight of the candidate path as a second parameter;

[0189] Take the sum of the second parameter and the initial density of the candidate path as the first density of the candidate path.

[0190] As an optional embodiment, the candidate path determination module 202 comprises:

[0191] A source node determination sub-module configured to determine each source node in the text citation network;

[0192] A maximum weight path determination sub-module configured to, for each source node, determine a maximum weight path corresponding to the source node;

[0193] A candidate path determination sub-module configured to, based on the maximum weight path respectively corresponding to each source node, determine the candidate path.

[0194] As an optional embodiment, the maximum weight path determination sub-module is configured to:

[0195] Take each node in the text citation network except the source node as a target node, and determine a citation level of each target node with respect to the source node;

[0196] In an order of sequentially increasing citation levels, sequentially determine an optimal path from the source node to each target node;

[0197] Determine the maximum weight path from the optimal paths of each target node;

[0198] For each target node, the determination of the optimal path from the source node to the target node comprises:

[0199] Determine at least one initial path from the source node to the target node;

[0200] According to the path weight respectively corresponding to each initial path, determine the optimal path.

[0201] As an optional embodiment, the candidate path determination module 202 is configured to:

[0202] determine a path weight corresponding to each path in the text citation network respectively;

[0203] select a preset number of paths with the largest path weight as the candidate paths based on the path weight corresponding to each path respectively.

[0204] As an optional embodiment, the apparatus further comprises a path weight determination module, and the path weight determination module is configured to:

[0205] determine a first topological weight of each path in the text citation network based on the second topological weight of each edge included in the path;

[0206] determine a first semantic weight of the path based on the text similarity between each node pair in the path; the node pair is any combination of two nodes in the path;

[0207] determine a path weight of the path based on the first topological weight of the path and the first semantic weight of the path.

[0208] As an optional embodiment, the network construction module 201 is configured to:

[0209] take each text in a set of texts to be analyzed in a target field as a node;

[0210] establish a connection relationship between each node corresponding based on the citation relationship between each text in the set of texts to be analyzed; the direction of an edge in the text citation network is from a cited text to a citing text.

[0211] The apparatuses of the embodiments of the present application can perform the methods provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the apparatuses of the embodiments of the present application are corresponding to the steps in the methods of the embodiments of the present application. The detailed function description of each module of the apparatus can be found in the description of the corresponding method in the foregoing, and will not be repeated here.

[0212] The embodiment of the present application provides an electronic device, including a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to realize the steps of the text analysis method, compared with the related art, the method can realize the following: a text reference network is constructed based on the reference relationship between each text in the text set to be analyzed, and a plurality of candidate paths are selected from the text reference network; at least two clustering centers obtained by clustering the plurality of candidate paths are taken as knowledge evolution paths corresponding to at least two sub-fields respectively, and the analysis result is more comprehensive by analyzing the text set to be analyzed in the target field and obtaining the development context corresponding to each sub-field in the target field. Meanwhile, in the process of clustering the plurality of candidate paths, the text semantic information of the candidate paths and the network structure information of the candidate paths are considered, the information utilization is more comprehensive, the accuracy of the clustering result is improved, and then the analysis result is more accurate.

[0213] In an optional embodiment, an electronic device is provided, as shown in Figure 6 Figure 6 The electronic device 4000 shown in the figure includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, connected through a bus 4002. Optionally, the electronic device 4000 can also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual application, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0214] The processor 4001 can be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can realize or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor 4001 can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0215] ​The bus 4002 can include a path over which the above-described components transmit information. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, or the like. For convenience of representation, Figure 6 Only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0216] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage medium, other magnetic storage device, or any other medium that can be used to carry or store computer programs and that can be accessed by a computer, without limitation.

[0217] The memory 4003 is used to store a computer program for implementing the embodiments of the present application, and is controlled by the processor 4001 to perform. The processor 4001 is used to execute the computer program stored in the memory 4003 to realize the steps shown in the foregoing method embodiments.

[0218] The embodiments of the present application provide a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to realize the steps and corresponding contents of the foregoing method embodiments.

[0219] The terms "first", "second", "third", "fourth", "1", "2", and the like (if any) in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described.

[0220] It should be understood that although each operation step in the flowchart of the embodiments of the present application is indicated by an arrow, the implementation order of the steps is not limited to the order indicated by the arrow. Unless explicitly stated herein, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart can be executed in other orders as required. In addition, part or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on the actual implementation scenario. Part or all of these sub-steps or stages can be executed at the same time, and each of these sub-steps or stages can also be executed at different times. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of the present application do not limit this.

[0221] The above is only an optional implementation of some implementation scenarios of the present application. It should be pointed out that, for ordinary skilled persons in the technical field, other similar implementation means based on the technical idea of the present application without departing from the technical concept of the present application also belong to the protection scope of the embodiments of the present application.

Claims

1. A method of text analysis, characterized by, The method comprises: constructing a text citation network corresponding to a set of texts to be analyzed based on citation relationships between the texts in the set of texts to be analyzed in a target field; selecting a plurality of candidate paths from the text citation network; performing at least one clustering operation on the plurality of candidate paths until a preset ending condition is met, and obtaining at least two cluster centers obtained when the preset ending condition is met as knowledge evolution paths corresponding to at least two subfields in the target field, respectively; the preset ending condition comprises clustering convergence; wherein the clustering operation comprises: determining a first parameter corresponding to the current clustering operation for each candidate path; the first parameter is positively correlated with the number of clustering operations; the first parameter is gradually increased by a preset step size based on an initial value with the increase of the number of clustering operations; multiplying the first parameter and a first topological weight of the candidate path to obtain a second parameter; summing the second parameter and initial densities of the candidate path to obtain a first density of the candidate path; the initial density of the candidate path is calculated based on a density peak value algorithm; the first parameter is used to adjust the proportion of the first topological weight of the candidate path; determining at least two cluster centers obtained by clustering the plurality of candidate paths based on the first densities of the plurality of candidate paths, respectively.

2. The text analysis method of claim 1, wherein, The method comprises: determining each source node in the text citation network; for each source node, determining a maximum weight path corresponding to the source node; determining the candidate paths based on the maximum weight paths corresponding to the source nodes, respectively.

3. The text analysis method of claim 2, wherein, The method comprises: for each source node, determining a maximum weight path corresponding to the source node, comprises: determining the reference level of each target node to the source node by taking each node in the text citation network except the source node as a target node; determining the optimal path from the source node to each target node in turn according to the order of increasing reference level; determining the maximum weight path from the optimal paths of each target node; wherein, for each target node, determining the optimal path from the source node to the target node comprises: determining at least one initial path from the source node to the target node; 4. The text analysis method of claim 1, wherein, determining the optimal path according to the path weights corresponding to each initial path. The method comprises: determining the path weights corresponding to each path in the text citation network; 5. The text analysis method of claim 3 or 4, characterized in that, selecting a preset number of paths with the largest path weights as the candidate paths based on the path weights corresponding to each path. The path weight is determined based on the following steps: for each path in the text citation network, determining a first topological weight of the path based on the second topological weights of the edges included in the path; determining a first semantic weight of the path based on the text similarity between each node pair in the path; the node pair is any combination of two nodes in the path. determine a path weight of the path based on a first topological weight of the path and a first semantic weight of the path.

6. The text analysis method of claim 1, wherein, construct a text citation network corresponding to the set of texts to be analyzed based on the citation relationship between each text in the set of texts to be analyzed in the target field. each text in the set of texts to be analyzed in the target field is taken as a node; a connection relationship between each node is established based on the citation relationship between each text in the set of texts to be analyzed; the direction of an edge in the text citation network is from a cited text to a citing text.

7. A text analysis apparatus characterized by comprising: comprise: a network construction module configured to construct a text citation network corresponding to the set of texts to be analyzed based on the citation relationship between each text in the set of texts to be analyzed in the target field; a candidate path determination module configured to select a plurality of candidate paths from the text citation network; a clustering module configured to perform at least one clustering operation on the plurality of candidate paths until a preset ending condition is met, and at least two clustering centers obtained when the preset ending condition is met are taken as knowledge evolution paths corresponding to at least two subfields in the target field respectively; the preset ending condition comprises clustering convergence; wherein the clustering operation comprises: for each candidate path, determining a first parameter corresponding to the current clustering operation; the first parameter is positively correlated with the number of clustering operations; the first parameter is gradually increased by a preset step size based on an initial value with the increase of the number of clustering operations; multiplying the first parameter and a first topological weight of the candidate path to obtain a second parameter; summing the second parameter and an initial density of the candidate path to obtain a first density of the candidate path; the initial density of the candidate path is calculated based on a density peak value algorithm; the first parameter is used to adjust the proportion of the first topological weight of the candidate path; determining at least two clustering centers obtained by clustering the plurality of candidate paths based on the first density of each candidate path.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 7. The processor executes the computer program to implement the steps of the method of any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-6.

Images