A text stance detection method and device based on text segmentation and hierarchical summary

Abstract

The embodiment of the present application provides a kind of text position detection method and device based on text segmentation and hierarchical abstract, the method comprises: the target text is divided to obtain multiple text segments, so that the length of text can be shortened and parallel processing;Text segment forms tree structure and hierarchical abstract, can understand target text from multiple dimensions;According to the abstract text of child node, the abstract text of intermediate node and the abstract text of root node, the text position of text segment is obtained;According to the text position of text segment, the similarity of the abstract text of child node and the similarity of the abstract text of intermediate node, determine the text position of the abstract text of root node, and the text position of the abstract text of root node is as the text position of target text, so that more comprehensive information of text can be obtained, greatly improve the detection capability of text position, and analysis process is transparent, provide good interpretability, facilitate user to understand in depth.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a text stance detection method and device based on text segmentation and hierarchical summarization, an electronic device and a storage medium. BACKGROUND

[0002] Long texts are ubiquitous in daily life, from long academic papers to blog articles on the Internet or personal sharing on social media. Long text stance detection can play a positive role in market forecasting, public opinion polling, and other fields. However, there are obvious challenges in analyzing long texts. First, long texts contain a large amount of information and may cover multiple topics or viewpoints. This increases the difficulty of stance detection, as it requires accurately identifying each stance and topic in the text. Second, the structure of long texts is usually complex and may contain multiple paragraphs, each with its own theme and stance. This requires the detection program to understand and analyze this complex structure, rather than just extracting keywords or phrases. Finally, the semantics of long texts are usually more abstract and rich. Therefore, if the algorithm cannot understand the true meaning of the text, the results of stance detection may be incorrect. In addition, long text processing requires high computational resources, including large memory and long processing time. SUMMARY

[0003] To solve the above problems, the embodiments of the present application disclose a text stance detection method and device based on text segmentation and hierarchical summarization, an electronic device and a storage medium.

[0004] In a first aspect, the embodiments of the present application provide a text stance detection method based on text segmentation and hierarchical summarization, which comprises:

[0005] Obtaining a target text and segmenting the target text to obtain at least one text segment;

[0006] Obtaining a summary text of a child node of a first layer of a tree structure according to the text segment, and obtaining a summary text of an intermediate node of an intermediate layer of the tree structure according to the summary text of the child node of the first layer of the tree structure, and obtaining a summary text of a root node of an uppermost layer of the tree structure according to the summary text of the intermediate node of the intermediate layer of the tree structure;

[0007] Obtaining a text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node;

[0008] Obtaining a similarity of the summary text of the child node and a similarity of the summary text of the intermediate node;

[0009] determine a text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary text of the sub-node, and the similarity of the summary text of the intermediate node, and take the text stance of the summary text of the root node as the text stance of the target text.

[0010] Optionally, the obtaining of the summary text of the sub-node at the first layer of the tree structure according to the text segment comprises:

[0011] concatenating the at least one text segment with the summary prompt text respectively to input a large model to obtain the summary text corresponding to the at least one text segment;

[0012] taking the summary text corresponding to the at least one text segment as the summary text of the sub-node at the first layer of the tree structure.

[0013] Optionally, the obtaining of the summary text of the intermediate node at the intermediate layer of the tree structure according to the summary text of the sub-node at the first layer of the tree structure comprises:

[0014] grouping the summary text of the sub-node at the first layer of the tree structure by a vector model and a clustering algorithm; each group comprises a preset number of the summary text of the sub-node;

[0015] concatenating the summary text of the sub-node in each group with the summary prompt text respectively to input a large model to obtain the summary text of the upper layer of the summary text of the sub-node;

[0016] taking the summary text of the upper layer of the summary text of the sub-node as the summary text of the intermediate node at the intermediate layer of the tree structure.

[0017] Optionally, the obtaining of the summary text of the root node at the uppermost layer of the tree structure according to the summary text of the intermediate node at the intermediate layer of the tree structure comprises:

[0018] grouping the summary text of the intermediate node by the vector model and the clustering algorithm;

[0019] concatenating the summary text of the intermediate node in each group with the summary prompt text to input a large model to obtain the summary text of the upper layer of the summary text of the intermediate node;

[0020] when the number of the summary text of the upper layer of the summary text of the intermediate node is 1, taking the summary text of the upper layer of the summary text of the intermediate node as the summary text of the root node at the uppermost layer of the tree structure.

[0021] Optionally, the obtaining of the text stance of the text segment according to the text stance of the summary text of the child node, the text stance of the summary text of the intermediate node, and the text stance of the summary text of the root node comprises:

[0022] The summary text of the child node, the summary text of the upper layer of the summary text of the child node, and the summary text of the root node are concatenated with the stance prompt text and input into a large model to obtain the text stance of the text segment.

[0023] Optionally, the obtaining of the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node comprises:

[0024] The similarity between the summary text of the child node and the text segment of the child node is calculated by a locality-sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0025] The similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node is calculated by the locality-sensitive hashing algorithm to obtain the similarity of the summary text of the intermediate node.

[0026] Optionally, the determining of the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary text of the child node, and the similarity of the summary text of the intermediate node comprises:

[0027] The text stance of the summary text of the child node is obtained by the text stance of the text segment and the similarity of the summary text of the child node;

[0028] The text stance of the summary text of the upper layer of the summary text of the intermediate node is obtained by the text stance of the summary text of the intermediate node and the similarity of the summary text of the upper layer of the summary text of the intermediate node;

[0029] The text stance of the summary text of the root node is obtained by the text stance of the summary text of the next layer of the root node and the similarity of the summary text of the root node.

[0030] In a second aspect, an embodiment of the present application provides a text stance detection device based on text segmentation and hierarchical summary, the device comprising:

[0031] A target text acquisition and segmentation module is configured to acquire a target text, and segment the target text to obtain at least one text segment.

[0032] a tree structure hierarchical summary module configured to obtain a summary text of a child node of a first layer of the tree structure according to the text segment, obtain a summary text of an intermediate node of an intermediate layer of the tree structure according to the summary text of the child node of the first layer of the tree structure, and obtain a summary text of a root node of an uppermost layer of the tree structure according to the summary text of the intermediate node of the intermediate layer of the tree structure;

[0033] a first text stance determination module configured to obtain a text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node;

[0034] a node similarity obtaining module configured to obtain a similarity of the summary text of the child node and a similarity of the summary text of the intermediate node;

[0035] a second text stance determination module configured to determine a text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and take the text stance of the summary text of the root node as a text stance of the target text.

[0036] Optionally, the tree structure hierarchical summary module comprises:

[0037] a first summary text generation submodule configured to input the at least one text segment concatenated with a summary prompt text respectively into a large model to obtain a summary text corresponding to the at least one text segment;

[0038] a first hierarchical summary submodule configured to take the summary text corresponding to the at least one text segment as the summary text of the child node of the first layer of the tree structure.

[0039] Optionally, the tree structure hierarchical summary module comprises:

[0040] a first hierarchical summary grouping submodule configured to group the summary texts of the child nodes of the first layer of the tree structure by a vector model and a clustering algorithm; each group comprises a preset number of the summary texts of the child nodes;

[0041] a second summary text generation submodule configured to input the summary texts of the child nodes in each group concatenated with the summary prompt text respectively into the large model to obtain a summary text of an upper layer of the summary text of the child node;

[0042] a second hierarchical summary submodule configured to take the summary text of the upper layer of the summary text of the child node as the summary text of the intermediate node of the intermediate layer of the tree structure.

[0043] Optionally, the tree structure level summary module comprises:

[0044] a second level summary grouping submodule for grouping the summary texts of the intermediate nodes by the vector model and the clustering algorithm;

[0045] a third summary text generation submodule for inputting the summary texts of the intermediate nodes in each group concatenated with the summary prompt text into a large model to obtain the summary texts of the upper level of the summary texts of the intermediate nodes;

[0046] a third level summary submodule for taking the summary text of the upper level of the summary texts of the intermediate nodes as the summary text of the root node of the uppermost level of the tree structure when the number of the summary texts of the upper level of the summary texts of the intermediate nodes is 1.

[0047] Optionally, the first text stance determination module comprises:

[0048] a first text stance determination submodule for inputting the summary text of the child node, the summary text of the upper level of the summary text of the child node, and the summary text of the root node concatenated with the stance prompt text into a large model to obtain the text stance of the text segment.

[0049] Optionally, the inter-node similarity obtaining module comprises:

[0050] a first inter-node similarity obtaining submodule for calculating the similarity between the summary text of the child node and the text segment of the child node by a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0051] a second inter-node similarity obtaining submodule for calculating the similarity between the summary text of the intermediate node and the summary text of the next level corresponding to the summary text of the intermediate node by the local sensitive hashing algorithm to obtain the similarity of the summary text of the intermediate node.

[0052] Optionally, the second text stance determination module comprises:

[0053] a second text stance determination submodule for obtaining the text stance of the summary text of the child node from the text stance of the text segment and the similarity of the summary text of the child node;

[0054] a third text stance determination submodule for obtaining the text stance of the summary text of the upper level of the summary text of the intermediate node from the text stance of the summary text of the intermediate node and the similarity of the summary text of the upper level of the summary text of the intermediate node;

[0055] The fourth text position determining submodule is configured to determine the text position of the summary text of the root node according to the text position of the summary text of the next layer of the summary text of the root node and the similarity of the summary text of the root node.

[0056] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the text position detection method based on text segmentation and hierarchical summarization when executing the computer program.

[0057] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program implements the steps of the text position detection method based on text segmentation and hierarchical summarization when executed by a processor.

[0058] The embodiments of the present application have the following advantages:

[0059] The embodiments of the present application can obtain a target text, segment the target text, and obtain at least one text segment, thereby shortening the length of a single text, solving the problem of text length limitation, allowing parallel processing, thereby reducing the memory occupancy rate, and improving the inference speed and inference effect; obtaining summary texts of child nodes of a first layer of a tree structure according to the text segments, obtaining summary texts of intermediate nodes of an intermediate layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, obtaining a summary text of a root node of an uppermost layer of the tree structure according to the summary texts of the intermediate nodes of the intermediate layer of the tree structure, deeply understanding the target text from multiple dimensions through the tree structure and the hierarchical summary, and improving the prediction accuracy; obtaining text positions of the text segments according to the summary texts of the child nodes, the summary texts of the intermediate nodes, and the summary text of the root node; obtaining similarities of the summary texts of the child nodes and similarities of the summary texts of the intermediate nodes; determining a text position of the summary text of the root node according to the text positions of the text segments, the similarities of the summary texts of the child nodes, and the similarities of the summary texts of the intermediate nodes, and taking the text position of the summary text of the root node as a text position of the target text, so that more comprehensive information of the text can be obtained, the detection capability of the text position is greatly improved, the analysis process is transparent, good interpretability is provided, and in-depth understanding of the user is facilitated. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0061] Figure 1 is a step flow chart of a text stance detection method based on text segmentation and hierarchical summary according to an embodiment of the present application;

[0062] Figure 2 is a step flow chart of another text stance detection method based on text segmentation and hierarchical summary according to an embodiment of the present application;

[0063] Figure 3 is a logic diagram of a text stance detection method based on text segmentation and hierarchical summary according to an embodiment of the present application;

[0064] Figure 4 is a structural block diagram of a text stance detection device based on text segmentation and hierarchical summary according to an embodiment of the present application. DETAILED DESCRIPTION

[0065] In recent years, large models have shown significant performance on a series of tasks in the field of natural language processing. However, there are still some problems when it comes to long text stance detection tasks. The processing of long text requires large models to capture and integrate rich contextual information, which requires large models to have deep reasoning capabilities. Existing large models still have limitations in deep semantic understanding and logical reasoning.

[0066] To solve the above problems, the present application proposes a text stance detection method based on text segmentation and hierarchical summary, aiming to improve the stance detection effect of large models on long text. To achieve this goal, embodiments of the present application shorten the length of individual text by segmenting the target text, reduce memory occupancy and improve reasoning speed and reasoning effect by parallel processing, and obtain more comprehensive information of the text by understanding the target text from multiple dimensions through tree structure and hierarchical summary, greatly improving the detection capability of the text stance, and the analysis process is transparent, providing good interpretability for users to understand in depth.

[0067] In order to make the above-mentioned purposes, characteristics and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0068] Referring to Figure 1 , a step flow chart of a text stance detection method based on text segmentation and hierarchical summary according to an embodiment of the present application is shown, which can specifically include the following steps:

[0069] Step 101, obtaining a target text and segmenting the target text to obtain at least one text segment;

[0070] In the embodiment of the present application, the length of the segmented text segment can be set as a preset character, for example, 2500 characters. Those skilled in the art can set the preset character to other appropriate values according to the idea of the present application, and the present application does not limit this. The segmentation symbol list for segmenting the target text can be ["\n", "", "."], that is, the newline character, space, and period. When the target text is segmented by the newline character, and the length of the text still exceeds the preset character, the text can be further segmented by moving to the next segmentation symbol, and so on, until the target text is segmented into multiple text segments with a length less than the preset character, so as to shorten the length of a single text, solve the problem of text length limitation of the large model, and the segmented text segments can be processed in parallel, so as to reduce the memory occupancy rate and improve the inference speed and inference effect of the large model.

[0071] In step 102, the summary text of the first layer of child nodes of the tree structure is obtained according to the text segment, the summary text of the intermediate nodes of the intermediate layer of the tree structure is obtained according to the summary text of the first layer of child nodes of the tree structure, and the summary text of the root node of the uppermost layer of the tree structure is obtained according to the summary text of the intermediate nodes of the intermediate layer of the tree structure.

[0072] In the embodiment of the present application, the text segment can be regarded as a leaf node of the tree structure, that is, a child node of the first layer of the tree structure, and the summary text corresponding to the text segment can be regarded as the parent node of the leaf node, that is, a node of the second layer of the tree structure. The summary text of the second layer of the tree structure can be used to obtain the summary text of the intermediate nodes of the higher layer, and the summary text of the root node of the uppermost layer of the tree structure.

[0073] In step 103, the text position of the text segment is obtained according to the summary text of the child node, the summary text of the intermediate node, and the summary text of the root node.

[0074] In the embodiment of the present application, in order to obtain more comprehensive information of the text, for each leaf node, in addition to its own summary text, the summary text of the intermediate node and the summary text of the root node to which it belongs can also be combined for text position detection, so as to obtain the text position corresponding to the text segment.

[0075] In step 104, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node are obtained.

[0076] In the embodiment of the present application, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node can be calculated using the SimHash algorithm (Locality-Sensitive Hashing). The SimHash is commonly used for similarity retrieval of high-dimensional vectors. The method calculates a relatively short binary string based on the features of the text, and similar texts will get similar SimHash values.

[0077] In step 105, the text stance of the summary text of the root node is determined according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and the text stance of the summary text of the root node is taken as the text stance of the target text.

[0078] The embodiment of the present application can obtain a target text, and split the target text to obtain at least one text segment, thereby shortening the length of a single text, solving the problem of text length limitation, allowing parallel processing, thereby reducing the memory occupancy rate, improving the inference speed and inference effect; obtaining the summary text of the child node of the first layer of the tree structure according to the text segment, and obtaining the summary text of the intermediate node of the intermediate layer of the tree structure according to the summary text of the child node of the first layer of the tree structure, obtaining the summary text of the root node of the uppermost layer of the tree structure according to the summary text of the intermediate node of the intermediate layer of the tree structure, the tree structure and the hierarchical summary can be used to understand the target text from multiple dimensions, and improve the prediction accuracy; obtaining the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node; obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node; determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and taking the text stance of the summary text of the root node as the text stance of the target text, thereby obtaining more comprehensive information of the text, greatly improving the detection capability of the text stance, and the analysis process is transparent, providing good interpretability, facilitating in-depth understanding of the user.

[0079] Referring to Figure 2 , another step flowchart of a text stance detection method based on text segmentation and hierarchical summary is shown, and the method can specifically include the following steps:

[0080] In step 201, a target text is obtained, and the target text is split to obtain at least one text segment.

[0081] Exemplarily, assuming that the preset character is 2500, the specific process of cutting the target text is as follows: first, the text is cut according to the line break symbol to obtain a plurality of segments; if the length of a segment a exceeds 2500 characters after the first cutting, the segment a is cut according to the second separator "space" in the symbol list for the second time; if the length of the segment a1, a2, etc. after the second cutting is within 2500 characters, the cutting is completed; if the length of the segment a2 is still greater than 2500 characters, the segment a2 is cut according to the third separator "period" in the symbol list; if the length of the segment is still greater than 2500 characters after the third cutting, the segment is cut into a plurality of segments with a length of 2500 characters; and finally, a plurality of text segments with a length of 2500 characters are obtained. The embodiment of the present application shortens the length of a single text, thereby solving the problem of text length limitation of a large model, and the cut text segments can be processed in parallel, thereby reducing the memory occupancy rate and improving the inference speed and inference effect of the large model.

[0082] In step 202, an abstract text of a child node of a first layer of a tree structure is obtained according to the text segment, an abstract text of an intermediate node of an intermediate layer of the tree structure is obtained according to the abstract text of the child node of the first layer of the tree structure, and an abstract text of a root node of an uppermost layer of the tree structure is obtained according to the abstract text of the intermediate node of the intermediate layer of the tree structure.

[0083] In an embodiment, the step of obtaining the abstract text of the child node of the first layer of the tree structure, the abstract text of the intermediate node of the intermediate layer of the tree structure, and the abstract text of the root node of the uppermost layer of the tree structure according to the text segment can further include the following sub-steps.

[0084] In sub-step S11, the at least one text segment is respectively spliced with an abstract prompt text to input a large model to obtain an abstract text corresponding to the at least one text segment.

[0085] In the embodiment of the present application, the abstract prompt text can be an abstract extraction prompt (a text or instruction used to guide a large model to generate a specific type of output). The plurality of text segments obtained above are respectively spliced with the abstract extraction prompt to input a large model to obtain an abstract text corresponding to the text segment.

[0086] Exemplarily, assuming that the target text is T, a plurality of text segments C are obtained after text segmentation, and the content after concatenating each text segment C with the prompt for extracting an abstract is as follows: "Please read the following paragraph and provide a brief abstract highlighting the key information and main points. The paragraph content is as follows: <text segment C>. The abstract should be concise and only contain the most important information. Please ensure that the abstract is a faithful representation of the original text and does not add any additional explanations or comments." The above content is input into the large model to obtain the abstract text corresponding to the text segment.

[0087] In substep S12, the abstract text corresponding to the at least one text segment is taken as the abstract text of the child node of the first layer of the tree structure.

[0088] In the embodiment of the present application, the text segment can be regarded as a leaf node of the tree structure, i.e., a child node of the first layer of the tree structure, and the abstract text corresponding to the text segment can be regarded as the parent node of the leaf node, i.e., a node of the second layer of the tree structure.

[0089] In substep S13, the abstract texts of the child nodes of the first layer of the tree structure are grouped by using a vector model and a clustering algorithm; each group includes a preset number of abstract texts of the child nodes.

[0090] In the embodiment of the present application, in order to obtain the relationship between different text segments, a low-dimensional vector representation of the abstract texts of all text segments can be obtained by using a vector model BGE (Big Graph Embedding), and the nodes of the second layer of the tree structure can be grouped by using a K-Means (K-means clustering) clustering algorithm. The abstract texts belonging to the same cluster are placed in a group, and each group can include a preset number of abstract texts. The preset number can be 4, and those skilled in the art can set the preset number to other appropriate values according to the idea of the present application, which is not limited in the present application.

[0091] In substep S14, the abstract texts of the child nodes in each group are respectively concatenated with the abstract prompt text and then input into the large model to obtain the abstract text of the upper layer of the abstract texts of the child nodes.

[0092] In substep S15, the abstract text of the upper layer of the abstract texts of the child nodes is taken as the abstract text of the intermediate node of the intermediate layer of the tree structure.

[0093] In the embodiment of the present application, the abstract texts in each group can be respectively concatenated with the abstract prompt text and then input into the large model to obtain the abstract text of the upper layer of the abstract texts of the child nodes. The abstract text can be regarded as the parent node of the parent node of the leaf node, i.e., a node of the third layer of the tree structure.

[0094] Sub-step S16, grouping the summary texts of the intermediate nodes by the vector model and the clustering algorithm;

[0095] Sub-step S17, inputting the summary texts of the intermediate nodes in each group concatenated with the summary prompt text into the large model to obtain the summary texts of the next level of the summary texts of the intermediate nodes;

[0096] Sub-step S18, when the number of the summary texts of the next level of the summary texts of the intermediate nodes is 1, taking the summary text of the next level of the summary texts of the intermediate nodes as the summary text of the root node of the uppermost level of the tree structure.

[0097] In the embodiment of the present application, the summary texts of the intermediate nodes of the intermediate layer of the tree structure can be repeatedly clustered by the vector model and the clustering algorithm, and the text summaries and prompts belonging to the same cluster are concatenated and input into the large model again to obtain the summary of the intermediate nodes of the next level, until the number of the summary texts is 1, that is, the root node of the tree structure can be obtained. The hierarchical summary of the embodiment of the present application can understand the target text from multiple dimensions and improve the detection accuracy.

[0098] Step 203, inputting the summary texts of the sub-nodes, the summary texts of the next level of the summary texts of the sub-nodes, and the summary text of the root node concatenated with the stance prompt text into the large model to obtain the text stance of the text segment;

[0099] In the embodiment of the present application, in order to obtain more comprehensive information of the text, for each leaf node, in addition to its own summary text, the summary text of the intermediate node to which it belongs and the summary text of the root node can also be combined for text stance detection. Specifically, the summary texts of the sub-nodes, the summary texts of the next level of the summary texts of the sub-nodes, and the summary text of the root node are concatenated with the stance prompt text and input into the large model, and the text segment corresponding to the summary text of the sub-node is also input into the large model to obtain the text stance of the text segment. The categories of the stance detection can include support, neutrality and opposition, and the stance prompt text can be a prompt outputting the stance detection result.

[0100] Exemplarily, the content of concatenating the summary text of the child node, the summary text of the upper layer of the summary text of the child node, and the summary text of the root node with the stance prompt text is as follows: "The following is a brief summary of the text, summarizing the main information and key points: <summary text of child node> + <summary text of upper layer of summary text of child node> + <summary text of root node>. The following is a document segment: <text segment>. Please start analysis and return a number representing the most relevant stance category, and the correspondence between the number and the category is as follows: 1. Support; 2. Neutral; 3. Oppose", inputting the above content into the large model can obtain the text stance corresponding to the text segment.

[0101] Step 204, obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node;

[0102] In an embodiment, the step of obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node can further include the following sub-steps:

[0103] Sub-step S21, calculating the similarity between the summary text of the child node and the text segment of the child node by using a local sensitive hash algorithm to obtain the similarity of the summary text of the child node;

[0104] Sub-step S22, calculating the similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node by using the local sensitive hash algorithm to obtain the similarity of the summary text of the intermediate node.

[0105] In the embodiment of the application, the similarity between the summary text of the child node and the text segment of the child node can be calculated by using a SimHash algorithm to obtain the similarity of the summary text of the child node, and the similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node can be calculated to obtain the similarity of the summary text of the intermediate node. Wherein, SimHash is commonly used for similarity retrieval of high-dimensional vectors, and this method calculates a relatively short binary string based on the features of the text, similar texts will get similar SimHash values, and this value is stored in the tree structure node corresponding to the summary text, which is used for the weight of the summary text of each layer of the tree structure.

[0106] Step 205, determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary text of the child node, and the similarity of the summary text of the intermediate node, and taking the text stance of the summary text of the root node as the text stance of the target text.

[0107] In an embodiment, the step of determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, can further comprise the following sub-steps:

[0108] Sub-step S31, obtaining the text stance of the summary text of the child node according to the text stance of the text segment and the similarity of the summary text of the child node;

[0109] Sub-step S32, obtaining the text stance of the summary text of the upper layer of the summary text of the intermediate node according to the text stance of the summary text of the intermediate node and the similarity of the summary text of the upper layer of the summary text of the intermediate node;

[0110] Sub-step S33, obtaining the text stance of the summary text of the root node according to the text stance of the summary text of the lower layer of the summary text of the root node and the similarity of the summary text of the root node.

[0111] In the embodiment of the present application, the text stance of the summary text of the child node can be obtained by multiplying the number of the stance category corresponding to the text stance of the text segment with the similarity weight of the summary text of the child node; the text stance of the summary text of the upper layer of the summary text of the intermediate node can be obtained by multiplying the text stance of the summary text of the intermediate node with the similarity weight of the summary text of the upper layer of the summary text of the intermediate node; and the text stance of the summary text of the root node can be obtained by multiplying the text stance of the summary text of the lower layer of the summary text of the root node with the similarity weight of the summary text of the root node.

[0112] Exemplarily, assuming that the tree structure has 4 layers, from top to bottom, which are respectively root, layer 1, layer 2 and leaf, and assuming that layer 1 has two nodes (1-1, 1-2) and layer 2 has four nodes (2-1, 2-2, 2-3, 2-4), the process of stance detection is as follows: according to layer 2+root+leaf, the stance detection result of each leaf is obtained; according to the similarity weight between all leaves and the nodes in layer 2 to which the leaves belong, the stance detection results of the nodes 2-1, 2-2, 2-3 and 2-4 in layer 2 are respectively calculated; according to the similarity weight between the nodes in layer 2 and the nodes in layer 1 to which the nodes in layer 2 belong, the stance detection results of the nodes 1-1 and 1-2 in layer 1 are respectively calculated; and according to the similarity weight between the nodes in layer 1 and the root node, the stance detection result of the root node is calculated as the final stance detection result.

[0113] The embodiment of the application calculates the weighted score of the summary text of the upper layer by multiplying the number of the stance category corresponding to the text stance of each summary text in the tree structure and the similarity weight of the summary text of the upper layer. Then, the scores are merged layer by layer upwards until the root node of the tree structure is reached, and finally the text stance of the summary text of the root node is taken as the text stance of the target text, the target text can be deeply understood from multiple dimensions, the detection accuracy of the text stance is greatly improved, and the analysis process is transparent, good explanation is provided, and the user can deeply understand.

[0114] Referring to Figure 3 , a logic diagram of a text stance detection method based on text segmentation and hierarchical summary provided by the embodiment of the application is shown, in order for those skilled in the art to better understand the embodiment of the application, the following will be described Figure 3 to the embodiment of the application:

[0115] Step 301, the target text is obtained, and the target text is segmented to obtain at least one text segment as a child node of the first layer of the tree structure;

[0116] Step 302, the text segment is spliced with the prompt of the extracted summary respectively and input into a large model to obtain the summary text corresponding to the text segment as the summary text of the child node of the tree structure, i.e. the intermediate node of the second layer;

[0117] Step 303, the similarity between the summary text of the child node and the text segment of the child node is calculated by using a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0118] Step 304, the summary text of the child node of the first layer of the tree structure is grouped by using a vector model and a clustering algorithm to obtain the summary text of the upper layer of the summary text of the child node;

[0119] Step 305, it is judged whether the number of the summary text of the upper layer of the intermediate node is 1;

[0120] Step 306, if the number of the summary text of the upper layer of the intermediate node is not 1, the summary text of the intermediate node is repeatedly grouped by using a vector model and a clustering algorithm to obtain the summary text of the upper layer of the summary text of the child node;

[0121] Step 307, if the number of the summary text of the upper layer of the intermediate node is 1, the summary text of the root node of the uppermost layer of the tree structure is obtained;

[0122] Step 308, the text stance of the text segment is obtained according to the summary text of the child node, the summary text of the upper layer of the summary text of the child node, and the summary text of the root node;

[0123] Step 309, multiply the number of the stance category corresponding to the text stance of each summary text in the tree structure with the similarity weight of the summary text of the previous layer, obtain the weighted score of the summary text of the previous layer, and merge the scores layer by layer upwards to calculate the stance detection result of the root node;

[0124] Step 310, take the text stance of the summary text of the root node as the text stance of the target text.

[0125] The embodiment of the application can obtain a target text, split the target text, and obtain at least one text segment, thereby shortening the length of a single text, solving the text length limitation problem, allowing parallel processing, thereby reducing the memory occupancy rate, improving the inference speed and inference effect; obtaining the summary text of the child node of the first layer of the tree structure according to the text segment, obtaining the summary text of the intermediate node of the intermediate layer of the tree structure according to the summary text of the child node of the first layer of the tree structure, obtaining the summary text of the root node of the uppermost layer of the tree structure according to the summary text of the intermediate node of the intermediate layer of the tree structure, deeply understanding the target text from multiple dimensions through the tree structure and hierarchical summary, and improving the prediction accuracy; obtaining the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node; obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node; determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and taking the text stance of the summary text of the root node as the text stance of the target text, so as to obtain more comprehensive information of the text, greatly improve the detection capability of the text stance, and the analysis process is transparent, providing good interpretability, facilitating in-depth understanding of users.

[0126] For the method embodiment, in order to simply describe, all are expressed as a series of action combinations, but those skilled in the art should know that the embodiment of the application is not limited by the action sequence described, because according to the embodiment of the application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily necessary for the embodiment of the application.

[0127] Referring to Figure 4 , a structure block diagram of a text stance detection device based on text segmentation and hierarchical summary provided by the embodiment of the application is shown, which can specifically include the following modules:

[0128] The target text acquisition and segmentation module 401 is used to obtain a target text, and split the target text to obtain at least one text segment.

[0129] The tree structure hierarchical summary module 402 is configured to obtain a summary text of a child node of a first layer of the tree structure according to the text segment, obtain a summary text of an intermediate node of an intermediate layer of the tree structure according to the summary text of the child node of the first layer of the tree structure, and obtain a summary text of a root node of an uppermost layer of the tree structure according to the summary text of the intermediate node of the intermediate layer of the tree structure.

[0130] The first text stance determination module 403 is configured to obtain a text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node.

[0131] The inter-node similarity obtaining module 404 is configured to obtain a similarity of the summary text of the child node and a similarity of the summary text of the intermediate node.

[0132] The second text stance determination module 405 is configured to determine a text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and take the text stance of the summary text of the root node as a text stance of the target text.

[0133] Optionally, the tree structure hierarchical summary module 402 comprises:

[0134] The first summary text generation submodule is configured to input the at least one text segment after being spliced with a summary prompt text respectively into a large model to obtain a summary text corresponding to the at least one text segment.

[0135] The first hierarchical summary submodule is configured to take the summary text corresponding to the at least one text segment as a summary text of a child node of a first layer of the tree structure.

[0136] Optionally, the tree structure hierarchical summary module 402 comprises:

[0137] The first hierarchical summary grouping submodule is configured to group the summary texts of the child nodes of the first layer of the tree structure by a vector model and a clustering algorithm; each group comprises a preset number of the summary texts of the child nodes.

[0138] The second summary text generation submodule is configured to input the summary texts of the child nodes in each group after being spliced with the summary prompt text respectively into the large model to obtain a summary text of an upper layer of the summary text of the child node.

[0139] The second hierarchical summary submodule is configured to take the summary text of the upper layer of the summary text of the child node as a summary text of an intermediate node of an intermediate layer of the tree structure.

[0140] Optionally, the tree structure level summary module 402 comprises:

[0141] a second level summary grouping submodule for grouping the summary texts of the intermediate nodes by the vector model and the clustering algorithm;

[0142] a third summary text generation submodule for inputting the summary texts of the intermediate nodes in each group concatenated with the summary prompt text into a large model to obtain the summary texts of the upper level of the summary texts of the intermediate nodes;

[0143] a third level summary submodule for, when the number of the summary texts of the upper level of the summary texts of the intermediate nodes is 1, taking the summary texts of the upper level of the summary texts of the intermediate nodes as the summary text of the root node of the uppermost level of the tree structure.

[0144] Optionally, the first text stance determination module 403 comprises:

[0145] a first text stance determination submodule for inputting the summary text of the child node, the summary text of the upper level of the summary text of the child node, and the summary text of the root node concatenated with the stance prompt text into a large model to obtain the text stance of the text segment.

[0146] Optionally, the inter-node similarity obtaining module 404 comprises:

[0147] a first inter-node similarity obtaining submodule for calculating the similarity between the summary text of the child node and the text segment of the child node by a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0148] a second inter-node similarity obtaining submodule for calculating the similarity between the summary text of the intermediate node and the summary text of the next level corresponding to the summary text of the intermediate node by the local sensitive hashing algorithm to obtain the similarity of the summary text of the intermediate node.

[0149] Optionally, the second text stance determination module 405 comprises:

[0150] a second text stance determination submodule for obtaining the text stance of the summary text of the child node from the text stance of the text segment and the similarity of the summary text of the child node;

[0151] a third text stance determination submodule for obtaining the text stance of the summary text of the upper level of the summary text of the intermediate node from the text stance of the summary text of the intermediate node and the similarity of the summary text of the upper level of the summary text of the intermediate node;

[0152] The fourth text position determining submodule is configured to determine the text position of the summary text of the root node according to the text position of the summary text of the next layer of the summary text of the root node and the similarity of the summary text of the root node.

[0153] The embodiment of the present application can obtain a target text, split the target text, and obtain at least one text segment, thereby shortening the length of a single text, solving the problem of text length limitation, allowing parallel processing, thereby reducing the memory occupancy rate, improving the inference speed and inference effect; obtaining the summary text of the first layer of the tree structure according to the text segment, obtaining the summary text of the intermediate node of the intermediate layer of the tree structure according to the summary text of the first layer of the tree structure, obtaining the summary text of the root node of the uppermost layer of the tree structure according to the summary text of the intermediate node of the intermediate layer of the tree structure, deeply understanding the target text from multiple dimensions through the tree structure and hierarchical summary, and improving the prediction accuracy; obtaining the text position of the text segment according to the summary text of the subnode, the summary text of the intermediate node and the summary text of the root node; obtaining the similarity of the summary text of the subnode and the similarity of the summary text of the intermediate node; determining the text position of the summary text of the root node according to the text position of the text segment, the similarity of the summary text of the subnode and the similarity of the summary text of the intermediate node, and taking the text position of the summary text of the root node as the text position of the target text, thereby obtaining more comprehensive information of the text, greatly improving the detection capability of the text position, and providing good interpretability, which is convenient for users to deeply understand.

[0154] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts refer to the part of the method embodiment.

[0155] The embodiment of the present application further provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the above-mentioned text position detection method based on text segmentation and hierarchical summary are realized.

[0156] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned text position detection method based on text segmentation and hierarchical summary are realized.

[0157] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other.

[0158] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, apparatus, or computer program product. Accordingly, embodiments of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application can take the form of a computer program product on one or more machine-readable media (including but not limited to diskette, CD-ROM, optical, and the like) embodying computer program code.

[0159] Embodiments of the present application are described herein with reference to the drawings, which are as follows: Figure 1 Figure 1

[0160] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 Figure 1

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 Figure 1

[0162] While preferred embodiments of the present application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the foregoing description. Therefore, the appended claims are intended to encompass within their scope all possible embodiments of the present application.

[0163] ​​​​​​Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0164] The above describes in detail the text-based stance detection method and device provided by the present application, and the principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A text stance detection method based on text segmentation and hierarchical summarization, characterized in that, The method includes: Obtain the target text and segment it to obtain at least one text fragment; Based on the text fragment, the summary text of the child nodes of the first layer of the tree structure is obtained, and based on the summary text of the child nodes of the first layer of the tree structure, the summary text of the middle nodes of the middle layer of the tree structure is obtained, and based on the summary text of the middle nodes of the middle layer of the tree structure, the summary text of the root node of the top layer of the tree structure is obtained. The text stance of the text fragment is obtained based on the summary text of the child nodes, the summary text of the intermediate nodes, and the summary text of the root node. The similarity between the summary text of the child node and the text fragment of the child node is calculated using the Locality Sensitive Hashing algorithm to obtain the similarity of the summary text of the child node; The similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node is calculated by the local sensitive hashing algorithm to obtain the similarity of the summary text of the intermediate node. The textual stance of the child node's summary text is obtained by comparing the textual stance of the text fragment with the similarity of the summary text of the child node. The textual stance of the upper-level summary text of the intermediate node is obtained by comparing the textual stance of the summary text of the intermediate node with the similarity of the summary text of the upper-level summary text of the intermediate node. The text position of the root node's summary text is obtained by comparing the text position of the summary text of the next level of the root node's summary text with the text position of the root node's summary text, and the text position of the root node's summary text is used as the text position of the target text.

2. The method according to claim 1, characterized in that, The step of obtaining the summary text of the child nodes of the first level of the tree structure based on the text fragment includes: The at least one text fragment is concatenated with the summary prompt text and then input into the large model to obtain the summary text corresponding to the at least one text fragment. The summary text corresponding to the at least one text fragment is used as the summary text of the child node of the first layer of the tree structure.

3. The method according to claim 2, characterized in that, The step of obtaining the summary text of the intermediate nodes of the intermediate layer of the tree structure based on the summary text of the child nodes of the first layer of the tree structure includes: The summary text of the child nodes in the first layer of the tree structure is grouped using a vector model and a clustering algorithm; each group includes a preset number of summary texts of the child nodes. The summary text of each sub-node in each group is concatenated with the summary prompt text and then input into the large model to obtain the summary text of the next layer above the summary text of the sub-node. The summary text of the child node's upper level is used as the summary text of the middle node of the middle layer of the tree structure.

4. The method according to claim 3, characterized in that, The step of obtaining the summary text of the root node of the top layer of the tree structure based on the summary text of the intermediate nodes of the intermediate layer of the tree structure includes: The summary text of the intermediate nodes is grouped using the vector model and the clustering algorithm. The summary text of the intermediate node in each group is concatenated with the summary prompt text and then input into the large model to obtain the summary text of the next layer above the summary text of the intermediate node. When the number of summary texts above the summary text of the intermediate node is 1, the summary text above the summary text of the intermediate node is taken as the summary text of the root node at the top level of the tree structure.

5. The method according to claim 1, characterized in that, The step of obtaining the text position of the text fragment based on the summary text of the child nodes, the summary text of the intermediate nodes, and the summary text of the root node includes: The summary text of the child node, the summary text of the child node's parent node, and the summary text of the root node are concatenated and spliced ​​with the stance prompt text and then input into the large model to obtain the text stance of the text fragment.

6. A text stance detection device based on text segmentation and hierarchical summarization, characterized in that, The device includes: The target text acquisition and segmentation module is used to acquire target text and segment the target text to obtain at least one text fragment; The tree structure hierarchical summarization module is used to obtain the summary text of the child nodes of the first level of the tree structure based on the text fragment, and to obtain the summary text of the middle nodes of the middle level of the tree structure based on the summary text of the child nodes of the first level of the tree structure, and to obtain the summary text of the root node of the top level of the tree structure based on the summary text of the middle nodes of the middle level of the tree structure. The first text stance determination module is used to obtain the text stance of the text segment based on the summary text of the child node, the summary text of the intermediate node, and the summary text of the root node. The node similarity acquisition module is used to calculate the similarity between the summary text of the child node and the text fragment of the child node using the Locality Sensitive Hashing algorithm, thereby obtaining the similarity of the summary text of the child node; and to calculate the similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node using the Locality Sensitive Hashing algorithm, thereby obtaining the similarity of the summary text of the intermediate node. The second text stance determination module is used to obtain the text stance of the child node's summary text by comparing the text stance of the text fragment with the similarity of the child node's summary text; to obtain the text stance of the summary text of the intermediate node's upper-level summary text by comparing the text stance of the intermediate node's summary text with the upper-level summary text of the intermediate node's summary text; to obtain the text stance of the root node's summary text by comparing the text stance of the lower-level summary text of the root node's summary text with the summary text of the root node, and to use the text stance of the root node's summary text as the text stance of the target text.

7. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of a text position detection method based on text segmentation and hierarchical summarization as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a text stance detection method based on text segmentation and hierarchical summarization as described in any one of claims 1-5.

Images