Text field detection method and device based on text segmentation and hierarchical abstract

Abstract

The embodiment of the invention provides a text field detection method and device based on text segmentation and hierarchical abstracts.The method comprises the steps that a target text is segmented to obtain a plurality of text segments, and therefore the text length can be shortened and parallelization processing can be conducted; the tree structure and the hierarchical abstract are formed through the text segments, so that the target text can be deeply understood from multiple dimensions; according to the abstract text of the child node, the abstract text of the middle node and the abstract text of the root node, obtaining a text field of the text fragment; determining the text field of the abstract text of the root node according to the text field of the text fragment, the similarity of the abstract text of the child node and the similarity of the abstract text of the intermediate node, and taking the text field of the abstract text of the root node as the text field of the target text, so that more comprehensive information of the text can be obtained, and the user experience is improved. The method greatly improves the detection capability of the text field, is transparent in analysis process, provides better interpretability, and is convenient for a user to deeply understand.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text stance detection method, device, electronic device and storage medium based on text segmentation and hierarchical summarization. Background Art

[0002] Long texts are ubiquitous in our daily lives, from long academic papers to blog posts on the Internet or personal sharing on social media. Long texts are widely present in all aspects of life. Stance detection of long texts can play a positive role in market forecasting, public opinion polls and other fields, but there are obvious challenges in the analysis of long texts. First, long texts contain a lot of information and may cover multiple topics or viewpoints. This increases the difficulty of stance detection because it is necessary to accurately identify each stance and topic in the text. Second, the structure of long texts is usually complex and may contain multiple paragraphs, each with its own topic and stance. This requires the detection program to understand and analyze this complex structure, rather than just extracting keywords or phrases. Finally, the semantics of long texts are usually more abstract and richer. Therefore, if the algorithm cannot understand the true meaning of the text, the results of stance detection may be wrong. In addition, long text processing has high requirements for computing resources, requiring large memory and long processing time. Summary of the invention

[0003] In order to solve the above problems, the embodiments of the present invention disclose a text stance detection method, device, electronic device and storage medium based on text segmentation and hierarchical summarization.

[0004] In a first aspect, an embodiment of the present invention provides a text stance detection method based on text segmentation and hierarchical summarization, the method comprising:

[0005] Acquire a target text, and segment the target text to obtain at least one text segment;

[0006] Obtain summary texts of child nodes of the first layer of the tree structure according to the text fragments, obtain summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtain summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure;

[0007] Obtaining the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node;

[0008] Obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node;

[0009] According to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes, the text stance of the summary text of the root node is determined, and the text stance of the summary text of the root node is used as the text stance of the target text.

[0010] Optionally, obtaining summary text of the child nodes of the first layer of the tree structure according to the text fragment includes:

[0011] splicing the at least one text segment with the summary prompt text respectively and inputting the resultant text into the large model to obtain the summary text corresponding to the at least one text segment;

[0012] The summary text corresponding to the at least one text segment is used as the summary text of the child node of the first layer of the tree structure.

[0013] Optionally, obtaining the summary text of the intermediate node of the intermediate layer of the tree structure according to the summary text of the child node of the first layer of the tree structure includes:

[0014] The summary texts of the sub-nodes of the first layer of the tree structure are grouped by using a vector model and a clustering algorithm; each group includes a preset number of summary texts of the sub-nodes;

[0015] The summary texts of the sub-nodes in each group are respectively connected in series with the summary prompt texts and then input into the large model to obtain the summary texts of the upper layer of the summary texts of the sub-nodes;

[0016] The summary text of the previous layer of the summary text of the child node is used as the summary text of the middle node of the middle layer of the tree structure.

[0017] Optionally, obtaining the summary text of the root node at the top layer of the tree structure according to the summary text of the intermediate node at the intermediate layer of the tree structure includes:

[0018] Grouping the summary texts of the intermediate nodes by using the vector model and the clustering algorithm;

[0019] The summary text of the intermediate node in each group is connected in series with the summary prompt text and then input into the large model to obtain the summary text of the upper layer of the summary text of the intermediate node;

[0020] When the number of the summary texts in the previous layer of the summary text of the intermediate node is 1, the summary text in the previous layer of the summary text of the intermediate node is used as the summary text of the root node at the uppermost layer of the tree structure.

[0021] Optionally, obtaining the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node includes:

[0022] The summary text of the child node, the summary text of the upper layer of the summary text of the child node, and the summary text of the root node are concatenated with the stance prompt text and input into the large model to obtain the text stance of the text segment.

[0023] Optionally, the obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node includes:

[0024] Calculating the similarity between the summary text of the child node and the text segment of the child node by using a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0025] The similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node is calculated by the local sensitive hash algorithm to obtain the similarity of the summary text of the intermediate node.

[0026] Optionally, determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes comprises:

[0027] Obtaining the text stance of the summary text of the subnode by the similarity between the text stance of the text segment and the summary text of the subnode;

[0028] Obtaining the text stance of the summary text of the intermediate node in the previous layer through the similarity between the text stance of the summary text of the intermediate node and the summary text of the previous layer of the summary text of the intermediate node;

[0029] The text stance of the summary text of the root node is obtained by the similarity between the text stance of the summary text of the next layer of the summary text of the root node and the summary text of the root node.

[0030] In a second aspect, an embodiment of the present invention provides a text stance detection device based on text segmentation and hierarchical summarization, the device comprising:

[0031] A target text acquisition and segmentation module is used to acquire a target text and segment the target text to obtain at least one text segment;

[0032] A tree structure hierarchical summary module, used to obtain summary texts of child nodes of the first layer of the tree structure according to the text fragments, obtain summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtain summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure;

[0033] A first text stance determination module, configured to obtain the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node;

[0034] An inter-node similarity acquisition module, used to acquire the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node;

[0035] The second text stance determination module is used to determine the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes, and use the text stance of the summary text of the root node as the text stance of the target text.

[0036] Optionally, the tree structure hierarchical summary module includes:

[0037] A first summary text generation submodule is used to splice the at least one text segment with the summary prompt text and input the spliced ​​texts into the large model to obtain a summary text corresponding to the at least one text segment;

[0038] The first-level summary submodule is used to use the summary text corresponding to the at least one text segment as the summary text of the child node of the first level of the tree structure.

[0039] Optionally, the tree structure hierarchical summary module includes:

[0040] A first-level summary grouping submodule, used to group the summary texts of the subnodes of the first level of the tree structure by using a vector model and a clustering algorithm; each group includes a preset number of summary texts of the subnodes;

[0041] The second summary text generation submodule is used to concatenate the summary texts of the subnodes in each group with the summary prompt texts and input them into the large model to obtain the summary texts of the upper layer of the summary texts of the subnodes;

[0042] The second-level summary submodule is used to use the summary text of the previous layer of the summary text of the child node as the summary text of the middle node of the middle layer of the tree structure.

[0043] Optionally, the tree structure hierarchical summary module includes:

[0044] A second-level summary grouping submodule, used to group the summary texts of the intermediate nodes by using the vector model and the clustering algorithm;

[0045] The third summary text generation submodule is used to connect the summary text of the intermediate node in each group with the summary prompt text in series and input them into the large model to obtain the summary text of the upper layer of the summary text of the intermediate node;

[0046] The third level summary submodule is used to use the summary text of the previous layer of the summary text of the intermediate node as the summary text of the root node at the top layer of the tree structure when the number of the summary text of the previous layer of the summary text of the intermediate node is 1.

[0047] Optionally, the first text stance determination module includes:

[0048] The first text stance determination submodule is used to concatenate the summary text of the child node, the summary text of the upper layer of the summary text of the child node, and the summary text of the root node with the stance prompt text, and input them into the large model to obtain the text stance of the text segment.

[0049] Optionally, the node similarity acquisition module includes:

[0050] A first inter-node similarity acquisition submodule is used to calculate the similarity between the summary text of the child node and the text segment of the child node by using a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0051] The second inter-node similarity acquisition submodule is used to calculate the similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node through the local sensitive hashing algorithm to obtain the similarity of the summary text of the intermediate node.

[0052] Optionally, the second text stance determination module includes:

[0053] A second text stance determination submodule, configured to obtain the text stance of the summary text of the subnode by comparing the text stance of the text segment with the similarity of the summary text of the subnode;

[0054] A third text stance determination submodule is used to obtain the text stance of the summary text of the intermediate node in the previous layer through the similarity between the text stance of the summary text of the intermediate node and the summary text of the previous layer of the summary text of the intermediate node;

[0055] The fourth text stance determining submodule is used to obtain the text stance of the summary text of the root node through the similarity between the text stance of the summary text of the next layer of the summary text of the root node and the summary text of the root node.

[0056] In a third aspect, the present invention shows an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned text stance detection method based on text segmentation and hierarchical summarization when executing the computer program.

[0057] In a fourth aspect, the present invention shows a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned text stance detection method based on text segmentation and hierarchical summarization are implemented.

[0058] The embodiments of the present invention include the following advantages:

[0059] The embodiment of the present invention can obtain a target text and segment the target text to obtain at least one text segment, thereby shortening the length of a single text, solving the problem of text length limitation, and allowing parallel processing, thereby reducing the video memory occupancy rate and improving the reasoning speed and reasoning effect; obtaining summary texts of child nodes of the first layer of the tree structure according to the text segments, and obtaining summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtaining summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure, and through the tree structure and hierarchical summary, the target text can be deeply understood from multiple dimensions. The text stance of the text segment is obtained according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node; the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node are obtained; the text stance of the summary text of the root node is determined according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and the text stance of the summary text of the root node is used as the text stance of the target text, thereby obtaining more comprehensive information of the text, greatly improving the detection capability of the text stance, and the parsing process is transparent, providing better interpretability, and facilitating users' in-depth understanding. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 is a flowchart of a method for detecting text stance based on text segmentation and hierarchical summarization according to an embodiment of the present invention;

[0062] Figure 2 is a flowchart of another method for detecting text stance based on text segmentation and hierarchical summarization according to an embodiment of the present invention;

[0063] Figure 3 It is a logic diagram of a text stance detection method based on text segmentation and hierarchical summarization according to an embodiment of the present invention;

[0064] Figure 4 It is a structural block diagram of a text stance detection device based on text segmentation and hierarchical summarization according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In recent years, large models have shown remarkable performance in a range of tasks in the field of natural language processing. However, there are still some problems in the task of stance detection in long texts. The processing of long texts requires large models to be able to capture and integrate rich contextual information, which requires large models to have deep reasoning capabilities. Existing large models still have limitations in deep semantic understanding and logical reasoning.

[0066] In view of the above problems, the present invention proposes a text stance detection method based on text segmentation and hierarchical summarization, the purpose of which is to improve the stance detection effect of large model long text. In order to achieve this goal, the embodiment of the present invention segments the target text to shorten the length of a single text, and through parallel processing, it can reduce the memory occupancy rate and improve the reasoning speed and reasoning effect; through the tree structure and hierarchical summary, the target text can be deeply understood from multiple dimensions, and more comprehensive information about the text can be obtained, which greatly improves the ability to detect the stance of the text, and the parsing process is transparent, providing better interpretability, which is convenient for users to understand deeply.

[0067] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0068] Reference Figure 1 , shows a flowchart of a method for detecting text stance based on text segmentation and hierarchical summarization according to an embodiment of the present invention. The method may specifically include the following steps:

[0069] Step 101, obtaining a target text, and segmenting the target text to obtain at least one text segment;

[0070] In an embodiment of the present invention, the length of the segmented text segment can be set to a preset character, such as 2500 characters. A person skilled in the art can set the preset character to other appropriate values ​​according to the concept of the present invention, and the present invention is not limited to this. The list of segmentation symbols for segmenting the target text can be ["\n", "", "."], i.e., line break, space, and period. When the target text is segmented by line breaks, and the text length still exceeds the preset characters, the text can be segmented by moving to the next segmentation symbol, and so on, until the target text is segmented into multiple text segments with a length less than the preset characters, thereby shortening the length of a single text and solving the problem of text length limitation of large models. Moreover, the cut text segments can be processed in parallel, thereby reducing the memory occupancy rate and improving the reasoning speed and reasoning effect of large models.

[0071] Step 102, obtaining summary texts of child nodes of the first layer of the tree structure according to the text fragments, obtaining summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtaining summary texts of the root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure;

[0072] In the embodiment of the present invention, the text fragment can be considered as the leaf node of the tree structure, that is, the child node of the first layer of the tree structure, and the summary text corresponding to the text fragment can be considered as the parent node of the leaf node, that is, the node of the second layer of the tree structure. The summary text of the second layer of the tree structure can be used to obtain the summary text of the intermediate node of the higher layer, and the summary text of the root node of the top layer of the tree structure can be obtained.

[0073] Step 103, obtaining the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node;

[0074] In an embodiment of the present invention, in order to obtain more comprehensive information about the text, for each leaf node, in addition to its own summary text, text stance detection can also be performed in combination with the summary text of the intermediate node to which it belongs and the summary text of the root node to obtain the text stance corresponding to the text fragment.

[0075] Step 104, obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node;

[0076] In an embodiment of the present invention, the SimHash algorithm (Locality-Sensitive Hashing) can be used to calculate the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node. Among them, SimHash is often used for similarity retrieval of high-dimensional vectors. This method calculates a relatively short binary string based on the characteristics of the text, and similar texts will obtain similar SimHash values.

[0077] Step 105, determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes, and using the text stance of the summary text of the root node as the text stance of the target text.

[0078] The embodiment of the present invention can obtain a target text and segment the target text to obtain at least one text segment, thereby shortening the length of a single text, solving the problem of text length limitation, and allowing parallel processing, thereby reducing the video memory occupancy rate and improving the reasoning speed and reasoning effect; obtaining summary texts of child nodes of the first layer of the tree structure according to the text segments, and obtaining summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtaining summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure, and through the tree structure and hierarchical summary, the target text can be deeply understood from multiple dimensions. The text stance of the text segment is obtained according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node; the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node are obtained; the text stance of the summary text of the root node is determined according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and the text stance of the summary text of the root node is used as the text stance of the target text, thereby obtaining more comprehensive information of the text, greatly improving the detection capability of the text stance, and the parsing process is transparent, providing better interpretability, and facilitating users' in-depth understanding.

[0079] Reference Figure 2 , shows a flowchart of another method for detecting text stance based on text segmentation and hierarchical summarization according to an embodiment of the present invention. The method may specifically include the following steps:

[0080] Step 201, obtaining a target text, and segmenting the target text to obtain at least one text segment;

[0081] Exemplarily, assuming that the preset characters are 2500, the specific process of segmenting the target text is as follows: first, perform the first segmentation according to the line break character and divide it into several segments; if after the first segmentation, there is a segment a with more than 2500 words, then the entire segment a can be segmented for the second time according to the second separator "space" in the symbol list; if the number of words in segments a1, a2, etc. after the second segmentation of segment a is within 2500 words, then the segmentation is completed; if there is a segment a2 whose length is still more than 2500 characters, then segment a2 is segmented according to the third separator "period" in the symbol list; if after the three segmentations, there is still a segment length greater than 2500, then the segment is truncated into multiple segments of 2500 characters; finally, multiple text segments with a length of 2500 characters are obtained. The embodiment of the present invention shortens the length of a single text, thereby solving the problem of text length limitation of a large model, and the cut text segments can be processed in parallel, thereby reducing the memory occupancy rate and improving the reasoning speed and reasoning effect of the large model.

[0082] Step 202, obtaining summary texts of child nodes of the first layer of the tree structure according to the text fragments, obtaining summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtaining summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure;

[0083] In one embodiment, the step of obtaining summary texts of child nodes of the first layer of the tree structure according to the text fragments, obtaining summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtaining summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure may further include the following sub-steps:

[0084] Sub-step S11, splicing the at least one text segment with the summary prompt text respectively and inputting the spliced ​​text into the large model to obtain the summary text corresponding to the at least one text segment;

[0085] In an embodiment of the present invention, the summary prompt text may be a prompt for extracting the summary (a text or instruction used to guide the large model to generate a specific type of output). The multiple text fragments obtained above may be spliced ​​with the prompt for extracting the summary and then input into the large model to obtain the summary text corresponding to the text fragment.

[0086] For example, assuming that the target text is T, multiple text fragments C are obtained after text segmentation. The content after splicing each text fragment C with the prompt of the extracted summary is as follows: "Please read the following paragraph and provide a brief summary, highlighting the key information and main points. The paragraph content is as follows: <text fragment C>. The summary should be concise and contain only the most important information. Please make sure that the summary is a faithful representation of the original text without adding any additional explanations or comments." The above content is input into the large model to obtain the summary text corresponding to the text fragment.

[0087] Sub-step S12, using the summary text corresponding to the at least one text segment as the summary text of the child node of the first layer of the tree structure;

[0088] In an embodiment of the present invention, a text fragment can be considered as a leaf node of a tree structure, that is, a child node of the first layer of the tree structure, and the summary text corresponding to the text fragment can be considered as the parent node of the leaf node, that is, a node of the second layer of the tree structure.

[0089] Sub-step S13, grouping the summary texts of the sub-nodes of the first layer of the tree structure by using a vector model and a clustering algorithm; each group includes a preset number of summary texts of the sub-nodes;

[0090] In an embodiment of the present invention, in order to obtain the relationship between different text fragments, a vector model BGE (Big Graph Embedding, large-scale graph embedding algorithm) can be used to obtain a low-dimensional vector representation of the summary text of all text fragments, and a K-Means (K-means clustering) clustering algorithm is used to group the nodes of the second layer of the tree structure, and the summary texts belonging to the same cluster are put in one group. Each group may include a preset number of summary texts, and the preset number may be 4. Those skilled in the art may set the preset number to other appropriate values ​​according to the idea of ​​the present invention, and the present invention is not limited to this.

[0091] Sub-step S14, concatenating the summary text of the sub-nodes in each group with the summary prompt text and inputting them into the large model to obtain the summary text of the upper layer of the summary text of the sub-nodes;

[0092] Sub-step S15, using the summary text of the previous layer of the summary text of the child node as the summary text of the middle node of the middle layer of the tree structure;

[0093] In an embodiment of the present invention, the summary text in each group can be concatenated with the summary prompt text and then input into the large model to obtain the summary text of the upper layer of the summary text of the child node. The summary text can be considered as the parent node of the parent node of the leaf node, that is, the node of the third layer of the tree structure.

[0094] Sub-step S16, grouping the summary texts of the intermediate nodes by using the vector model and the clustering algorithm;

[0095] Sub-step S17, concatenating the summary text of the intermediate node in each group with the summary prompt text and inputting them into the large model to obtain the summary text of the upper layer of the summary text of the intermediate node;

[0096] Sub-step S18, when the number of the summary texts in the previous layer of the summary text of the intermediate node is 1, the summary text in the previous layer of the summary text of the intermediate node is used as the summary text of the root node at the top layer of the tree structure.

[0097] In the embodiment of the present invention, the summary texts of the intermediate nodes in the middle layer of the tree structure can be repeatedly clustered through the vector model and the clustering algorithm cycle, and the text summary and prompt belonging to the same cluster are connected in series and input into the large model again to obtain the summary of the intermediate node at a higher level, until the number of summary texts finally obtained is 1, and the root node of the tree structure can be obtained. The embodiment of the present invention can deeply understand the target text from multiple dimensions through the hierarchical summary and improve the detection accuracy.

[0098] Step 203, the summary text of the child node, the summary text of the previous layer of the summary text of the child node, and the summary text of the root node are concatenated with the stance prompt text and input into the large model to obtain the text stance of the text segment;

[0099] In an embodiment of the present invention, in order to obtain more comprehensive information about the text, for each leaf node, in addition to its own summary text, the summary text of the intermediate node to which it belongs and the summary text of the root node can also be combined to perform text stance detection. Specifically, the summary text of the child node, the summary text of the upper layer of the summary text of the child node, and the summary text of the root node are connected in series with the stance prompt text and input into the large model, and the text segment corresponding to the summary text of the child node is also input into the large model to obtain the text stance of the text segment. Among them, the categories of stance detection can include support, neutrality and opposition, and the stance prompt text can be a prompt for outputting the stance detection result.

[0100] Exemplarily, the summary text of the child node, the summary text of the previous layer of the summary text of the child node, and the summary text of the root node are concatenated with the position prompt text as follows: "The following is a brief summary of the text, summarizing the main information and key points: <summary text of the child node> + <summary text of the previous layer of the summary text of the child node> + <summary text of the root node>. The following is a document fragment: <text fragment>. Please start the analysis and return a number representing the most relevant position category. The correspondence between the number and the category is as follows: 1. Support; 2. Neutral; 3. Oppose". The above content is input into the large model to obtain the text position corresponding to the text fragment.

[0101] Step 204, obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node;

[0102] In one embodiment, the step of obtaining the similarity between the summary text of the child node and the summary text of the intermediate node may further include the following sub-steps:

[0103] Sub-step S21, calculating the similarity between the summary text of the child node and the text segment of the child node by using a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0104] Sub-step S22, calculating the similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node by the local sensitive hash algorithm, to obtain the similarity of the summary text of the intermediate node.

[0105] In an embodiment of the present invention, the SimHash algorithm can be used to calculate the similarity between the summary text of a child node and the text fragment of the child node to obtain the similarity of the summary text of the child node, and calculate the similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node to obtain the similarity of the summary text of the intermediate node. Among them, SimHash is often used for similarity retrieval of high-dimensional vectors. This method calculates a relatively short binary string based on the characteristics of the text. Similar texts will obtain similar SimHash values. This value is stored in the tree structure node corresponding to the summary text, and is used for the weight of the summary text of each layer of the subsequent tree structure.

[0106] Step 205, determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes, and using the text stance of the summary text of the root node as the text stance of the target text.

[0107] In one embodiment, the step of determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes may further include the following sub-steps:

[0108] Sub-step S31, obtaining the text stance of the summary text of the sub-node through the similarity between the text stance of the text segment and the summary text of the sub-node;

[0109] Sub-step S32, obtaining the text stance of the summary text of the intermediate node in the previous layer through the similarity between the text stance of the summary text of the intermediate node and the summary text of the previous layer of the summary text of the intermediate node;

[0110] Sub-step S33, obtaining the text stance of the summary text of the root node through the similarity between the text stance of the summary text of the root node and the summary text of the root node.

[0111] In the embodiment of the present invention, the text stance of the summary text of the child node can be obtained by multiplying the number of the stance category corresponding to the text stance of the text segment with the similarity weight of the summary text of the child node; the text stance of the summary text of the intermediate node can be obtained by multiplying the text stance of the summary text of the intermediate node with the similarity weight of the summary text of the upper layer of the summary text of the intermediate node; the text stance of the summary text of the root node can be obtained by multiplying the text stance of the summary text of the next layer of the summary text of the root node with the similarity weight of the summary text of the root node.

[0112] Exemplarily, assuming that the tree structure has 4 layers, represented from top to bottom as root, layer 1, layer 2, and leaves, assuming that there are two nodes (1-1, 1-2) in layer 1 and 4 nodes (2-1, 2-2, 2-3, 2-4) in layer 2, the process of stance detection is as follows: according to layer 2+root+leaf, the stance detection result of each leaf is obtained; according to the similarity weights between all leaves and the nodes in layer 2 to which they belong, the stance detection results of nodes 2-1, 2-2, 2-3, and 2-4 in layer 2 are calculated respectively; according to the similarity weights between the nodes in layer 2 and the nodes in layer 1 to which they belong, the stance detection results of nodes 1-1 and 1-2 in layer 1 are calculated respectively; according to the similarity weights between the nodes in layer 1 and the root node, the stance detection result of the root node is calculated as the final stance detection result.

[0113] The embodiment of the present invention calculates the weighted score of the summary text of the previous layer by multiplying the number of the stance category corresponding to the text stance of each summary text in the tree structure by the similarity weight of the summary text of the previous layer. Subsequently, these scores are merged upward layer by layer until the root node of the tree structure is reached, and finally the text stance of the summary text of the root node is used as the text stance of the target text, which can deeply understand the target text from multiple dimensions, greatly improving the detection accuracy of the text stance, and the analysis process is transparent, providing good interpretability, which is convenient for users to deeply understand.

[0114] Reference Figure 3 , shows a logic diagram of a text stance detection method based on text segmentation and hierarchical summarization provided by an embodiment of the present invention. In order to enable those skilled in the art to better understand the embodiment of the present invention, the following is a Figure 3 The embodiments of the present invention are described as follows:

[0115] Step 301, obtaining a target text, and segmenting the target text to obtain at least one text segment as a child node of the first layer of the tree structure;

[0116] Step 302, the text segments are spliced ​​with the prompt of the extracted summary and then input into the large model to obtain the summary text corresponding to the text segment, which is used as the summary text of the child node of the tree structure, that is, the middle node of the second layer;

[0117] Step 303, calculating the similarity between the summary text of the child node and the text segment of the child node by using a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0118] Step 304, grouping the summary texts of the child nodes of the first layer of the tree structure by using a vector model and a clustering algorithm to obtain the summary texts of the previous layer of the summary texts of the child nodes;

[0119] Step 305, determining whether the number of summary texts in the previous layer of the summary text of the intermediate node is 1;

[0120] Step 306, if the number of summary texts in the previous layer of the summary text of the intermediate node is not 1, repeatedly grouping the summary texts of the intermediate node by using the vector model and the clustering algorithm to obtain the summary texts in the previous layer of the summary text of the child node;

[0121] Step 307, if the number of summary texts in the previous layer of the summary text of the intermediate node is 1, then the summary text of the root node at the top layer of the tree structure is obtained;

[0122] Step 308, obtaining the text stance of the text segment according to the summary text of the child node, the summary text of the previous layer of the summary text of the child node, and the summary text of the root node;

[0123] Step 309, multiplying the position category number corresponding to the text position of each summary text in the tree structure by the similarity weight of the summary text of the previous layer to obtain the weighted score of the summary text of the previous layer, and merging the scores upward layer by layer to calculate the position detection result of the root node;

[0124] Step 310: Using the text stance of the summary text of the root node as the text stance of the target text.

[0125] The embodiment of the present invention can obtain a target text and segment the target text to obtain at least one text segment, thereby shortening the length of a single text, solving the problem of text length limitation, and allowing parallel processing, thereby reducing the video memory occupancy rate and improving the reasoning speed and reasoning effect; obtaining summary texts of child nodes of the first layer of the tree structure according to the text segments, and obtaining summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtaining summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure, and through the tree structure and hierarchical summary, the target text can be deeply understood from multiple dimensions. The text stance of the text segment is obtained according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node; the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node are obtained; the text stance of the summary text of the root node is determined according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and the text stance of the summary text of the root node is used as the text stance of the target text, thereby obtaining more comprehensive information of the text, greatly improving the detection capability of the text stance, and the parsing process is transparent, providing better interpretability, and facilitating users' in-depth understanding.

[0126] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0127] Reference Figure 4 , shows a structural block diagram of a text stance detection device based on text segmentation and hierarchical summarization provided by an embodiment of the present invention, which may specifically include the following modules:

[0128] The target text acquisition and segmentation module 401 is used to acquire the target text and segment the target text to obtain at least one text segment;

[0129] A tree structure hierarchical summary module 402, configured to obtain summary texts of child nodes of the first layer of the tree structure according to the text fragments, obtain summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtain summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure;

[0130] A first text stance determination module 403, configured to obtain the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node;

[0131] The node similarity acquisition module 404 is used to acquire the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node;

[0132] The second text stance determination module 405 is used to determine the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes, and use the text stance of the summary text of the root node as the text stance of the target text.

[0133] Optionally, the tree structure level summary module 402 includes:

[0134] A first summary text generation submodule is used to splice the at least one text segment with the summary prompt text and input the spliced ​​texts into the large model to obtain a summary text corresponding to the at least one text segment;

[0135] The first-level summary submodule is used to use the summary text corresponding to the at least one text segment as the summary text of the child node of the first level of the tree structure.

[0136] Optionally, the tree structure level summary module 402 includes:

[0137] A first-level summary grouping submodule, used to group the summary texts of the subnodes of the first level of the tree structure by using a vector model and a clustering algorithm; each group includes a preset number of summary texts of the subnodes;

[0138] The second summary text generation submodule is used to concatenate the summary texts of the subnodes in each group with the summary prompt texts and input them into the large model to obtain the summary texts of the upper layer of the summary texts of the subnodes;

[0139] The second-level summary submodule is used to use the summary text of the previous layer of the summary text of the child node as the summary text of the middle node of the middle layer of the tree structure.

[0140] Optionally, the tree structure level summary module 402 includes:

[0141] A second-level summary grouping submodule, used to group the summary texts of the intermediate nodes by using the vector model and the clustering algorithm;

[0142] The third summary text generation submodule is used to connect the summary text of the intermediate node in each group with the summary prompt text in series and input them into the large model to obtain the summary text of the upper layer of the summary text of the intermediate node;

[0143] The third level summary submodule is used to use the summary text of the previous layer of the summary text of the intermediate node as the summary text of the root node at the top layer of the tree structure when the number of the summary text of the previous layer of the summary text of the intermediate node is 1.

[0144] Optionally, the first text stance determination module 403 includes:

[0145] The first text stance determination submodule is used to concatenate the summary text of the child node, the summary text of the upper layer of the summary text of the child node, and the summary text of the root node with the stance prompt text, and input them into the large model to obtain the text stance of the text segment.

[0146] Optionally, the node similarity acquisition module 404 includes:

[0147] A first inter-node similarity acquisition submodule is used to calculate the similarity between the summary text of the child node and the text segment of the child node by using a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node;

[0148] The second inter-node similarity acquisition submodule is used to calculate the similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node through the local sensitive hashing algorithm to obtain the similarity of the summary text of the intermediate node.

[0149] Optionally, the second text stance determination module 405 includes:

[0150] A second text stance determination submodule, configured to obtain the text stance of the summary text of the subnode by comparing the text stance of the text segment with the similarity of the summary text of the subnode;

[0151] A third text stance determination submodule is used to obtain the text stance of the summary text of the intermediate node in the previous layer through the similarity between the text stance of the summary text of the intermediate node and the summary text of the previous layer of the summary text of the intermediate node;

[0152] The fourth text stance determining submodule is used to obtain the text stance of the summary text of the root node through the similarity between the text stance of the summary text of the next layer of the summary text of the root node and the summary text of the root node.

[0153] The embodiment of the present invention can obtain a target text and segment the target text to obtain at least one text segment, thereby shortening the length of a single text, solving the problem of text length limitation, and allowing parallel processing, thereby reducing the video memory occupancy rate and improving the reasoning speed and reasoning effect; obtaining summary texts of child nodes of the first layer of the tree structure according to the text segments, and obtaining summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtaining summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure, and through the tree structure and hierarchical summary, the target text can be deeply understood from multiple dimensions. The text stance of the text segment is obtained according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node; the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node are obtained; the text stance of the summary text of the root node is determined according to the text stance of the text segment, the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node, and the text stance of the summary text of the root node is used as the text stance of the target text, thereby obtaining more comprehensive information of the text, greatly improving the detection capability of the text stance, and the parsing process is transparent, providing better interpretability, and facilitating users' in-depth understanding.

[0154] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0155] An embodiment of the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned method for detecting text stance based on text segmentation and hierarchical summarization are implemented.

[0156] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for detecting text stance based on text segmentation and hierarchical summarization are implemented.

[0157] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0158] Those skilled in the art will appreciate that the embodiments of the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more machine-readable media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0159] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0162] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0163] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0164] The above is a detailed introduction to the text stance detection method and device based on text segmentation and hierarchical summarization provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A text stance detection method based on text segmentation and hierarchical summarization, characterized in that: The method comprises: Acquire a target text, and segment the target text to obtain at least one text segment; Obtain summary texts of child nodes of the first layer of the tree structure according to the text fragments, obtain summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtain summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure; Obtaining the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node; Obtaining the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node; According to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes, the text stance of the summary text of the root node is determined, and the text stance of the summary text of the root node is used as the text stance of the target text.

2. The method according to claim 1, characterized in that The step of obtaining summary text of the child nodes of the first layer of the tree structure according to the text fragments includes: splicing the at least one text segment with the summary prompt text respectively and inputting the resultant text into the large model to obtain the summary text corresponding to the at least one text segment; The summary text corresponding to the at least one text segment is used as the summary text of the child node of the first layer of the tree structure.

3. The method according to claim 2, characterized in that The step of obtaining the summary text of the intermediate node of the intermediate layer of the tree structure according to the summary text of the child node of the first layer of the tree structure includes: The summary texts of the sub-nodes of the first layer of the tree structure are grouped by using a vector model and a clustering algorithm; each group includes a preset number of summary texts of the sub-nodes; The summary texts of the sub-nodes in each group are respectively connected in series with the summary prompt texts and then input into the large model to obtain the summary texts of the upper layer of the summary texts of the sub-nodes; The summary text of the previous layer of the summary text of the child node is used as the summary text of the middle node of the middle layer of the tree structure.

4. The method according to claim 3, characterized in that The step of obtaining the summary text of the root node at the top layer of the tree structure according to the summary text of the intermediate node at the intermediate layer of the tree structure includes: Grouping the summary texts of the intermediate nodes by using the vector model and the clustering algorithm; The summary text of the intermediate node in each group is connected in series with the summary prompt text and then input into the large model to obtain the summary text of the upper layer of the summary text of the intermediate node; When the number of the summary texts in the previous layer of the summary text of the intermediate node is 1, the summary text in the previous layer of the summary text of the intermediate node is used as the summary text of the root node at the uppermost layer of the tree structure.

5. The method according to claim 1, characterized in that The obtaining of the text position of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node comprises: The summary text of the child node, the summary text of the upper layer of the summary text of the child node, and the summary text of the root node are concatenated with the stance prompt text and input into the large model to obtain the text stance of the text segment.

6. The method according to claim 1, characterized in that The obtaining of the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node includes: Calculating the similarity between the summary text of the child node and the text segment of the child node by using a local sensitive hashing algorithm to obtain the similarity of the summary text of the child node; The similarity between the summary text of the intermediate node and the summary text of the next layer corresponding to the summary text of the intermediate node is calculated by the local sensitive hash algorithm to obtain the similarity of the summary text of the intermediate node.

7. The method according to claim 1, characterized in that Determining the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes includes: Obtaining the text stance of the summary text of the subnode by the similarity between the text stance of the text segment and the summary text of the subnode; Obtaining the text stance of the summary text of the intermediate node in the previous layer through the similarity between the text stance of the summary text of the intermediate node and the summary text of the previous layer of the summary text of the intermediate node; The text stance of the summary text of the root node is obtained by the similarity between the text stance of the summary text of the next layer of the summary text of the root node and the summary text of the root node.

8. A text stance detection device based on text segmentation and hierarchical summarization, characterized in that: The device comprises: A target text acquisition and segmentation module is used to acquire a target text and segment the target text to obtain at least one text segment; A tree structure hierarchical summary module, used to obtain summary texts of child nodes of the first layer of the tree structure according to the text fragments, obtain summary texts of intermediate nodes of the middle layer of the tree structure according to the summary texts of the child nodes of the first layer of the tree structure, and obtain summary texts of root nodes of the top layer of the tree structure according to the summary texts of the intermediate nodes of the middle layer of the tree structure; A first text stance determination module, configured to obtain the text stance of the text segment according to the summary text of the child node, the summary text of the intermediate node and the summary text of the root node; An inter-node similarity acquisition module, used to acquire the similarity of the summary text of the child node and the similarity of the summary text of the intermediate node; The second text stance determination module is used to determine the text stance of the summary text of the root node according to the text stance of the text segment, the similarity of the summary texts of the child nodes and the similarity of the summary texts of the intermediate nodes, and use the text stance of the summary text of the root node as the text stance of the target text.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of a text stance detection method based on text segmentation and hierarchical summarization as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a text stance detection method based on text segmentation and hierarchical summarization as described in any one of claims 1 to 7 are implemented.

Images