Method and device for extracting key statements in text and electronic equipment

Abstract

The invention provides a method and device for extracting key statements in a text and electronic equipment, and relates to the technical field of artificial intelligence, in particular to the technical fields of deep learning, cloud computing, natural language processing, large models and the like. According to the specific implementation scheme, a to-be-processed text and a graph model constructed based on the text are obtained; determining initial importance information of the candidate statements according to paragraph positions of paragraphs to which the candidate statements belong in the text and / or statement positions of the candidate statements in the paragraphs to which the candidate statements belong; performing importance iterative calculation processing according to the initial importance information and the graph model to obtain a target importance score of each candidate statement; according to the target importance score, key statements are extracted from the corresponding candidate statements; wherein the semantic similarity between the candidate sentences and the position information of the candidate sentences in the text are automatically combined to determine the importance information of the candidate sentences, and the accuracy of evaluating the importance of the candidate sentences in the whole text and the accuracy of extracting key sentences are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to technical fields such as deep learning, cloud computing, natural language processing, and large models, and in particular to a method, device, and electronic device for extracting key sentences from a text. Background Art

[0002] In some text processing tasks, due to the extremely long text to be processed, severe challenges are encountered in text processing tasks. To effectively deal with this challenge, key sentences can be extracted from the extremely long text to reduce the amount of data to be processed.

[0003] However, how to extract key sentences from the text is very important. Summary of the Invention

[0004] The present disclosure provides a method, device and electronic device for extracting key sentences from a text.

[0005] According to one aspect of the present disclosure, a method for extracting key sentences in a text is provided, the method comprising: obtaining a text to be processed and a graph model constructed based on the text; in the graph model, candidate sentences in the text are used as nodes, and semantic similarities between the candidate sentences are used as edges; initial importance information of the candidate sentences is determined based on the paragraph position of the paragraph to which the candidate sentences belong in the text, and / or the sentence position of the candidate sentences in the paragraph to which they belong; iterative importance calculation processing is performed based on the initial importance information and the graph model to obtain target importance information of each of the candidate sentences; and key sentences are extracted from the corresponding candidate sentences based on the target importance information.

[0006] According to another aspect of the present disclosure, a device for extracting key sentences in a text is provided, the device comprising: a first acquisition module for acquiring a text to be processed and a graph model constructed based on the text; in the graph model, candidate sentences in the text are nodes, and semantic similarities between the candidate sentences are edges; a first determination module for determining initial importance information of the candidate sentences based on the paragraph position of the paragraph to which the candidate sentences belong in the text, and / or the sentence position of the candidate sentences in the paragraph to which they belong; a processing module for performing iterative importance calculation processing based on the initial importance information and the graph model to obtain target importance information of each of the candidate sentences; and an extraction module for extracting key sentences from the corresponding candidate sentences based on the target importance information.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for extracting key sentences from the text proposed above in the present disclosure.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method for extracting key sentences from a text proposed above in the present disclosure.

[0009] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the method for extracting key sentences from a text proposed above in the present disclosure.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 5 This is a schematic diagram of the summary generation framework;

[0017] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0018] Figure 7 It is a block diagram of an electronic device used to implement the method for extracting key sentences in a text according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] In some application scenarios, such as the document input scenario of a large language model (LLM), due to the limited length of the context window of the large language model, when the input text length exceeds the model window limit, the model performance will drop sharply. At this time, it is necessary to extract key information (such as key sentence extraction) from the long input text so that the ultra-long text can be processed without changing the LLM model itself.

[0021] In related technologies, when using the traditional TextRank (text sorting or text ranking) algorithm to extract key sentences, the article is divided into nodes according to the sentences, and a weighted undirected graph model is constructed using the similarity between the sentences. Then, based on the graph model, the PageRank algorithm (page ranking algorithm) is used to iteratively calculate the importance scores of the sentences, and finally, several sentences with the highest importance scores are selected as key information. However, this method mainly calculates weights based on the co-occurrence relationship between words or sentences, so it may focus more on the surface similarity of words and ignore the deep semantics and logical structure of the text. Because it mainly relies on co-occurrence relationships, when using the traditional TextRank algorithm to generate summary text, it may lack sufficient coherence and logical connections.

[0022] In response to at least one of the above problems, the present disclosure provides a method, device, and electronic device for extracting key sentences from a text.

[0023] Figure 1 This is a schematic diagram according to the first embodiment of the present disclosure. It should be noted that the method for extracting key sentences in a text in the embodiment of the present disclosure can be applied to a device for extracting key sentences in a text. The device can be configured in an electronic device so that the electronic device can perform the function of extracting key sentences in a text.

[0024] Among them, the electronic device can be any device with computing capabilities, such as a personal computer (PC), a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a smart speaker, a server, a server cluster, and other hardware devices with various operating systems, touch screens and / or display screens.

[0025] The device for extracting key sentences from a text may also be software in an electronic device, such as software for extracting key sentences from a text, etc. In the following embodiments, the execution subject is an electronic device as an example for description.

[0026] like Figure 1 As shown, the method for extracting key sentences in the text may include the following steps:

[0027] Step 101: Obtain the text to be processed and a graph model constructed based on the text.

[0028] In the graph model, the candidate sentences in the text are nodes, and the semantic similarities between the candidate sentences are edges.

[0029] The candidate sentence may be a single sentence, or may be a combination of multiple sentences, which is not limited in the present disclosure.

[0030] Semantic similarity can be used to measure the similarity between sentences in a text. Semantic similarity can be positively correlated with similarity, and the higher the semantic similarity, the higher the similarity.

[0031] In the embodiment of the present disclosure, the process of the electronic device executing step 101 may, for example, be to obtain the text to be processed; perform sentence segmentation on the text to obtain candidate sentences in the text, and determine the semantic similarity between the candidate sentences in the text, thereby constructing a graph model with the candidate sentences in the text as nodes and the semantic similarity between the candidate sentences as edges.

[0032] Step 102 : determining initial importance information of the candidate sentence based on the paragraph position of the candidate sentence in the text and / or the sentence position of the candidate sentence in the paragraph.

[0033] In an embodiment of the present disclosure, in one example, a paragraph position can be used to indicate the spatial or sequential position of a corresponding paragraph relative to other paragraphs. For example, a paragraph position can include a first paragraph, a middle paragraph, a last paragraph, etc., which is not limited in the present disclosure.

[0034] In another example, the paragraph position can be used to indicate the text content and structure of the corresponding paragraph in the text. For example, the paragraph position can include an introduction paragraph, a method paragraph, a result paragraph, a discussion paragraph, a conclusion paragraph, a transition paragraph, an appendix or a supplementary material paragraph, etc., and the present disclosure does not limit this.

[0035] In the embodiment of the present disclosure, the sentence position can be used to indicate the order and layout of the sentence relative to other sentences in the paragraph. For example, the sentence position can include the beginning position, the middle position, the end position, etc., and the present disclosure does not limit this.

[0036] In embodiments of the present disclosure, initial importance information may be used to indicate the importance of a corresponding sentence in its initial state, wherein the importance information may be used to measure the importance of the sentence within the entire text or paragraph. In one example, the importance information may include, but is not limited to, an importance value, an importance level, etc., and correspondingly, the initial importance information may include, but is not limited to, an initial importance value, an initial importance level, etc.

[0037] Among them, a scoring system can be used to represent the importance value of a sentence. That is, in the scoring system, the importance information can be quantified into a specific value with a certain value range, such as 0 to 100, 0 to 10, and so on.

[0038] Among them, a rating system can be used to represent the importance level of a sentence. That is, in the rating system, the importance information can be divided into several discrete levels, such as "high", "medium", "low", or numerical levels such as 1, 2, 3, etc., or letter levels such as "A+", "A", "A-", "B+", etc.

[0039] In an embodiment of the present disclosure, the electronic device may obtain initial importance information of candidate sentences at different paragraph positions and / or different sentence positions in the text in response to a user's input operation.

[0040] Step 103 : Perform iterative importance calculation based on the initial importance information and the graph model to obtain target importance information of each candidate sentence.

[0041] In an embodiment of the present disclosure, in one example, the target importance information may include but is not limited to a target importance value, a target importance level, and the like.

[0042] In an embodiment of the present disclosure, when the importance information includes an importance value, in one example, the electronic device can adopt a target sorting algorithm to perform iterative importance calculation processing based on the initial importance value of each candidate sentence in the text and the graph model to obtain the target importance value of each candidate sentence.

[0043] The target ranking algorithm may be, for example, a PageRank algorithm or other model that can calculate sentence importance data based on a graph model.

[0044] In the embodiment of the present disclosure, when the importance information includes the importance level, in one example, the electronic device can query the mapping relationship based on the initial importance level of the candidate sentence to determine the corresponding importance value, and determine the importance value as the initial importance value of the corresponding candidate sentence; then adopt the target sorting algorithm to perform iterative importance calculation processing based on the initial importance value of each candidate sentence in the text and the graphical model to obtain the target importance value of each candidate sentence; finally, based on the target importance value of each candidate sentence, query the mapping relationship to determine the corresponding importance level, and determine the importance level as the target importance level of the corresponding candidate sentence.

[0045] The mapping relationship may indicate a corresponding relationship between importance levels and importance values.

[0046] Step 104: extract key sentences from corresponding candidate sentences according to the target importance information.

[0047] In the embodiment of the present disclosure, when the target importance information includes a target importance value, in one example, the electronic device can sort the candidate sentences in the text in descending order according to the target importance value of each sentence to obtain a sorting sequence, and determine each candidate sentence in the sorting sequence whose sorting number is less than the first set value as a key sentence.

[0048] The first setting value may be pre-set, and the present disclosure does not impose any restrictions on its value.

[0049] In another example, the electronic device may sort the candidate sentences in the text in ascending order according to their target importance values to obtain a sorting sequence, and determine each candidate sentence in the sorting sequence whose sorting number is greater than a second set value as a key sentence.

[0050] The second setting value may be pre-set, and the present disclosure does not limit its value.

[0051] In another example, the electronic device may determine, among the candidate sentences, a candidate sentence whose target importance value is greater than a first score threshold as a key sentence.

[0052] The first score threshold may be pre-set, and the present disclosure does not impose any restriction on its value.

[0053] In an embodiment of the present disclosure, when the target importance information includes a target importance level, in one example, the electronic device can sort the candidate sentences in the text in descending order according to the target importance level to obtain a sorted sequence, and determine each candidate sentence in the sorted sequence whose sort number is less than a third set value as a key sentence.

[0054] Among them, the third setting value can be pre-set, and the present disclosure does not limit its value.

[0055] In another example, the electronic device may sort the candidate sentences in the text in ascending order according to their target importance level to obtain a sorting sequence, and determine the candidate sentences in the sorting sequence whose sorting numbers are greater than the fourth set value as key sentences.

[0056] Among them, the fourth setting value can be pre-set, and the present disclosure does not limit its value.

[0057] In another example, the electronic device may determine, among the candidate sentences, a candidate sentence whose target importance level is higher than a first set level as a key sentence.

[0058] The first setting level may be pre-set, and the present disclosure does not limit its value.

[0059] The method for extracting key sentences in a text of an embodiment of the present disclosure obtains a text to be processed and a graph model constructed based on the text; in the graph model, candidate sentences in the text are used as nodes, and semantic similarities between candidate sentences are used as edges; initial importance information of the candidate sentences is determined based on the paragraph position of the paragraph to which the candidate sentences belong in the text, and / or the sentence position of the candidate sentences in the paragraph to which they belong; iterative importance calculation processing is performed based on the initial importance information and the graph model to obtain target importance information of each candidate sentence; key sentences are extracted from the corresponding candidate sentences based on the target importance information; wherein, the importance information of the candidate sentences can be automatically determined by combining the semantic similarity between the candidate sentences and the position information of the candidate sentences in the text, that is, the importance information of the candidate sentences can be determined by combining the deep semantics and logical structure of the candidate sentences in the text, which can improve the accuracy of evaluating the importance of the candidate sentences in the entire text, thereby improving the quality and accuracy of the extracted key sentences.

[0060] In order to accurately determine the initial importance information of the candidate sentences, the electronic device can set the initial importance information of the candidate sentences in the text in combination with the importance setting information. Figure 2 As shown, Figure 2 is a schematic diagram according to a second embodiment of the present disclosure, Figure 2 The illustrated embodiment may include the following steps:

[0061] Step 201: Obtain the text to be processed and a graph model constructed based on the text.

[0062] It should be noted that the details of step 201 can be found in Figure 1 Step 101 in the illustrated embodiment will not be described in detail here.

[0063] In the embodiments of the present disclosure, the text to be processed may include at least one of the following: text obtained after performing voice recognition on audio; text obtained after performing image recognition on an image; and original text. It should be noted that the source type of the text to be processed is not limited here.

[0064] In an embodiment of the present disclosure, after performing speech recognition on the audio to obtain text, in one example, stop word removal processing can be performed on the text.

[0065] In an embodiment of the present disclosure, when an image originates from a video, in one example, an image recognition algorithm can be used to recognize subtitles in the image to obtain text to be processed. For example, an optical character recognition (OCR) algorithm can be used to recognize subtitles in the image to obtain text to be processed.

[0066] In another example, a video understanding model can be used to generate text to be processed based on the video.

[0067] Among them, the video understanding model can be a large model with the function of deeply understanding and parsing video content.

[0068] In another example, the text obtained by performing image subtitle recognition on a video can be combined with the text generated by the video using a video understanding model to generate the text to be processed. In this way, through multimodal information fusion, the information richness and accuracy of the text can be improved.

[0069] This allows for multimodal text acquisition to meet text processing needs in different scenarios and enhance user experience.

[0070] In one example of the disclosed embodiment, the electronic device may further tag the text to be processed based on the source type of the text to be processed, thereby facilitating rapid and accurate identification of the source of the data in subsequent data processing.

[0071] In one example of the disclosed embodiments, the text to be processed may be text involved in a text processing task, where the text processing task may include at least one of the following: a question-and-answer processing task, a text retrieval task, a meeting record task, or a summary generation task. This allows for support for extracting key sentences from text in multiple scenarios, improving applicability. It should be noted that there is no restriction on the type of text processing task, and additional tasks may be added based on actual needs.

[0072] Step 202: Obtain importance setting information.

[0073] The importance setting information may include an importance value and location condition information corresponding to the importance value. The location condition information may be used to indicate the location information that the corresponding importance value should satisfy. It should be noted that the present disclosure does not limit the number of importance values in the importance setting information, nor does it limit the number of location condition information corresponding to each importance value. In other words, any importance value may correspond to one location condition information, or may correspond to multiple location condition information.

[0074] Among them, in the importance setting information, different importance values can be set for different paragraph positions and / or different sentence positions. In other words, the importance values of sentences at different paragraph positions and / or different sentence positions can be the same or different, and can be set according to actual needs.

[0075] In an embodiment of the present disclosure, in one example, importance setting information may be pre-constructed and saved in a database, so that in subsequent applications, the electronic device may directly obtain the saved importance setting information from the database.

[0076] In another example, the electronic device can obtain importance setting information in response to a user input operation. For example, the user can enter importance values for different paragraph positions and / or different sentence positions in the text on the importance value setting page as needed, and the electronic device can obtain importance setting information in response to such user input operation.

[0077] It should be noted that the focus of the text may vary in different text processing tasks. In other words, the importance of the same sentence may vary in different text processing tasks due to different task objectives. Therefore, in one example, corresponding importance setting information can be set for different text processing tasks.

[0078] In an embodiment of the present disclosure, in one example, when the text to be processed is the text involved in a text processing task, importance setting information corresponding to the text processing task can be obtained, wherein the importance setting information corresponding to the text processing task can be determined in combination with the task objective of the text processing task.

[0079] Among them, customizing the importance setting information in combination with the task objectives of the text processing task can ensure that the importance setting information is highly matched with the task requirements, improve the flexibility of the importance information setting, better adapt to the task requirements, and improve the task pertinence and accuracy of the text processing task.

[0080] Step 203: Obtain first position condition information that the candidate sentence meets in the importance setting information.

[0081] In an embodiment of the present disclosure, the electronic device can query the position condition information corresponding to each importance value in the importance setting information based on the paragraph position of the candidate sentence in the text, and / or the sentence position of the candidate sentence in the paragraph, thereby determining the position condition information that matches the candidate sentence from the position condition information corresponding to each importance value, and determine the position condition information as the first position condition information that the candidate sentence meets.

[0082] Step 204: Determine the importance value corresponding to the first location condition information as initial importance information.

[0083] That is, the importance value corresponding to the first position condition information is determined as the initial importance information of the candidate sentence that meets the first position condition information.

[0084] Step 205 : Perform iterative importance calculation based on the initial importance information and the graph model to obtain target importance information for each candidate sentence.

[0085] Step 206: extract key sentences from corresponding candidate sentences according to the target importance information.

[0086] It should be noted that the details of steps 205 to 206 can be found in Figure 1 Steps 103 to 104 in the illustrated embodiment will not be described in detail here.

[0087] The method for extracting key sentences in a text of an embodiment of the present disclosure obtains importance setting information; the importance setting information includes: an importance value, and position condition information corresponding to the importance value; in the importance setting information, different scores are set for different paragraph positions and / or different sentence positions; the first position condition information that the candidate sentence meets in the importance setting information is obtained; the importance value corresponding to the first position condition information is determined as the initial importance information; wherein, the initial importance information of the candidate sentence is determined in combination with the importance setting information, which can fully consider the positional particularity of the candidate sentence in the text, so that the determination of the initial importance information is more accurate, and the initial importance information of the candidate sentence is determined by predefined importance setting information, which can avoid complex real-time analysis and calculation of each candidate sentence, improve the processing speed of determining the initial importance information of the candidate sentence, and reduce the computational overhead of the processing process.

[0088] In order to effectively capture the deep semantic information of candidate sentences instead of relying solely on word-to-word similarity, and to improve the accuracy of evaluating the importance of candidate sentences in the entire text and the accuracy of extracting key sentences, electronic devices can perform similarity calculations based on semantic representation vectors and construct a graph model based on the obtained similarity. Figure 3 As shown, Figure 3 is a schematic diagram according to a third embodiment of the present disclosure, Figure 3 The illustrated embodiment may include the following steps:

[0089] Step 301: Obtain text and candidate sentences.

[0090] It should be noted that the explanation of the text acquisition method in any of the aforementioned embodiments is also applicable to this embodiment and will not be repeated here.

[0091] In an embodiment of the present disclosure, after acquiring the text, the electronic device may perform sentence segmentation processing on the text to obtain candidate sentences in the text.

[0092] In an embodiment of the present disclosure, when performing sentence segmentation processing on a text, in one example, the electronic device may obtain sentence segmentation rules and perform sentence segmentation processing on the text according to the sentence segmentation rules to obtain candidate sentences in the text.

[0093] Among them, the sentence division rules may include sentence division rules designed based on Chinese punctuation marks, sentence division rules designed based on English abbreviations, etc., and the present disclosure does not impose any restrictions on this.

[0094] In an embodiment of the present disclosure, when the text to be processed is the text obtained after voice recognition of audio, when the text is segmented, in one example, the electronic device can use a voice recognition algorithm to identify voice pauses in the audio and mark the voice pauses in the corresponding text; based on the marked voice pauses in the text, the text is segmented.

[0095] The speech pauses may include at least one of the following situations: natural pauses, grammatical pauses, emphasis pauses, and emotional pauses. It should be noted that the present disclosure does not limit the situations of speech pauses, and may add pauses according to actual needs.

[0096] It should be noted that by using speech recognition algorithms to identify speech pauses in audio and marking and sentence-segmenting them in the corresponding text, spoken text can be more accurately segmented into sentences with complete meaning, providing a more reliable foundation for subsequent natural language processing tasks (such as text analysis, sentiment analysis, etc.).

[0097] It should be noted that when the text to be processed is the text obtained after voice recognition of audio, when the text is segmented, in addition to the voice pauses, the voice clarity, voice coherence, speaking speed, intonation, pronunciation characteristics, etc. can also be combined to segment the text, and the present disclosure does not impose any restrictions on this.

[0098] In an embodiment of the present disclosure, after obtaining candidate sentences in the text, the electronic device may also perform the following process: when there is a semantically incomplete first sentence in the candidate sentences, determine the sentence to be merged from the second sentence adjacent to the first sentence in the text, and the candidate sentences may include the first sentence and the second sentence; merge the first sentence and the sentence to be merged.

[0099] The semantically incomplete first sentence may satisfy at least one of the following conditions: the number of characters included is less than or equal to a character number threshold; sentence components are missing; and the semantic incompleteness is determined based on a semantic completeness judgment model.

[0100] The character count threshold may be pre-set, and the present disclosure does not restrict its value. Sentence components may include, for example, a subject, a predicate, an object, an attributive, an adverbial, and the like, and in practical applications, these components may be configured as needed. The semantic integrity judgment model may be a model that has semantic integrity judgment functionality.

[0101] In an embodiment of the present disclosure, in one example, when the number of characters included in a candidate sentence is less than or equal to a character number threshold, it can be determined that the candidate sentence is semantically incomplete, and the candidate sentence can be determined as the first sentence. In another example, when a candidate sentence lacks sentence components, it can be determined that the candidate sentence is semantically incomplete, and the candidate sentence can be determined as the first sentence. In yet another example, when a candidate sentence is input into a semantic completeness judgment model, and the judgment result output by the semantic completeness judgment model indicates that the candidate sentence is semantically incomplete, it can be determined that the candidate sentence is semantically incomplete, and the candidate sentence can be determined as the first sentence.

[0102] It should be noted that the above conditions for the first statement to be satisfied are not limited, and in practical applications, they can be added as needed.

[0103] Therefore, the semantically incomplete sentences in the text can be quickly identified and determined with the help of the judgment conditions of the semantic incompleteness of the sentences.

[0104] The number of statements to be merged may be, but is not limited to, one, and this disclosure does not impose any restrictions on this.

[0105] Among them, by merging semantically incomplete sentences in the text, compound sentences with more complete information and more coherent semantics can be formed, effectively processing fragmented text data, and avoiding the important information from being broken up or underestimated due to sentence fragmentation. In the subsequent graph model construction, the moderation of the model node granularity is improved, and in the subsequent key sentence screening, the interference of fragmented sentences on the screening of important sentences is effectively reduced, thereby improving the accuracy and information completeness of the screened key sentences.

[0106] Step 302: Determine the semantic representation vector of the candidate sentence.

[0107] In an embodiment of the present disclosure, in one example, an electronic device can obtain each word in a vocabulary; for each word, a TF-IDF (Term Frequency–Inverse Document Frequency) algorithm is used to determine the TF-IDF value of the word in a candidate sentence; based on the TF-IDF value of each word in the candidate sentence, a semantic representation vector of the candidate sentence is determined. For example, the TF-IDF values of each word in the sentence can be combined to obtain a semantic representation vector of the sentence.

[0108] The word library may include multiple words. The word library may be a word library of a specific field (such as the medical field, the computer science field, etc.), or a word library of a general language, etc., and may be set according to actual needs.

[0109] As an example, the text to be processed may be segmented to obtain a plurality of words, and the plurality of words may be combined to obtain a word library.

[0110] It should be noted that applying the words in the vocabulary to determine the TF-IDF value of each word in the candidate sentence and constructing the semantic representation vector of the candidate sentence based on this can capture the key information in the candidate sentence and improve the accuracy of text representation.

[0111] It should also be noted that in addition to using the TF-IDF algorithm, a weighted embedding vector representation method, to determine the semantic representation vectors of candidate sentences in a text, other types of weighted embedding vector representation methods can also be used in actual applications to determine the semantic representation vectors of candidate sentences in a text, and this disclosure does not impose any restrictions on this.

[0112] Step 303: Determine the semantic similarity between any two candidate sentences based on the semantic representation vectors of the any two candidate sentences.

[0113] In an embodiment of the present disclosure, for any two candidate sentences in a text, in one example, the electronic device may use a similarity algorithm to calculate the semantic similarity between the corresponding two candidate sentences based on the semantic representation vectors of the corresponding two candidate sentences.

[0114] The similarity algorithm may be, for example, a cosine similarity algorithm, a Euclidean distance algorithm, etc., which is not limited in the present disclosure.

[0115] In step 304, a graph model is constructed with candidate sentences as nodes and semantic similarities as edges.

[0116] In one example of the embodiment of the present disclosure, when the electronic device detects that there is an edge in the graph model whose semantic similarity is less than or equal to the similarity threshold, the edge whose semantic similarity is less than or equal to the similarity threshold may be deleted.

[0117] The similarity threshold may be pre-set, and the present disclosure does not impose any restrictions on its value.

[0118] Among them, in the graph model, each edge is screened according to semantic similarity and edges below the similarity threshold are deleted. Since low-similarity edges often contain more noise and uncertainty, deleting these edges can reduce noise interference, remove redundant and weakly correlated information in the graph model, make the graph structure more concise and clear, reduce the amount of data processing for subsequent data processing, and reduce computing overhead. Retaining edges with higher semantic similarity can more accurately capture key information in the graph model, which helps to improve the accuracy and reliability of subsequent data processing.

[0119] Step 305 : determining initial importance information of the candidate sentence based on the paragraph position of the candidate sentence in the text and / or the sentence position of the candidate sentence in the paragraph.

[0120] Step 306 : Perform iterative importance calculation based on the initial importance information and the graph model to obtain target importance information for each candidate sentence.

[0121] Step 307: extract key sentences from corresponding candidate sentences according to the target importance information.

[0122] It should be noted that the details of steps 305 to 307 can be found in Figure 1 Steps 102 to 104 in the illustrated embodiment will not be described in detail here.

[0123] The method for extracting key sentences from a text in an embodiment of the present disclosure obtains text and candidate sentences; determines the semantic representation vectors of the candidate sentences; determines the semantic similarity between any two candidate sentences based on the semantic representation vectors of any two candidate sentences; and constructs a graph model with the candidate sentences as nodes and the semantic similarity as edges; wherein, determining the semantic similarity between candidate sentences by the semantic representation vectors of the candidate sentences can effectively capture the deep semantic information of the candidate sentences, rather than relying solely on the literal similarity, and provides reliable edge weights for constructing the graph model, which helps to more accurately evaluate the importance of the candidate sentences in the text, thereby improving the accuracy of key sentence extraction.

[0124] It is understandable that there may be a single text scene and multiple text scenes. In order to improve the flexibility and accuracy of key sentence extraction, the electronic device can extract key sentences based on the two scenes. Figure 4 As shown, Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure, Figure 4 The illustrated embodiment may include the following steps:

[0125] Step 401: Obtain the text to be processed and the graph model constructed based on the text.

[0126] Step 402 : determining initial importance information of the candidate sentence based on the paragraph position of the candidate sentence in the text and / or the sentence position of the candidate sentence in the paragraph.

[0127] Step 403: Perform iterative importance calculation based on the initial importance information and the graph model to obtain target importance information for each candidate sentence.

[0128] It should be noted that the details of steps 401 to 403 can be found in Figure 1 Steps 101 to 103 in the illustrated embodiment will not be described in detail here.

[0129] Step 404: Determine the target sentence from the candidate sentences based on the target importance information.

[0130] In the embodiment of the present disclosure, when the target importance information includes a target importance value, in one example, the electronic device can sort the candidate sentences in the text in descending order according to the target importance values of the candidate sentences to obtain a sorted sequence, and determine each candidate sentence in the sorted sequence whose sort number is less than the fifth set value as a target sentence.

[0131] Among them, the fifth setting value can be pre-set, and the present disclosure does not limit its value.

[0132] In another example, the electronic device may sort the candidate sentences in the text in ascending order according to the target importance information to obtain a sorting sequence, and determine each candidate sentence in the sorting sequence whose sorting number is greater than the sixth set value as a candidate sentence.

[0133] Among them, the sixth setting value can be pre-set, and the present disclosure does not limit its value.

[0134] In another example, the electronic device may determine a candidate sentence in the text whose target importance value is greater than a second score threshold as a target sentence.

[0135] The second score threshold may be pre-set, and the present disclosure does not impose any restriction on its value.

[0136] In an embodiment of the present disclosure, when the target importance information includes a target importance level, in one example, the electronic device can sort the candidate sentences in the text in descending order according to the target importance level, obtain a sorted sequence, and determine each candidate sentence in the sorted sequence whose sort number is less than the seventh set value as a target sentence.

[0137] Among them, the seventh setting value can be pre-set, and the present disclosure does not limit its value.

[0138] In another example, the electronic device may sort the candidate sentences in the text in ascending order of target importance to obtain a sorting sequence, and determine the candidate sentences in the sorting sequence whose sorting numbers are greater than the eighth set value as target sentences.

[0139] Among them, the eighth setting value can be pre-set, and the present disclosure does not limit its value.

[0140] In another example, the electronic device may determine, among the candidate sentences, a candidate sentence whose target importance level is higher than a second set level as the target sentence.

[0141] The second setting level may be pre-set, and the present disclosure does not limit its value.

[0142] Step 405: When the number of texts is one, determine the target sentence as a key sentence.

[0143] In an embodiment of the present disclosure, when the number of texts is one, the electronic device may determine the target sentence as a key sentence.

[0144] Step 406: When there are multiple texts, extract the first target sentence from the target sentences of each text according to the sentence extraction strategy.

[0145] The sentence extraction strategy includes at least one of the following: the number of first target sentences extracted from each text is greater than or equal to a sentence number threshold; the proportion of first target sentences extracted from each text is greater than or equal to a sentence proportion threshold. It should be noted that there are no restrictions on sentence extraction strategies here and they can be configured and added based on actual needs.

[0146] The statement quantity threshold may be pre-set, such as 5, 8, etc., and the present disclosure does not impose any restriction on its value.

[0147] The statement ratio threshold may be pre-set, and the present disclosure does not impose any restrictions on its value.

[0148] It should be noted that the proportion of the first target sentence extracted from the text can represent the ratio of the first target sentence extracted from the corresponding text to the candidate sentences in the text, or it can also indicate the ratio of the first target sentence extracted from the corresponding text to the candidate sentences corresponding to multiple texts. The present disclosure does not impose any restrictions on this.

[0149] Among them, extracting the first target sentence in combination with the sentence number threshold can extract sentences with sufficient data to avoid the situation where some sentences are completely ignored; and extracting the first target sentence in combination with the sentence proportion threshold can avoid the situation where sentences in a single text dominate the extracted first target sentence.

[0150] Step 407: Deduplication processing is performed on the first target sentence to obtain a deduplicated first target sentence.

[0151] In an embodiment of the present disclosure, in one example, the process of the electronic device executing step 307 may be, for example: clustering the first target sentences in each text to obtain multiple clusters; for each cluster, selecting a first target sentence from the cluster, and deleting the unselected first target sentences in the cluster.

[0152] In an embodiment of the present disclosure, when clustering the first target sentences in each text, in one example, for any first target sentence in any text, the electronic device can determine from the first target sentence of the corresponding text a second candidate sentence whose semantic similarity with the first target sentence is greater than a set threshold, and merge the first target sentence with the corresponding second candidate sentence, so as to effectively obtain a cluster cluster.

[0153] The threshold value may be preset, and the present disclosure does not impose any restrictions on its value.

[0154] In an embodiment of the present disclosure, for each cluster, when selecting a first target sentence from the cluster, in one example, the electronic device may determine the first target sentence to be selected based on the target importance information of each first target sentence in the cluster. For example, the first target sentence with the highest target importance value or the highest target importance level in each cluster may be determined as the first target sentence to be selected.

[0155] Among them, through clustering processing, the first target sentences with similar expressions or repeated semantics in each text can be quickly identified, the first target sentences with similar expressions or repeated semantics in the text can be deduplicated, and the most representative first target sentences in the cluster cluster can be retained, which can effectively reduce redundant information and improve the accuracy and representativeness of the extracted key sentences.

[0156] Step 408: Determine the first target sentence after deduplication as a key sentence.

[0157] In an embodiment of the present disclosure, in one example, the electronic device may also perform the following process: for any text, when there are multiple first target sentences after deduplication in the text, the key sentences may be determined based on the target importance information of each first target sentence after deduplication in the text. For example, the first target sentences after deduplication in each text may be sorted in descending order of target importance values, or in descending order of target importance values, to obtain a sorted sequence, and each first target sentence in the sorted sequence whose sorting number is less than the ninth set value may be determined as a key sentence.

[0158] Among them, the ninth setting value can be pre-set, and the present disclosure does not limit its value.

[0159] In another example, for any text, if there are multiple first target sentences after deduplication in the text, the electronic device can determine the sentences in the first target sentences after deduplication in the text whose target importance values are greater than the third score threshold as key sentences, or determine the sentences in the first target sentences after deduplication in the text whose target importance levels are greater than the third set level as key sentences.

[0160] The third score threshold and the third setting level may both be pre-set, and the present disclosure does not impose any restrictions on their values.

[0161] It should be noted that step 405 and steps 406 to 408 can be performed selectively.

[0162] In an embodiment of the present disclosure, when the text to be processed is the text involved in the summary generation task, the electronic device may also perform the following process: input the key sentence into the summary generation model, obtain the generated text output by the summary generation model, and determine the generated text as the summary text corresponding to the text.

[0163] The summary generation model may be a model with a summary generation function.

[0164] Among them, combining the summary generation model to automatically generate summary text corresponding to the text based on key sentences can improve the efficiency of information processing, reduce the cost of manual intervention, improve the readability of the generated summary text, and improve user satisfaction.

[0165] The method for extracting key sentences in a text of an embodiment of the present disclosure determines a target sentence from candidate sentences based on target importance information; when the number of texts is one, the target sentence is determined as a key sentence; when the number of texts is multiple, a first target sentence is extracted from the target sentences of each text according to a sentence extraction strategy; the first target sentence is deduplicated to obtain a first target sentence after deduplication; and the first target sentence after deduplication is determined as a key sentence; wherein, screening candidate sentences based on target importance information and extracting target sentences in combination with a sentence extraction strategy can improve the accuracy and effectiveness of key sentence extraction, and deduplicating the first target sentence can avoid extracting repeated key sentences and effectively remove redundant information; and since key sentence extraction of a single text and multiple texts can be achieved, flexibility and wide applicability are improved.

[0166] The following example illustrates the method for extracting key sentences from a text in an embodiment of the present disclosure by applying it to a summary generation framework. Figure 5 As shown, the following steps may be included:

[0167] Step 1. Get the text to be processed by the large language model.

[0168] There may be one or more texts to be processed.

[0169] The text to be processed may be multimodal input text, for example, including at least one of the following:

[0170] 1. Text obtained after speech recognition of audio

[0171] For example, the Automatic Speech Recognition (ASR) algorithm is used to perform speech recognition on the audio to obtain text.

[0172] 2. Text obtained after image recognition on the image

[0173] For example, an optical character recognition algorithm is used to recognize the text in an image to obtain text.

[0174] For example, for videos, an optical character recognition algorithm is used to extract subtitles from the video to obtain text; or a video understanding model is used to generate descriptive text for the video, and then text is obtained based on the descriptive text.

[0175] 3. Original text.

[0176] Step 2: Perform sentence preprocessing on the text to be processed to obtain candidate sentences in the text.

[0177] When performing sentence preprocessing on the text to be processed, sentence segmentation rules may be obtained, and sentence segmentation processing may be performed on the text to be processed according to the sentence segmentation rules.

[0178] Among them, the sentence division rules may include sentence division rules designed based on Chinese punctuation marks, sentence division rules designed based on English abbreviations, etc., and the present disclosure does not impose any restrictions on this.

[0179] Step 3. Perform merge preprocessing on the candidate sentences.

[0180] For each candidate sentence in the text, determine whether the candidate sentence is a semantically incomplete first sentence; when the candidate sentence is the first sentence, select a sentence to be merged from the second sentence adjacent to the first sentence in the text, and merge the candidate sentence and the sentence to be merged; wherein the candidate sentence may include the first sentence and the second sentence.

[0181] Among them, the process of determining whether a candidate sentence is semantically complete may include: when the candidate sentence has at least one of the following conditions, it can be determined that the candidate sentence is semantically incomplete: the number of characters included in the candidate sentence is less than or equal to the character number threshold; the candidate sentence lacks sentence components; when the candidate sentence is input into the semantic completeness judgment model, the judgment result output by the semantic completeness judgment model indicates that the candidate sentence is semantically incomplete.

[0182] Therefore, merging candidate sentences can avoid the separation or underestimation of important information due to sentence fragmentation, improve the node granularity of the subsequently constructed graph model, and prevent the ranking of important sentences from being disturbed by a large number of fragmented sentences in the subsequent extraction of key sentences.

[0183] Step 4: For any two candidate sentences in the text, determine the semantic similarity between the two candidate sentences.

[0184] Specifically, the TF-IDF algorithm is used to determine the semantic representation vectors of candidate sentences in the text; for any two candidate sentences in the text, the semantic representation vectors of the two candidate sentences are combined and the cosine similarity algorithm is used to determine the semantic similarity between the two candidate sentences.

[0185] It should be noted that the semantic representation vectors of candidate sentences are determined in combination with the TF-IDF algorithm, and then the semantic representation vectors of the candidate sentences determined by this algorithm are used to calculate the semantic similarity between the candidate sentences. Compared with calculating similarity based on simple word overlap, this similarity measurement comprehensively considers the frequency weight and importance of words in the full text, and can more accurately reflect the topic relationship between sentences.

[0186] Step 5. Build a graph model using the candidate sentences in the text as nodes and the semantic similarity between any two candidate sentences in the text as edges.

[0187] When edges with semantic similarity less than or equal to the similarity threshold are detected in the graph model, these edges can be deleted. This mechanism allows us to retain sentences with high relevance, effectively controlling the sparsity of the constructed graph model and reducing noise interference.

[0188] Step 6. Combined with the graph model, the importance scores of the candidate sentences in the text are iteratively calculated to obtain the target importance score of each candidate sentence (referred to as the target importance value in this disclosure).

[0189] In one example, the initial importance scores of candidate sentences in the text (referred to as initial importance values in this disclosure) can be determined in advance; combined with the initial importance scores of the candidate sentences in the text and the graph model, the PageRank algorithm is used to iteratively calculate the importance scores of the sentences in the text to obtain the target importance scores of each candidate sentence.

[0190] It is understandable that the paragraph position of the paragraph to which the candidate sentence belongs in the text, and / or the sentence position of the candidate sentence in the paragraph to which it belongs, can reflect the importance of the candidate sentence.

[0191] In order to accurately obtain the target importance score of each candidate sentence, the initial parameter, i.e., the initial importance score of the candidate sentence, can be adjusted. In one example, importance setting information is obtained; the importance setting information includes at least one score (recorded as an importance value in this disclosure), and position condition information corresponding to at least one score; in the importance setting information, different scores are set for different paragraph positions and / or different sentence positions; the first position condition information that the candidate sentence meets in the importance setting information is obtained; and the score corresponding to the first position condition information is determined as the initial importance score of the candidate sentence.

[0192] In one example, the paragraph position can be used to indicate the spatial or sequential position of the corresponding paragraph relative to other paragraphs. For example, the paragraph position can include the first paragraph, the middle paragraph, the last paragraph, etc., which is not limited in the present disclosure.

[0193] In another example, the paragraph position can be used to indicate the text content and structure of the corresponding paragraph in the text. For example, the paragraph position can include an introduction paragraph, a method paragraph, a result paragraph, a discussion paragraph, a conclusion paragraph, a transition paragraph, an appendix or a supplementary material paragraph, etc., and the present disclosure does not limit this.

[0194] The sentence position can be used to indicate the order and layout of a sentence relative to other sentences in a paragraph. For example, the sentence position can include the beginning position, the middle position, the end position, etc., which is not limited in the present disclosure.

[0195] For example, improving the initial importance score of the candidate sentence belonging to the first paragraph in the text can effectively prevent the situation where the initial importance score of the candidate sentence is low due to the lack of connection between the first paragraph and the candidate sentences in other paragraphs in the text when determining the initial importance score of the candidate sentence, when the candidate sentence is important but belongs to the first paragraph.

[0196] In the process of obtaining importance setting information, in one example, importance setting information corresponding to a text processing task can be obtained. The importance setting information corresponding to a text processing task is determined in conjunction with the task objective of the text processing task. Thus, different importance setting information can be set for different text processing tasks, improving flexibility and targeting, and making the initial importance score of the determined candidate sentence more suitable for the corresponding text processing task.

[0197] Step 7. Extract key sentences from each candidate sentence based on the target importance score of each candidate sentence.

[0198] In one example, the candidate sentences may be sorted in descending order of target importance scores to obtain a sorted sequence, and key sentences may be determined from the candidate sentences based on the sorted sequence.

[0199] In another example, when there are multiple texts, for each text, a target sentence is selected from each candidate sentence in the text according to the target importance score of each candidate sentence in the text; a first target sentence is extracted from the target sentences of each text according to a sentence extraction strategy; and the first target sentence in each text is deduplicated to obtain a deduplicated first target sentence; and the deduplicated first target sentence is determined as a key sentence.

[0200] The statement extraction strategy includes at least one of the following:

[0201] 1. The number of first target sentences extracted from each text is greater than or equal to the sentence number threshold;

[0202] Among them, the first target sentence is extracted in combination with the sentence quantity threshold, so that sentences with sufficient data can be extracted, and the situation where some sentences are completely ignored can be avoided.

[0203] 2. The proportion of the first target sentence extracted from each text is greater than or equal to the sentence proportion threshold.

[0204] Among them, combining the sentence proportion threshold to extract the first target sentence can avoid the situation where the extracted sentences are dominated by sentences in a single text.

[0205] In one example, the process of deduplicating the first target sentences in each text may include: for any text, clustering the first target sentences in the text according to the semantic similarity between the first target sentences in the text to obtain multiple clusters; for each cluster, selecting a first target sentence from the cluster, and deleting the unselected first target sentences in the cluster.

[0206] Therefore, for sentences with similar expressions in different documents, the most representative sentence can be retained to remove redundant information.

[0207] In another example, for any text, the first target sentences after deduplication corresponding to the text may be sorted in descending order of importance scores to obtain a sorted sequence, and key sentences may be selected based on the sorted sequence.

[0208] Step 8. Integrate the key sentences and generate summary text.

[0209] In one example, key sentences can be input into a summary generation model to obtain generated text output by the summary generation model; the generated text is then determined as the summary text corresponding to the text. The summary generation model can then provide appropriate sentence cohesion and smoothing for key sentences, improving readability after multimodal information fusion.

[0210] The method for extracting key sentences from the text disclosed in the present invention is applied to the summary generation method. Compared with the application of the traditional TextRank algorithm in the summary generation method, it effectively reduces redundant and irrelevant sentences while ensuring that the summary text covers the main content of the text; in the scenario of single text input, it realizes compression of the single text and improves the information accuracy and conciseness of the generated summary text; in the scenario of multiple text input, key sentences can be extracted and integrated from multiple texts to avoid duplication and redundancy; in the scenario of multimodal information input, the core information of multi-source information can be integrated, and the generated summary text can cover the key information of each text without duplication and redundancy, and grasp the common points and complementary information across texts as a whole.

[0211] It should be noted that the above summary generation method can be flexibly applied to a wide range of natural language processing and generation tasks, especially in the scenario where large language models (LLMs) are applied to long text content understanding and reasoning, where it has obvious advantages.

[0212] (1) In the scenario of knowledge base question answering and intelligent retrieval:

[0213] In this scenario, it is usually necessary to extract accurate answers or relevant information from a large number of documents. In related technologies, a large amount of relevant text or fragments needs to be directly input into the LLM. This information can easily exceed the model context length limit, easily leading to information omission or processing difficulties. However, by applying the above-mentioned summary generation method to this scenario, the large number of retrieved candidate texts can first be subjected to the key sentence extraction method in the text disclosed in this disclosure to extract key sentences and remove duplicates. The extracted key information is then input into the LLM. This can achieve efficient and accurate question and answer output, effectively avoid the interference of redundant information on the large model, and improve the response speed and accuracy of the question and answer system.

[0214] (2) In the scenario of generating summaries of legal and contractual documents:

[0215] Legal documents such as contracts, clauses, judgments, and case files are often lengthy, complex, and filled with details. In practice, lawyers, judges, and legal professionals need to quickly grasp the core content. Using the aforementioned summary generation method to generate summary text can quickly identify key clauses or core points of contention in contracts or documents, automatically generating accurate summaries. This allows relevant personnel to efficiently understand the document's key information, accelerate decision-making, and significantly improve the efficiency and quality of legal text analysis.

[0216] (3) In financial research and industry intelligence analysis scenarios:

[0217] In the financial sector, many users need to quickly grasp the key points of lengthy research reports, market commentaries, and financial statements to assist in investment decision-making. Using the summary generation method described above, long financial reports, market research materials, and financial statements can be quickly compressed to highlight core performance data, industry trends, and company highlights. This provides clearer and more accurate information input for financial analysis, thereby improving the accuracy of financial decisions and reducing risk.

[0218] (4) Automatic summarization of academic literature and research reviews:

[0219] In scientific research, researchers need to quickly review large numbers of academic papers or technical documents. Traditionally, reading each paper individually is time-consuming and laborious. By employing the summary generation method described above, we can efficiently extract the key content or innovative points of each paper or patent document. This approach then combines the essential information from different documents to form an efficient literature review or research report. This method significantly reduces researchers' time spent reading and organizing literature, accelerating the research process.

[0220] (5) In the scenarios of multimodal content analysis and intelligent meeting recording:

[0221] The above summary generation method can be used to uniformly convert multimodal content, such as audio speech recognition transcription, video subtitles, and image OCR recognition text, into text before summarizing it. This summary generation method is particularly suitable for intelligent meeting record systems, as meetings may simultaneously include multiple input sources, such as speech recordings, PowerPoint presentations, and presentation subtitles. Traditional methods are difficult to quickly integrate and analyze. However, this summary generation method can uniformly compress this multimodal information before inputting it into the LLM, producing clear and concise meeting summaries, to-do items, and key decision points, effectively improving meeting efficiency.

[0222] (6) In the scenario of intelligent assistance for customer service:

[0223] Customer service platforms often need to quickly extract relevant information from lengthy documents such as knowledge bases, historical conversations, and product manuals to respond to user questions. Using this summary generation method, service robots can quickly locate relevant content, extract precise summary information, and generate summary text. This, when fed into the LLM, allows them to quickly generate accurate and coherent answers, improving customer satisfaction and reducing the burden on human service personnel.

[0224] In summary, the summary generation method integrated with the extraction method of key sentences in the text disclosed in this invention can be widely used in various intelligent scenarios with information-intensive and multimodal fusion due to its excellent generalization and information refinement capabilities, and provide strong support for knowledge question answering, legal document processing, financial analysis, academic document management, intelligent conferencing, multimedia public opinion monitoring and customer service assistance.

[0225] In order to implement the above embodiment, the present disclosure also provides a device for extracting key sentences in a text. Figure 6 As shown, Figure 6 6 is a schematic diagram of a fifth embodiment of the present disclosure. The apparatus 60 for extracting key sentences from a text may include: a first acquisition module 601 , a first determination module 602 , a processing module 603 and an extraction module 604 .

[0226] The first acquisition module 601 is used to acquire the text to be processed and a graph model constructed based on the text; in the graph model, candidate sentences in the text are used as nodes, and semantic similarities between candidate sentences are used as edges.

[0227] The first determining module 602 is configured to determine initial importance information of a candidate sentence based on the paragraph position of the candidate sentence in the text and / or the sentence position of the candidate sentence in the paragraph.

[0228] The processing module 603 is used to perform iterative importance calculation processing based on the initial importance information and the graph model to obtain target importance information of each candidate sentence.

[0229] The extraction module 604 is configured to extract key sentences from corresponding candidate sentences according to target importance information.

[0230] As a possible implementation method of the embodiment of the present disclosure, the first determination module 602 is used to: obtain importance setting information; the importance setting information includes: an importance value, and position condition information corresponding to the importance value; in the importance setting information, different importance values are set for different paragraph positions and / or different sentence positions; obtain the first position condition information that the candidate sentence meets in the importance setting information; and determine the importance value corresponding to the first position condition information as the initial importance information of the sentence.

[0231] As a possible implementation method of an embodiment of the present disclosure, the text to be processed is the text involved in the text processing task; the first determination module 602 is used to: obtain the importance setting information corresponding to the text processing task; the importance setting information corresponding to the text processing task is determined based on the task objective of the text processing task.

[0232] As a possible implementation method of the embodiment of the present disclosure, the first acquisition module 601 is used to: acquire text and candidate sentences; determine the semantic representation vectors of the candidate sentences; determine the semantic similarity between any two candidate sentences based on the semantic representation vectors of any two candidate sentences; and construct a graph model with the candidate sentences in the text as nodes and the semantic similarity as edges.

[0233] As a possible implementation of the embodiment of the present disclosure, the device 60 for extracting key sentences from the text may further include:

[0234] The second determining module is configured to determine, in response to a semantically incomplete first sentence in the candidate sentences, sentences to be merged from second sentences adjacent to the first sentence in the text, wherein the candidate sentences include the first sentence and the second sentence.

[0235] The merging module is used to merge the first statement and the statement to be merged.

[0236] As a possible implementation method of the embodiment of the present disclosure, the first sentence satisfies at least one of the following conditions: the number of characters included is less than or equal to the character number threshold; sentence components are missing; and the semantics are determined to be incomplete based on a semantic completeness judgment model.

[0237] As a possible implementation of the embodiment of the present disclosure, the first acquisition module 601 is used to: acquire each word in the vocabulary; determine the TF-IDF value of each word in the candidate sentence; and determine a semantic representation vector based on the TF-IDF value.

[0238] As a possible implementation of the embodiment of the present disclosure, the device 60 for extracting key sentences from the text may further include:

[0239] The deletion module is configured to delete the edge with a semantic similarity less than or equal to the similarity threshold in response to detecting that the edge has a semantic similarity less than or equal to the similarity threshold in the graph model.

[0240] As a possible implementation of the embodiment of the present disclosure, the extraction module 604 is configured to: determine a target sentence from candidate sentences based on target importance information; and determine the target sentence as a key sentence when the number of texts is one.

[0241] As a possible implementation method of the embodiment of the present disclosure, the extraction module 604 is used to: when there are multiple texts, extract the first target sentence from the target sentences of each text according to the sentence extraction strategy; deduplicate the first target sentence to obtain the deduplicated first target sentence; and determine the deduplicated first target sentence as a key sentence.

[0242] As a possible implementation method of an embodiment of the present disclosure, the sentence extraction strategy includes at least one of the following: the number of first target sentences extracted from each text is greater than or equal to a sentence number threshold; the proportion of first target sentences extracted from each text is greater than or equal to a sentence proportion threshold.

[0243] As a possible implementation of the embodiment of the present disclosure, the extraction module 604 is configured to: cluster the first target sentences in each text to obtain multiple clusters; select a first target sentence from each cluster, and delete the unselected first target sentences.

[0244] As a possible implementation method of an embodiment of the present disclosure, the text to be processed is the text involved in a text processing task; the text processing task includes at least one of the following: a question and answer processing task, a text retrieval task, a meeting record task, and a summary generation task.

[0245] As a possible implementation of the embodiment of the present disclosure, the text to be processed is the text involved in the summary generation task; the key sentence extraction device 60 in the text may further include:

[0246] The second acquisition module is used to input the key sentence into the summary generation model and obtain the generated text output by the summary generation model.

[0247] The third determining module is used to determine the generated text as the summary text corresponding to the text.

[0248] As a possible implementation of the embodiment of the present disclosure, the text to be processed includes at least one of the following: text obtained after voice recognition is performed on audio; text obtained after image recognition is performed on an image; and original text.

[0249] The device for extracting key sentences in a text of an embodiment of the present disclosure obtains a text to be processed and a graph model constructed based on the text; determines the initial importance information of the candidate sentence according to the paragraph position of the paragraph to which the candidate sentence belongs in the text and / or the sentence position of the candidate sentence in the paragraph to which it belongs; performs iterative importance calculation processing based on the initial importance information and the graph model to obtain target importance information of each candidate sentence; extracts key sentences from the corresponding candidate sentences based on the target importance information; wherein, the importance information of the candidate sentences is automatically determined by combining the semantic similarity between the candidate sentences and the position information of the candidate sentences in the text, thereby improving the accuracy of evaluating the importance of the candidate sentences in the entire text and the accuracy of extracting key sentences.

[0250] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, comply with relevant laws and regulations, and do not violate public order and good morals.

[0251] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0252] Figure 7 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0253] like Figure 7As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0254] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0255] The computing unit 801 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units for running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the method for extracting key sentences in the text. For example, in some embodiments, the method for extracting key sentences in the text can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the method for extracting key sentences in the text described above can be executed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the method for extracting key sentences from a text in any other appropriate manner (eg, by means of firmware).

[0256] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0257] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0258] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0259] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0260] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0261] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0262] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0263] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for extracting key sentences from a text, the method comprising: Obtaining text to be processed and a graph model constructed based on the text; In the graph model, candidate sentences in the text are used as nodes, and semantic similarities between the candidate sentences are used as edges; determining initial importance information of the candidate sentence based on the paragraph position of the paragraph to which the candidate sentence belongs in the text and / or the sentence position of the candidate sentence in the paragraph to which it belongs; Performing iterative importance calculation processing based on the initial importance information and the graph model to obtain target importance information of each candidate sentence; According to the target importance information, key sentences are extracted from the corresponding candidate sentences.

2. The method according to claim 1, wherein The determining of the initial importance information of the candidate sentence based on the paragraph position of the paragraph to which the candidate sentence belongs in the text and / or the sentence position of the candidate sentence in the paragraph to which the candidate sentence belongs includes: Obtaining importance setting information; the importance setting information includes: an importance value and position condition information corresponding to the importance value; in the importance setting information, different importance values are set for different paragraph positions and / or different sentence positions; Obtaining first position condition information that the candidate sentence meets in the importance setting information; The importance value corresponding to the first location condition information is determined as the initial importance information.

3. The method according to claim 2, wherein: The text to be processed is the text involved in the text processing task; the obtaining of importance setting information includes: Obtaining importance setting information corresponding to the text processing task; the importance setting information corresponding to the text processing task is determined based on the task objective of the text processing task.

4. The method according to claim 1, wherein The obtaining of the text to be processed and the graph model constructed based on the text includes: Obtaining the text and the candidate sentences; Determining a semantic representation vector of the candidate sentence; Determining the semantic similarity between any two candidate sentences based on the semantic representation vectors of any two candidate sentences; The graph model is constructed with the candidate sentences as nodes and the semantic similarities as edges.

5. The method according to claim 4, wherein After obtaining the text and the candidate sentences, the method includes: In response to a semantically incomplete first sentence among the candidate sentences, determining sentences to be merged from second sentences adjacent to the first sentence in the text, the candidate sentences including the first sentence and the second sentence; The first statement and the statement to be merged are merged.

6. The method according to claim 5, wherein: The first statement satisfies at least one of the following conditions: The number of characters included is less than or equal to the character count threshold; Missing sentence components; Semantic incompleteness is determined based on a semantic completeness judgment model.

7. The method according to claim 4, wherein: Determining the semantic representation vector of the candidate sentence includes: Get each word in the word library; Determine the TF-IDF value of each of the words in the candidate sentence; The semantic representation vector is determined according to the TF-IDF value.

8. The method according to claim 4, wherein The method further comprises: In response to detecting that there are edges in the graph model whose semantic similarity is less than or equal to a similarity threshold, the edges whose semantic similarity is less than or equal to the similarity threshold are deleted.

9. The method according to claim 1, wherein The step of extracting key sentences from the corresponding candidate sentences according to the target importance information includes: determining a target sentence from the candidate sentences according to the target importance information; When the number of the text is one, the target sentence is determined as the key sentence.

10. The method according to claim 9, wherein: The step of determining a target sentence from the candidate sentences based on the target importance information further includes: In the case where there are multiple texts, extracting a first target sentence from target sentences in each of the texts according to a sentence extraction strategy; performing deduplication processing on the first target sentence to obtain a deduplicated first target sentence; The first target sentence after deduplication is determined as the key sentence.

11. The method according to claim 10, wherein: The statement extraction strategy includes at least one of the following: The number of first target sentences extracted from each text is greater than or equal to the sentence number threshold; The proportion of the first target sentence extracted from each text is greater than or equal to the sentence proportion threshold.

12. The method according to claim 10, wherein: The performing deduplication processing on the first target sentence to obtain the deduplicated first target sentence includes: performing clustering processing on the first target sentences in each of the texts to obtain a plurality of clusters; A first target sentence is selected from each of the clusters, and unselected first target sentences are deleted.

13. The method according to claim 1, wherein The text to be processed is the text involved in the text processing task; The text processing task includes at least one of the following: Question answering tasks, text retrieval tasks, conference record tasks, and summary generation tasks.

14. The method according to claim 1 or 13, wherein: The text to be processed is the text involved in the summary generation task; the method further includes: Inputting the key sentence into a summary generation model to obtain a generated text output by the summary generation model; The generated text is determined as the summary text corresponding to the text.

15. The method according to claim 1, wherein The text to be processed includes at least one of the following: The text obtained after performing speech recognition on the audio; The text obtained after image recognition on the image; Original text.

16. A device for extracting key sentences from a text, the device comprising: A first acquisition module, configured to acquire a text to be processed and a graph model constructed based on the text; In the graph model, candidate sentences in the text are used as nodes, and semantic similarities between the candidate sentences are used as edges; A first determining module, configured to determine initial importance information of the candidate sentence based on the paragraph position of the paragraph to which the candidate sentence belongs in the text and / or the sentence position of the candidate sentence in the paragraph to which it belongs; A processing module, configured to perform iterative importance calculation based on the initial importance information and the graph model to obtain target importance information of each candidate sentence; The extraction module is used to extract key sentences from the corresponding candidate sentences according to the target importance information.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 15.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 15.

19. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 15.

Images