Text abstract generation method and device based on semantic blocks, equipment and storage medium

Abstract

The embodiment of the application belongs to the field of artificial intelligence, and relates to a text abstract generation method based on semantic blocks, comprising the following steps: obtaining sample data, performing component syntax analysis on the sample data to obtain a syntax tree, determining semantic blocks according to the syntax tree, inputting the semantic blocks into a pre-constructed initial text abstract model, encoding the semantic blocks to obtain semantic block codes, performing weighted calculation on the semantic block codes to obtain semantic block sequence vectors, performing attention mechanism calculation on the semantic block sequence vectors to obtain semantic block relationship vectors, inputting the semantic block relationship vectors into a CRF layer for training, outputting a predicted classification result, calculating a loss function, adjusting model parameters of the initial abstract generation model based on the loss function, ending iteration, generating a text abstract model, inputting a text to be processed into the text abstract model, and outputting a text abstract. The application also provides a text abstract generation device based on semantic blocks, equipment and a storage medium. The application can make the abstract more concise and refined.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for generating text summaries based on semantic blocks. Background Technology

[0002] With the development of internet technology and the rise of self-media platforms, the amount of information on the internet has grown exponentially. People are exposed to massive amounts of information every day, such as current affairs news, trending topics, social chat, lectures, reports, and academic papers. Faced with this complex and diverse internet environment, the ability to quickly and effectively obtain valuable information is crucial. To alleviate the reading pressure caused by information overload and help users quickly obtain useful information from the internet, automatic text summarization technology has emerged. Automatic text summarization technology can extract a concise and coherent short text reflecting the central content of the original text from a single text or a collection of texts, making it easier for users to obtain and search for the information they need and reducing reading pressure.

[0003] Currently, mainstream automatic text summarization technologies are divided into two types based on their generation methods: extractive and generative. Generative text summarization, as a form of text generation, inevitably suffers from poor readability, content selection errors, and OOV (Out of Vocabulary) issues. Traditional extractive methods, on the other hand, treat text summarization as a binary classification problem, determining whether each sentence in the text belongs to the summary content. The resulting summaries have advantages over generative methods, such as faithfulness to the original text and higher fluency. However, extractive methods often produce summaries that are not concise enough, containing redundant text and unnecessary information. This is likely due to extracting entire sentences as summaries, as even an important sentence may contain unnecessary information, and repeated text may appear in different sentences, leading to information redundancy in the final generated summary. Summary of the Invention

[0004] The purpose of this application is to propose a text summarization method, apparatus, device, and storage medium based on semantic blocks, so as to solve the technical problem that the text summaries generated in related technologies contain redundant text and unnecessary information.

[0005] To address the aforementioned technical problems, this application provides a text summarization method based on semantic blocks, employing the following technical solution:

[0006] Obtain sample data of text summarization, perform syntactic analysis on the sample data to obtain a syntactic tree, and determine semantic blocks based on the syntactic tree;

[0007] The semantic block is input into a pre-built initial text summarization model, wherein the initial text summarization model includes a BERT layer, an encoder layer, a selection mechanism layer, and a CRF layer;

[0008] The semantic block is encoded by the BERT layer to obtain the semantic block encoding;

[0009] The semantic block encoding is weighted and calculated through the selection mechanism layer to obtain a semantic block sequence vector;

[0010] The semantic block sequence vector is input into the encoder layer for attention mechanism calculation to obtain the semantic block relation vector;

[0011] The semantic block relation vector is input into the CRF layer for training, and the predicted classification result is output.

[0012] The loss function is calculated based on the language prediction classification result and the sequence label. The model parameters of the initial summary generation model are adjusted based on the loss function. When the iteration termination condition is met, a text summarization model is generated based on the model parameters.

[0013] Input the text to be processed into the text summarization model, and output the text summary corresponding to the text to be processed.

[0014] Furthermore, the step of determining semantic blocks based on the syntax tree includes:

[0015] Traverse the syntax tree and obtain clause labels from the syntax tree;

[0016] Determine whether the clause node corresponding to the clause tag is the highest-level ancestor node. If the clause node is the highest-level ancestor node, then the clause node is determined to be a semantic block.

[0017] Furthermore, the step of encoding the semantic block through the BERT layer to obtain the semantic block encoding includes:

[0018] The text sequence corresponding to the sample data is input into the BERT layer to obtain the text sequence encoding;

[0019] Obtain the text sequence encoding corresponding to the semantic block and output the semantic block encoding.

[0020] Furthermore, the selection mechanism layer includes an average pooling layer and a multilayer perceptron layer. The step of weighting the semantic block encoding through the selection mechanism layer to obtain the semantic block sequence vector includes:

[0021] The semantic block encoding is pooled by the average pooling layer to obtain the pooled semantic block encoding.

[0022] The sentence encoding is pooled by the average pooling layer to generate a document encoding corresponding to the sample data, wherein the sentence encoding is generated by the BERT layer based on each sentence of the sample data;

[0023] The sentence encoding, the pooled semantic block encoding, and the document encoding are input into the multilayer perceptron layer to construct a weighted representation matrix;

[0024] Based on the pooled semantic block encoding and the weighted representation matrix, the semantic block sequence vector is calculated.

[0025] Furthermore, the encoder layer includes a multi-head attention mechanism layer and a feedforward neural network layer. The step of inputting the semantic block sequence vector into the encoder layer for attention mechanism calculation to obtain the semantic block relation vector includes:

[0026] The semantic block sequence vector is input into the multi-head attention mechanism layer to learn the multi-head attention mechanism and obtain the multi-head attention vector;

[0027] The sum of the multi-head attention vector and the semantic block sequence vector is normalized to obtain an intermediate variable;

[0028] The intermediate variables are calculated using the feedforward neural network layer to obtain the semantic block relation vector.

[0029] Furthermore, the step of inputting the semantic block relation vector into the CRF layer for training and outputting the predicted classification result includes:

[0030] The semantic block relation vector is input into the CRF layer, and the semantic block relation vector is mapped and calculated to obtain the score sequence of the semantic block;

[0031] The predicted label probability of the semantic block is obtained based on the score sequence;

[0032] The semantic block is obtained by decoding the semantic block relation vector through beam search;

[0033] The predicted classification result corresponding to the semantic block is obtained based on the predicted label probability.

[0034] Furthermore, after the step of obtaining the predicted classification result corresponding to the semantic block based on the predicted label probability, the method further includes:

[0035] Based on the predicted classification results, the semantic blocks are reorganized according to preset rules to obtain a sample summary.

[0036] To address the aforementioned technical problems, this application also provides a text summarization device based on semantic blocks, employing the following technical solution:

[0037] The syntactic analysis module is used to acquire sample data of text summarization, perform syntactic analysis on the sample data to obtain a syntactic tree, and determine semantic blocks based on the syntactic tree;

[0038] The input module is used to input the semantic block into a pre-built initial text summarization model, wherein the initial text summarization model includes a BERT layer, an encoder layer, a selection mechanism layer, and a CRF layer;

[0039] The encoding module is used to encode the semantic block through the BERT layer to obtain the semantic block encoding;

[0040] The selection mechanism module is used to perform weighted calculations on the semantic block encoding through the selection mechanism layer to obtain a semantic block sequence vector.

[0041] The attention mechanism module is used to input the semantic block sequence vector into the encoder layer for attention mechanism calculation to obtain the semantic block relation vector.

[0042] The classification prediction module is used to input the semantic block relation vector into the CRF layer for training and output the predicted classification result.

[0043] The calculation and adjustment module is used to calculate a loss function based on the language prediction classification result and the sequence label, adjust the model parameters of the initial summary generation model based on the loss function, and generate a text summarization model based on the model parameters when the iteration termination condition is met.

[0044] The text summarization module is used to input the text to be processed into the text summarization model and output a text summary corresponding to the text to be processed.

[0045] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0046] The computer device includes a memory and a processor, the memory storing computer-readable instructions, and the processor executing the computer-readable instructions to implement the steps of the semantic block-based text summarization method as described above.

[0047] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0048] The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the semantic block-based text summarization method described above.

[0049] Compared with the prior art, the embodiments of this application have the following main advantages:

[0050] This application obtains sample data of text summarization, performs component syntactic analysis on the sample data to obtain a syntactic tree, and determines semantic blocks based on the syntactic tree. The semantic blocks are then input into a pre-constructed initial text summarization model, which includes a BERT layer, an encoder layer, a selection mechanism layer, and a CRF layer. The BERT layer encodes the semantic blocks to obtain semantic block codes. The selection mechanism layer performs weighted calculations on the semantic block codes to obtain semantic block sequence vectors. These semantic block sequence vectors are then input into the encoder layer for attention mechanism calculations to obtain semantic block relation vectors. Finally, these semantic block relation vectors are input into the CRF layer for training, outputting a predicted classification result. Based on the linguistic prediction classification results and sequence labels, a loss function is calculated. The model parameters of the initial summary generation model are adjusted based on the loss function. When the iteration termination condition is met, a text summarization model is generated based on the model parameters. The text to be processed is input into the text summarization model, and the output is a text summary corresponding to the text to be processed. This application uses syntactic analysis to distinguish between important and unimportant semantic blocks in a sentence at a finer granularity. By generating text summaries through the trained text summarization model, important information can be extracted from the given text and combined into a summary based on semantic blocks, making the summary more concise and refined, and further ensuring the effectiveness of information acquisition. Attached Figure Description

[0051] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0053] Figure 2 This is a flowchart of an embodiment of the semantic block-based text summarization method according to this application;

[0054] Figure 3 This is a schematic diagram of a specific implementation of the syntax tree of this application;

[0055] Figure 4 This is a schematic diagram of another specific implementation of the syntax tree in this application;

[0056] Figure 5 This is a structural schematic diagram of an embodiment of the text summarization model according to this application;

[0057] Figure 6 This is a schematic diagram of the structure of an embodiment of the semantic block-based text summarization apparatus according to this application;

[0058] Figure 7 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0060] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0061] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0062] This application provides a text summarization method based on semantic blocks, which can be applied to, for example... Figure 1 In the system architecture 100 shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 is used as a medium to provide a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0063] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0064] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0065] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0066] It should be noted that the semantic block-based text summarization method provided in this application is generally executed by a server / terminal device, and correspondingly, the semantic block-based text summarization device is generally located in the server / terminal device.

[0067] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0068] Continue to refer to Figure 2 The flowchart illustrates an embodiment of a semantic block-based text summarization method according to this application, including the following steps:

[0069] Step S201: Obtain sample data of text summary, perform syntactic analysis on the sample data to obtain a syntactic tree, and determine semantic blocks based on the syntactic tree.

[0070] In this embodiment, the sample data includes the original text corresponding to the text summary and the sequence labels of each sentence in the original text. The sequence labels can be annotated using methods such as BIO, BIOE, BIOES, and BMES. Taking BIOE as an example, the sequence labels are divided into four categories: B, I, E, and O. B represents the beginning of the summary segment, I represents the middle sentence of the summary segment, E represents the end sentence of the summary segment, and O represents a non-summary sentence.

[0071] Sample data can be obtained from public datasets, such as the CNN / DailyMail dataset, the LCSTS dataset, and the NLPCC2017 dataset. Among them, the CNN / DailyMail dataset is an English dataset, while the LCSTS dataset and the NLPCC2017 dataset are Chinese text datasets.

[0072] Syntactic analysis includes constituent parsing and dependency parsing. In this embodiment, constituent parsing is used to traverse each sentence in the sample data to obtain a syntactic tree. See the generated syntactic tree for details. Figure 3 and Figure 4 As shown, the labels of the nodes in the syntax tree correspond to the components contained in each sentence. Among them, ADVP is an adverbial phrase, NP is a noun phrase, VP is a verb phrase, VBD is a past tense verb, PP is a prepositional phrase, S is a simple declarative sentence, SBAR is a clause, WHNP is a WH noun phrase, and CC is a conjunction.

[0073] There are two main types of tags in the constituent syntactic analysis tag set: phrase tags and clause tags. Phrase tags include ADVP, NP, VP, PP, etc., while clause tags include S, SBAR, etc. Since clauses contain more complete information than phrases, this embodiment can use clause tags to determine semantic blocks, thereby achieving the purpose of sentence segmentation.

[0074] A semantic block is a next-level semantic unit of a sentence. Correspondingly, segmenting a sentence means dividing a sentence into several semantic blocks. In this embodiment, a semantic block contains at least one clause.

[0075] Specifically, the steps for determining semantic blocks based on the syntax tree include:

[0076] Traverse the syntax tree and retrieve clause labels from the syntax tree;

[0077] Determine whether the clause node corresponding to the clause tag is the highest-level ancestor node. If the clause node is the highest-level ancestor node, then the clause node is determined to be a semantic block.

[0078] like Figure 3 and Figure 4 As shown, each node in the syntax tree that marks a clause tag, i.e., the nodes of S and SBAR, is considered a candidate for a semantic block. If an ancestor node of a clause is also a clause node, then the highest-level ancestor node (excluding the root node) is selected as the semantic block to include more complete information. In the diagram, solid circles represent selected clause nodes that are semantic blocks, while dashed circles represent unselected clause nodes that are not semantic blocks.

[0079] It should be understood that multiple semantic blocks are allowed in a sentence. If no clause can be found in a sentence, the entire sentence is used as the semantic block.

[0080] Step S202: Input the semantic block into the pre-built initial text summarization model, wherein the initial text summarization model includes a BERT layer, an encoder layer, a selection mechanism layer, and a CRF layer.

[0081] In this embodiment, see Figure 5 As shown, the initial text summarization model includes at least a BERT layer, an encoder layer, a selection mechanism layer, and a CRF layer. The BERT layer (Bidirectional Encoder Representations from Transformers) aims to train and learn the rich semantic information representation contained in the text using a large-scale unlabeled corpus, and then fine-tune the semantic representation of the text in a specific NLP (Natural Language Processing) task before finally applying it to that NLP task.

[0082] The selection mechanism layer includes an average pooling layer and an MLP (Multilayer Perceptron) layer. The average pooling layer is used to perform average pooling operations, specifically taking the average value of the pooling region as the feature value output. Average pooling can reduce the error caused by the increased variance of the estimated value due to the limited neighborhood size. MLP is a neural network-based algorithm model consisting of an input layer, a hidden layer, and an output layer. The MLP layer is used to construct a weighted representation of semantic blocks.

[0083] A Conditional Random Field (CRF) is an undirected graphical model. A set of random variables is called a stochastic process, and a stochastic process indexed within a spatial set of variables is called a random field. In sequence labeling tasks, assuming x and y are a set of random variables, and p(y|x) represents the conditional probability distribution of the output variable y given a set of inputs x, where y represents an MRF (Markov random field), then p(y|x) is called a Conditional Random Field.

[0084] Step S203: Encode the semantic block using the BERT layer to obtain the semantic block encoding.

[0085] Specifically, the text sequence corresponding to the sample data is input into the BERT layer to obtain the text sequence encoding, the text sequence encoding corresponding to the semantic block is obtained, and the semantic block encoding is output.

[0086] In this embodiment, it is assumed that the input sample data is document d, and document d contains n sentences. m characters The i-th sentence contains l semantic blocks, represented as follows: Sentences i In the above, the j-th semantic block composed of k characters is denoted as .

[0087] Before inputting the text sequence corresponding to the sample data into the BERT layer, the sample data needs to be cleaned to remove irrelevant characters, such as special punctuation marks, whitespace characters, newlines, spaces, etc.

[0088] After cleaning the data, the sample data is divided into sentences, with each sentence preceded by a [cls] symbol and followed by a [sep] symbol at the end. The [cls] symbol outputs a vector representing the sentence's semantics, and the [sep] symbol indicates the end of the sentence.

[0089] Before inputting the sample data into the BERT layer, the sample data needs to be vectorized to obtain a text sequence. Therefore, the text sequence of document d is... The text sequence contains the [cls] and [sep] symbols. Inputting this text sequence into the BERT layer yields the text sequence encoding, which is represented as: .

[0090] Obtain the text sequence encoding corresponding to the semantic block, output the semantic block encoding through the BERT layer, and use the semantic block c i,j For example, the corresponding semantic block encoding is: .

[0091] After passing through the BERT layer, the [cls] vector code preceding each sentence becomes the vector code corresponding to that sentence, which is the sentence code, denoted as S. i .

[0092] Step S204: The semantic block encoding is weighted by selecting a mechanism layer to obtain a semantic block sequence vector.

[0093] Specifically, the semantic block encoding is pooled using an average pooling layer to obtain the pooled semantic block encoding; the sentence encoding is pooled using an average pooling layer to generate the document encoding corresponding to the sample data, wherein the sentence encoding is generated by the BERT layer based on each sentence of the sample data; the sentence encoding, the pooled semantic block encoding, and the document encoding are input into a multilayer perceptron layer to construct a weighted representation matrix; based on the pooled semantic block encoding and the weighted representation matrix, the semantic block sequence vector is calculated.

[0094] In this embodiment, the average pooling layer has the ability to reduce dimensionality. It performs average pooling on the semantic block encoding output by the BERT layer to obtain the average-pooled semantic block encoding, denoted as... The formula is as follows:

[0095]

[0096] Average pooling is used to pool the sentence encoding, so that the sentence encoding generates the document encoding D. The specific calculation formula is as follows:

[0097]

[0098] Where n is the number of sentences in document d, S i Encode the i-th sentence.

[0099] Encode the sentence S i Document encoding D and semantic block encoding after average pooling The input is processed by the MLP layer, which constructs a weighted representation of semantic blocks through a selection mechanism, i.e., constructs a weighted representation matrix to highlight important semantic blocks in the document.

[0100] In this embodiment, a weighted representation matrix c is constructed based on sentence encoding, pooled semantic block encoding, and document encoding. weight Then, based on the weighted representation matrix c weight Multiplying the semantic block encoding by the average pooling result in the semantic block vector C. i,j The formula is as follows:

[0101] ;

[0102]

[0103] Where U, V, and W1 are the parameter matrices of the MLP layer. This represents multiplying by corresponding elements.

[0104] According to semantic block vector C i,j Then document d can be represented as a sequence of semantic block vectors, i.e., semantic block sequence vector C, denoted as:

[0105]

[0106] Among them, C 1,1 C represents the first semantic block of the first sentence. 1,l1 C represents the l-th semantic block of the first sentence. n,ln This represents the first semantic block of the nth sentence.

[0107] Step S205: Input the semantic block sequence vector into the encoder layer for attention mechanism calculation to obtain the semantic block relation vector.

[0108] In this embodiment, the encoder layer is a Transformer encoder used to capture the relationships between semantic blocks, including a multi-head attention mechanism layer and a feedforward neural network layer.

[0109] The attention mechanism is similar to the characteristics of human vision: observing a specific part of a target scene and focusing attention on that part. Traditional attention mechanisms first create three vectors for the model input: a query vector (Q), a key vector (K), and a value vector (V). Generally, calculating attention involves three parts: weight calculation, weight normalization, and weighted summation of the normalized weights and their corresponding values ​​(V) to obtain the attention value. In contrast, the multi-head attention mechanism projects the Q, K, and V vectors separately by each head, calculates the attention value, and then concatenates them to obtain the multi-head attention, which can improve the model's ability to capture features at different locations.

[0110] Specifically, the semantic block sequence vector is input into the multi-head attention mechanism layer for multi-head attention mechanism learning to obtain the multi-head attention vector. The sum of the multi-head attention vector and the semantic block sequence vector is then normalized to obtain an intermediate variable, calculated as follows:

[0111]

[0112] Here, H is an intermediate variable, Multihead(C) is the multihead attention vector calculated by the multihead attention mechanism in the Transformer encoder, and LN represents the hierarchical normalization operation.

[0113] In this embodiment, the semantic block relation vector is obtained by calculating the intermediate variables through a feedforward neural network layer. The calculation formula is as follows:

[0114]

[0115] Here, FFN(·) is a feedforward neural network consisting of two linear transformations, with ReLU as the activation function in between. It is a sequence of semantic block relation vectors processed by the Transformer encoder.

[0116] Step S206: Input the semantic block relation vector into the CRF layer for training and output the predicted classification result.

[0117] In this embodiment, the text summarization is transformed into a sequence labeling problem through a CRF layer for training. The specific steps include:

[0118] The semantic block relation vector is input into the CRF layer, and the semantic block relation vector is mapped and calculated to obtain the score sequence of the semantic block;

[0119] The predicted label probability of the semantic block is obtained based on the score sequence;

[0120] The semantic block is obtained by decoding the semantic block relation vector through beam search;

[0121] The predicted classification result corresponding to the semantic block is obtained based on the predicted label probability.

[0122] The semantic block relation vector after passing through the Transformer encoder The following mapping calculation is performed, using the following formula:

[0123]

[0124] Where W2 is the training parameter matrix, b is the bias term, and z i This represents the score sequence of the i-th semantic block belonging to the B, I, O, E tags.

[0125] The above score sequence z i The CRF layer is input for training to obtain the predicted label probabilities.

[0126] Specifically, the parameters of the CRF layer are a (k+2)-power matrix T. Since a start state and a termination state need to be added to the beginning and end of the sentence respectively, 2 needs to be added, where T... ij This represents the transition probability from the i-th annotation position to the j-th annotation position, thus enabling full utilization of previously annotated information when annotating a certain position. Assume the semantic block z to be identified is represented as (x1, x2, ..., xj). N-1 A tag sequence y of length equal to the number of semantic blocks in a document is (y1, y2, ..., y...). N If the score is 1, the corresponding score is as follows:

[0127]

[0128] Where T is the state transition matrix, T yi，yi+1 Indicates from label y i Transfer to label y i+1 The fractions, where y1 and y N These are the start and end tags of the document; therefore, the dimension of T is (k+2)*(k+2) (where k is the number of tags); P i,yi This represents the emission matrix, where y is the label predicted by the model for the i-th semantic block based on the current state. i The probability value.

[0129] The scores are normalized using the softmax activation function to obtain the probability of label y, calculated as follows:

[0130]

[0131] in, This represents the possible annotation sequence corresponding to semantic block z. In other words, each annotation sequence corresponding to the semantic block has a score and a probability. The goal is to maximize the probability of the true annotation sequence corresponding to the semantic block.

[0132] After calculating the predicted label probability, the semantic block relation vector is decoded by Beam Search to obtain the semantic block. The predicted classification result corresponding to the semantic block is obtained according to the predicted label probability, that is, each semantic block is classified as B, I, E or O.

[0133] In beam search, there is a hyperparameter called beam size, denoted as k. At the first time step, the k words with the highest conditional probabilities are selected as the first word of the candidate output sequence. At each subsequent time step, based on the output sequence of the previous step, the k words with the highest conditional probabilities from all combinations are selected as the candidate output sequence for that time step. This process maintains a set of k candidates, and finally, the optimal candidate is chosen from these k candidates.

[0134] In this embodiment, after outputting the predicted classification result, the semantic block is reorganized according to the predicted classification result according to the preset rules to obtain the sample summary.

[0135] Specifically, the predicted classification results output by the CRF layer are reorganized, ignoring semantic blocks labeled 'O' and also ignoring invalid labels (such as standalone 'I' or 'E' labels). 'B' is used as the beginning of a segment, 'I' is assigned to the segment containing the previous 'B', and 'E' is used as the end of a segment. This reorganization of semantic blocks is performed according to the above rules to generate sample summaries corresponding to the sample data.

[0136] Step S207: Calculate the loss function based on the predicted classification results and sequence labels, adjust the model parameters of the initial summary generation model based on the loss function, and generate a text summarization model based on the model parameters when the iteration termination condition is met.

[0137] In this embodiment, a pre-built initial text summarization model is used. Sample data is input into the initial text summarization model for training. After one round of training, the loss function of the initial text summarization model is calculated, and the loss function value is obtained. The model parameters are adjusted based on the loss function value, and iterative training continues. When the model is trained to a certain extent, its performance reaches its optimal state, and the loss function value can no longer decrease, i.e., it converges. The convergence is determined by calculating the loss function value in the two iterations. If the loss function value is still changing, the training sample data is selected again for iterative training; if the loss function value does not change significantly, the model can be considered converged.

[0138] The model converges when the iteration termination condition is met. After the model converges, a text summarization model is generated based on the final model parameters.

[0139] Step S208: Input the text to be processed into the text summarization model and output the text summary corresponding to the text to be processed.

[0140] In this embodiment, the text to be processed is input into the trained text summarization model, and then the text summary is output.

[0141] This application uses syntactic analysis to distinguish between important and unimportant semantic blocks in a sentence at a finer granularity. By generating text summaries through a trained text summarization model, important information can be extracted from a given text and combined into a summary based on semantic blocks, making the summary more concise and refined, and further ensuring the effectiveness of information acquisition.

[0142] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0143] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0144] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0145] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0146] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0147] Further reference Figure 6 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a semantic block-based text summarization generation apparatus, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0148] like Figure 6As shown, the semantic block-based text summarization generation device 600 described in this embodiment includes: a syntactic analysis module 601, an input module 602, an encoding module 603, a selection mechanism module 604, an attention mechanism module 605, a classification prediction module 606, a calculation and adjustment module 607, and a text summarization module 608. Wherein:

[0149] The syntactic analysis module 601 is used to acquire sample data of text summarization, perform syntactic analysis on the sample data to obtain a syntactic tree, and determine semantic blocks based on the syntactic tree;

[0150] The input module 602 is used to input the semantic block into a pre-built initial text summarization model, wherein the initial text summarization model includes a BERT layer, an encoder layer, a selection mechanism layer, and a CRF layer;

[0151] Encoding module 603 is used to encode the semantic block through the BERT layer to obtain semantic block encoding;

[0152] The selection mechanism module 604 is used to perform weighted calculations on the semantic block encoding through the selection mechanism layer to obtain a semantic block sequence vector.

[0153] Attention mechanism module 605 is used to input the semantic block sequence vector into the encoder layer for attention mechanism calculation to obtain semantic block relation vector;

[0154] The classification prediction module 606 is used to input the semantic block relation vector into the CRF layer for training and output the prediction classification result;

[0155] The calculation and adjustment module 607 is used to calculate a loss function based on the language prediction classification result and the sequence label, adjust the model parameters of the initial summary generation model based on the loss function, and generate a text summary model based on the model parameters when the iteration termination condition is met.

[0156] The text summarization module 608 is used to input the text to be processed into the text summarization model and output a text summary corresponding to the text to be processed.

[0157] The aforementioned semantic block-based text summarization device, through syntactic analysis, can distinguish between important and unimportant semantic blocks in a sentence at a finer granularity. By generating text summaries using a trained text summarization model, it can extract important information from a given text and combine it into a summary based on semantic blocks, making the summary more concise and refined, and further ensuring the effectiveness of information acquisition.

[0158] In this embodiment, the syntactic analysis module 602 includes a traversal submodule and a semantic block determination submodule, wherein:

[0159] The traversal submodule is used to traverse the syntax tree and obtain clause labels from the syntax tree;

[0160] The semantic block determination submodule is used to determine whether the clause node corresponding to the clause tag is the highest-level ancestor node. If the clause node is the highest-level ancestor node, then the clause node is determined to be a semantic block.

[0161] This embodiment uses clause tags to determine semantic blocks, which makes the information contained in the semantic blocks more complete.

[0162] In some optional implementations of this embodiment, the encoding module 604 includes an encoding submodule and an output submodule, wherein:

[0163] The encoding submodule is used to input the text sequence corresponding to the sample data into the BERT layer to obtain the text sequence encoding;

[0164] The output submodule is used to obtain the text sequence encoding corresponding to the semantic block and output the semantic block encoding.

[0165] In this embodiment, the selection mechanism module 605 includes a pooling submodule, a construction submodule, and a computation submodule.

[0166] The pooling submodule is used to perform pooling operations on the semantic block encoding through the average pooling layer to obtain the pooled semantic block encoding;

[0167] The pooling submodule is also used to perform pooling operations on the sentence encoding through the average pooling layer to generate document encoding corresponding to the sample data, wherein the sentence encoding is generated by the BERT layer based on each sentence of the sample data;

[0168] The construction submodule is used to input the sentence encoding, the pooled semantic block encoding, and the document encoding into the multilayer perceptron layer to construct a weighted representation matrix;

[0169] The computation submodule is used to calculate the semantic block sequence vector based on the pooled semantic block encoding and the weighted representation matrix.

[0170] This embodiment reduces feature dimensionality through average pooling and further obtains important semantic blocks through weighted representation, thus avoiding unnecessary information in subsequent text summarization.

[0171] In this embodiment, the attention mechanism module 606 includes a learning submodule, a normalization submodule, and a computation submodule:

[0172] The learning submodule is used to input the semantic block sequence vector into the multi-head attention mechanism layer to learn the multi-head attention mechanism and obtain the multi-head attention vector;

[0173] The normalization submodule is used to normalize the sum of the multi-head attention vector and the semantic block sequence vector to obtain an intermediate variable;

[0174] The computation submodule is used to compute the intermediate variables through the feedforward neural network layer to obtain the semantic block relation vector.

[0175] This embodiment uses an attention mechanism to capture the relationships between semantic blocks.

[0176] In this embodiment, the classification prediction module 607 includes a mapping submodule, a prediction submodule, a decoding submodule, and a classification submodule. The mapping submodule is used to input the semantic block relationship vector into the CRF layer, perform mapping calculation on the semantic block relationship vector, and obtain the score sequence of the semantic block. The prediction submodule is used to obtain the predicted label probability of the semantic block based on the score sequence. The decoding submodule is used to decode the semantic block relationship vector to obtain the semantic block. The classification submodule is used to obtain the predicted classification result corresponding to the semantic block based on the predicted label probability.

[0177] In some optional implementations, the classification prediction module 607 further includes a recombination submodule, which is used to recombine the semantic block according to preset rules based on the predicted classification result to obtain a sample summary.

[0178] This embodiment obtains a sample summary by reorganizing semantic blocks, which makes the summary more concise and refined.

[0179] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 7 , Figure 7 This is a basic structural block diagram of the computer device in this embodiment.

[0180] The computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected via a system bus. It should be noted that only the computer device 7 with components 71-73 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0181] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0182] The memory 71 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 71 may be an internal storage unit of the computer device 7, such as the hard disk or memory of the computer device 7. In other embodiments, the memory 71 may also be an external storage device of the computer device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 7. Of course, the memory 71 may also include both the internal storage unit and its external storage device of the computer device 7. In this embodiment, the memory 71 is typically used to store the operating system and various application software installed on the computer device 7, such as computer-readable instructions for semantic block-based text summarization generation methods. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or will be output.

[0183] In some embodiments, the processor 72 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 72 is typically used to control the overall operation of the computer device 7. In this embodiment, the processor 72 is used to execute computer-readable instructions stored in the memory 71 or to process data, for example, to execute computer-readable instructions of the semantic block-based text summarization generation method.

[0184] The network interface 73 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 7 and other electronic devices.

[0185] This embodiment implements the steps of the semantic block-based text summarization method described above by executing computer-readable instructions stored in memory through the processor. Through syntactic analysis, important and unimportant semantic blocks in a sentence can be distinguished at a finer granularity. By generating text summaries through a trained text summarization model, important information can be extracted from a given text and combined into a summary based on semantic blocks, making the summary more concise and refined, and further ensuring the effectiveness of information acquisition.

[0186] This application also provides another implementation method, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to perform the steps of the semantic block-based text summarization method described above. Through syntactic analysis, important and unimportant semantic blocks in a sentence can be distinguished at a finer granularity. By generating text summaries through a trained text summarization model, important information can be extracted from a given text and combined into a summary based on semantic blocks, making the summary more concise and refined, and further ensuring the effectiveness of information acquisition.

[0187] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0188] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A text summarization method based on semantic blocks, characterized in that, Includes the following steps: Obtain sample data of text summarization, perform syntactic analysis on the sample data to obtain a syntactic tree, and determine semantic blocks based on the syntactic tree; The semantic block is input into a pre-built initial text summarization model, wherein the initial text summarization model includes a BERT layer, an encoder layer, a selection mechanism layer, and a CRF layer; The semantic block is encoded by the BERT layer to obtain the semantic block encoding; The semantic block encoding is weighted and calculated through the selection mechanism layer to obtain a semantic block sequence vector; The semantic block sequence vector is input into the encoder layer for attention mechanism calculation to obtain the semantic block relation vector; The semantic block relation vector is input into the CRF layer for training, and the predicted classification result is output. The loss function is calculated based on the predicted classification results and the sequence labels. The model parameters of the initial summary generation model are adjusted based on the loss function. When the iteration termination condition is met, a text summarization model is generated based on the model parameters. Input the text to be processed into the text summarization model, and output the text summary corresponding to the text to be processed; The selection mechanism layer includes an average pooling layer and a multilayer perceptron layer. The step of weighting the semantic block encoding through the selection mechanism layer to obtain the semantic block sequence vector includes: The semantic block encoding is pooled by the average pooling layer to obtain the pooled semantic block encoding. The sentence encoding is pooled by the average pooling layer to generate a document encoding corresponding to the sample data, wherein the sentence encoding is generated by the BERT layer based on each sentence of the sample data; The sentence encoding, the pooled semantic block encoding, and the document encoding are input into the multilayer perceptron layer to construct a weighted representation matrix; Multiply the pooled semantic block encoding with the weighted representation matrix to obtain a semantic block vector, and concatenate the semantic block vector to obtain a semantic block sequence vector.

2. The text summarization method based on semantic blocks according to claim 1, characterized in that, The step of determining semantic blocks based on the syntax tree includes: Traverse the syntax tree and obtain clause labels from the syntax tree; Determine whether the clause node corresponding to the clause tag is the highest-level ancestor node. If the clause node is the highest-level ancestor node, then the clause node is determined to be a semantic block.

3. The text summarization method based on semantic blocks according to claim 1, characterized in that, The step of encoding the semantic block through the BERT layer to obtain the semantic block encoding includes: The text sequence corresponding to the sample data is input into the BERT layer to obtain the text sequence encoding; Obtain the text sequence encoding corresponding to the semantic block and output the semantic block encoding.

4. The text summarization method based on semantic blocks according to claim 1, characterized in that, The encoder layer includes a multi-head attention mechanism layer and a feedforward neural network layer. The step of inputting the semantic block sequence vector into the encoder layer for attention mechanism calculation to obtain the semantic block relation vector includes: The semantic block sequence vector is input into the multi-head attention mechanism layer to learn the multi-head attention mechanism and obtain the multi-head attention vector; The sum of the multi-head attention vector and the semantic block sequence vector is normalized to obtain an intermediate variable; The intermediate variables are calculated using the feedforward neural network layer to obtain the semantic block relation vector.

5. The text summarization method based on semantic blocks according to claim 1, characterized in that, The step of inputting the semantic block relation vector into the CRF layer for training and outputting the predicted classification result includes: The semantic block relation vector is input into the CRF layer, and the semantic block relation vector is mapped and calculated to obtain the score sequence of the semantic block; The predicted label probability of the semantic block is obtained based on the score sequence; The semantic block is obtained by decoding the semantic block relation vector through beam search; The predicted classification result corresponding to the semantic block is obtained based on the predicted label probability.

6. The text summarization method based on semantic blocks according to claim 5, characterized in that, Following the step of obtaining the predicted classification result corresponding to the semantic block based on the predicted label probability, the method further includes: Based on the predicted classification results, the semantic blocks are reorganized according to preset rules to obtain a sample summary.

7. A text summarization device based on semantic blocks, characterized in that, include: The syntactic analysis module is used to acquire sample data of text summarization, perform syntactic analysis on the sample data to obtain a syntactic tree, and determine semantic blocks based on the syntactic tree; The input module is used to input the semantic block into a pre-built initial text summarization model, wherein the initial text summarization model includes a BERT layer, an encoder layer, a selection mechanism layer, and a CRF layer; The encoding module is used to encode the semantic block through the BERT layer to obtain the semantic block encoding; The selection mechanism module is used to perform weighted calculations on the semantic block encoding through the selection mechanism layer to obtain a semantic block sequence vector. The attention mechanism module is used to input the semantic block sequence vector into the encoder layer for attention mechanism calculation to obtain the semantic block relation vector. The classification prediction module is used to input the semantic block relation vector into the CRF layer for training and output the predicted classification result. The calculation and adjustment module is used to calculate a loss function based on the predicted classification result and the sequence label, adjust the model parameters of the initial summary generation model based on the loss function, and generate a text summarization model based on the model parameters when the iteration termination condition is met. The text summarization module is used to input the text to be processed into the text summarization model and output a text summary corresponding to the text to be processed. The selection mechanism module includes a pooling submodule, a construction submodule, and a computation submodule, among which: The pooling submodule is used to perform pooling operations on the semantic block encoding through the average pooling layer to obtain the pooled semantic block encoding; The pooling submodule is also used to perform pooling operations on the sentence encoding through the average pooling layer to generate document encoding corresponding to the sample data, wherein the sentence encoding is generated by the BERT layer based on each sentence of the sample data; The construction submodule is used to input the sentence encoding, the pooled semantic block encoding, and the document encoding into the multilayer perceptron layer to construct a weighted representation matrix; The computation submodule is used to multiply the pooled semantic block encoding and the weighted representation matrix to obtain a semantic block vector, and then concatenate the semantic block vector to obtain a semantic block sequence vector.

8. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the semantic block-based text summarization method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the semantic block-based text summarization method as described in any one of claims 1 to 6.

Images