A Chinese abstract generation and early warning method, system, device and medium

Abstract

The application relates to a Chinese abstract generation and early warning method, system, device and medium, comprising the following steps: obtaining a first Chinese text; inputting the first Chinese text into a GPT-2 model, determining a first abstract corresponding to the first Chinese text and a first category corresponding to the first abstract through the GPT-2 model, and the first category represents the theme of the first abstract; determining early warning information according to the first category, and the early warning information represents whether the second abstract contains sensitive information. The problems that the abstract generated by the GPT-2 model is generally not as good as the quality of the artificial summary, and some sensitive information is contained in the abstract are solved.

Description

Technical Field

[0001] This invention relates to the field of abstract generation technology, and in particular to a method, system, device and medium for generating and issuing early warnings for Chinese abstracts. Background Technology

[0002] Neural networks are increasingly becoming a mainstream technology in the field of summarization. Their advantage is that the model can automatically generate summaries. For example, the GPT-2 model can be used for summarization. The GPT-2 model is an architecture very similar to the Transformer decoder. However, GPT-2 is a huge Transformer-based language model. Although its summarization quality is comparable to that of models such as BART, its summarization speed is much faster than other models, which is very important in practical applications.

[0003] However, the summaries generated by the GPT-2 model are usually not as good as those summarized by humans, and the summaries may contain some sensitive information. Summary of the Invention

[0004] To overcome the problem that summaries generated by the GPT-2 model are generally of lower quality than those summarized manually, and that the summaries may contain sensitive information, this invention provides a method, system, device, and medium for generating and issuing early warnings for Chinese summaries.

[0005] Firstly, in order to solve the above-mentioned technical problems, the present invention provides a method for generating and issuing early warnings for Chinese abstracts, comprising the following steps:

[0006] Get the first Chinese text;

[0007] The first Chinese text is input into the GPT-2 model. The GPT-2 model determines the first summary corresponding to the first Chinese text and the first category corresponding to the first summary. The first category represents the topic of the first summary.

[0008] Based on the first category, the warning information is determined, and the warning information indicates whether the second summary contains sensitive information;

[0009] The GPT-2 model mentioned above was trained in the following way:

[0010] S11, obtain multiple second Chinese texts and the second summaries corresponding to the second Chinese texts;

[0011] S12, the initial model is trained based on multiple second Chinese texts to obtain the first predicted summary corresponding to each second Chinese text and the second category corresponding to each first predicted summary. For each second category, the second category represents the topic of the first predicted summary.

[0012] S13, For each first predicted summary, determine each first keyword in the first predicted summary based on the first predicted summary. For each first keyword, the first keyword is a word that has a degree of relevance to each word in the first predicted summary.

[0013] S14, for each first predicted summary, combine the first keywords into a third Chinese text;

[0014] S15, input each third Chinese text into the initial model, train the initial model, determine the second predicted summary corresponding to the third Chinese text, and the third category corresponding to the second predicted summary. The third category represents the topic of the second predicted summary.

[0015] S16, Based on each second prediction summary and each second summary, determine the loss value of the initial model, whereby the loss value characterizes the difference between each second prediction summary and each second summary;

[0016] S17. If the loss value meets the preset termination condition, the initial model that meets the preset termination condition is used as the GPT-2 model. If the loss value does not meet the preset termination condition, the network parameters of the initial model are adjusted, and the initial model is retrained according to the adjusted network parameters until the first loss value of the initial model meets the preset termination condition.

[0017] The beneficial effects of the Chinese summary generation and early warning method provided by this invention are as follows: A first summary corresponding to a first Chinese text and a first category corresponding to the first summary are obtained through the GPT-2 model. Early warning information is determined based on the first category, and the presence of sensitive information in the first summary is judged based on the early warning information. During training, the GPT-2 model extracts each first keyword from the generated first predicted summary and uses these first keywords to form a third Chinese text to train the initial model again. Since the quality of the first predicted summary is low, and each first keyword is a representative word in the first predicted summary (the first keyword is related to each word in the first predicted summary), the second predicted summary generated from the third Chinese text contains various problems of the low-quality first predicted summary. At this point, the loss value of the initial model is constructed based on the second predicted summary and the second summary, enabling the model to better learn the differences between low-quality and high-quality summaries, thereby improving the quality of the summaries generated by the GPT-2 model.

[0018] Based on the above technical solution, the method for generating and issuing early warnings of Chinese abstracts according to the present invention can be further improved as follows.

[0019] Furthermore, in the above method, for each first predicted summary, based on the first predicted summary, each first keyword in the first predicted summary is determined, including:

[0020] S21, For each first predicted summary, extract each word from the first predicted summary;

[0021] S22, For each first predicted summary, determine each target word based on each word and the preset part of speech. For each target word, the target word is the word among the words that satisfies the preset part of speech.

[0022] S23, For each first predicted summary, construct a candidate keyword graph. The candidate keyword graph contains multiple vertices, multiple edge points, and edges between each vertex and each edge point. Multiple vertices form a vertex set, and multiple edge points form an edge set.

[0023] For each vertex, the vertex represents a target word; for each edge point, the edge point represents a target word other than the target word represented by the vertex; and for each edge, the edge represents the grammatical relationship between the vertex and the edge point corresponding to that edge.

[0024] S24, for each first predicted summary, determine the score corresponding to each vertex based on the vertex set, edge set and each edge. For each score, the score represents the sum of the scores of the correlation between the target word corresponding to the vertex and each edge point in the edge set corresponding to the vertex.

[0025] S25. For each first predicted summary, sort the scores from smallest to largest, starting from the largest score, and select a preset number of scores as target scores, and use the target words corresponding to each target score as the first keywords.

[0026] The beneficial effect of adopting the above-mentioned further scheme is that by constructing a candidate keyword graph, the relationship between vertices and edge points can be clarified, thereby determining the degree of association between the target words corresponding to the vertices and each edge point in the edge set corresponding to the vertices, and selecting the first keyword from the target words according to the degree of association (the score corresponding to the vertex).

[0027] Furthermore, in the above method, for each first predicted summary, the score corresponding to each vertex is determined based on the vertex set, edge set, and each edge, including:

[0028] For each first predicted summary, based on the vertex set, edge set, and each edge, the score corresponding to each vertex is determined using a first formula, where the first formula is:

[0029]

[0030] Among them, WS(V i ) represents the score corresponding to the i-th vertex, d represents the damping factor, and Out(V) j) represents the vertex set, In(V i V represents the edge set. i V represents the i-th vertex. j V represents the j-th edge point. k w represents the edge between the i-th vertex and the j-th edge point. jk V represents k The corresponding preset weight, WS(V) j ) represents the score corresponding to the j-th edge point and the i-th vertex.

[0031] The beneficial effect of adopting the above-mentioned further scheme is that, by using the first formula, the score corresponding to each vertex is determined, thereby determining the degree of association between the target word corresponding to the vertex and each edge point in the edge set corresponding to the vertex.

[0032] Furthermore, the method also includes:

[0033] For each first predicted summary, based on each first keyword, determine the second keyword in each first keyword. For each second keyword, the second keyword is a word that cannot be found in the second Chinese text through the GPT-2 model.

[0034] For each first predicted summary, each second keyword is input into the pointer generation network. The pointer generation network determines the third keyword corresponding to each second keyword. For each third keyword, the third keyword is a word queried in the second Chinese text through the GPT-2 model.

[0035] For each first predicted summary, the first keywords are combined to form the third Chinese text, including:

[0036] For each first predicted summary, the third Chinese text is composed of each second keyword and each third keyword, excluding the third keyword.

[0037] The beneficial effect of adopting the above-mentioned further scheme is that, since the determined second keyword may not be found in the second Chinese text through the GPT-2 model, the second keyword is replaced with the third keyword through the pointer generation network so that the GPT-2 model can find the third keyword in the second Chinese text and thus generate the corresponding summary.

[0038] Furthermore, in the above method, for each first predicted summary, each second keyword is input into a pointer generation network. The pointer generation network then determines the third keyword corresponding to each second keyword, including:

[0039] S31, combine all the words in the second Chinese text with a dictionary to form an expanded dictionary;

[0040] S32, for each first predicted summary, based on each second keyword, determine the probability of each second keyword being found in the expanded dictionary using the second formula, where the second formula is:

[0041]

[0042] Among them, P vocab V', b' represents the probability of the second keyword in the expanded dictionary, and V', b' represents the preset first network parameters. s represents the vector corresponding to the first Chinese text. t This represents the hidden state of the decoder in the pointer generation network;

[0043] S33, for each first predicted summary, based on the probability of each second keyword in the expanded dictionary, the probability distribution of each second keyword in the second Chinese text is determined using the third formula, whereby:

[0044]

[0045] Where P(w) represents the probability distribution, t represents time, and w represents the set of all second keywords. i This represents the i-th second keyword. p represents the attention score obtained at time t corresponding to the i-th second keyword. gen This represents the value at time t that maps any word in the expanded dictionary to (0,1). b ptr This represents the preset second network parameters, where σ() is the sigmoid function, and x t This represents the t-th word in the extended dictionary;

[0046] S34. For each first predicted summary, based on each second keyword and the probability distribution corresponding to the second keyword, determine the position of the second keyword in the second Chinese text, and take the word at the position as the third keyword.

[0047] The beneficial effect of adopting the above-mentioned further scheme is that, since the third keyword is a word in the second Chinese text and its position is determined according to the probability distribution of the second keyword in the second Chinese text, the third keyword can be queried in the second Chinese text through the GPT-2 model.

[0048] Secondly, the present invention provides a Chinese abstract generation and early warning system, comprising:

[0049] The first acquisition module is used to acquire the first Chinese text.

[0050] The second acquisition module is used to input the first Chinese text into the GPT-2 model, and through the GPT-2 model, determine the first summary corresponding to the first Chinese text and the first category corresponding to the first summary. The first category represents the topic of the first summary.

[0051] The early warning module is used to determine the early warning information based on the first category. The early warning information indicates whether the second summary contains sensitive information.

[0052] In the second acquisition unit, the GPT-2 model is trained through the first unit, which is specifically used for:

[0053] S11, obtain multiple second Chinese texts and the second summaries corresponding to the second Chinese texts;

[0054] S12, the initial model is trained based on multiple second Chinese texts to obtain the first predicted summary corresponding to each second Chinese text and the second category corresponding to each first predicted summary. For each second category, the second category represents the topic of the first predicted summary.

[0055] S13, For each first predicted summary, determine each first keyword in the first predicted summary based on the first predicted summary. For each first keyword, the first keyword is a word that has a degree of relevance to each word in the first predicted summary.

[0056] S14, for each first predicted summary, combine the first keywords into a third Chinese text;

[0057] S15, input each third Chinese text into the initial model, train the initial model, determine the second predicted summary corresponding to the third Chinese text, and the third category corresponding to the second predicted summary. The third category represents the topic of the second predicted summary.

[0058] S16, Based on each second prediction summary and each second summary, determine the loss value of the initial model, whereby the loss value characterizes the difference between each second prediction summary and each second summary;

[0059] S17. If the loss value meets the preset termination condition, the initial model that meets the preset termination condition is used as the GPT-2 model. If the loss value does not meet the preset termination condition, the network parameters of the initial model are adjusted, and the initial model is retrained according to the adjusted network parameters until the first loss value of the initial model meets the preset termination condition.

[0060] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the Chinese summary generation and warning method described above.

[0061] Fourthly, the present invention also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the steps of the Chinese summary generation and warning method described above. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0063] Figure 1 This is a flowchart illustrating a method for generating and issuing early warnings for Chinese abstracts according to an embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram of the structure of a Chinese abstract generation and early warning system according to an embodiment of the present invention. Detailed Implementation

[0065] The following embodiments are further explanations and supplements to the present invention and do not constitute any limitation on the present invention.

[0066] The following describes, in conjunction with the accompanying drawings, a method, system, device, and medium for generating and issuing early warnings of Chinese abstracts according to embodiments of the present invention.

[0067] like Figure 1 As shown, an embodiment of the present invention provides a method for generating and issuing early warnings for Chinese abstracts, comprising the following steps:

[0068] Get the first Chinese text;

[0069] The first Chinese text is input into the GPT-2 model. The GPT-2 model determines the first summary corresponding to the first Chinese text and the first category corresponding to the first summary. The first category represents the topic of the first summary.

[0070] Based on the first category, the warning information is determined, and the warning information indicates whether the second summary contains sensitive information.

[0071] Optionally, the warning information is mainly determined by the classification of the first category to determine whether the second summary contains sensitive information. For example, if the first category is classified as a topic related to sensitive issues, then the second summary is determined to contain sensitive information.

[0072] Optionally, the GPT-2 model is trained in the following way:

[0073] S11, obtain multiple second Chinese texts and the second summaries corresponding to the second Chinese texts.

[0074] In this embodiment, the second summary is a reference summary corresponding to the second Chinese text, which is a high-quality summary compiled by humans.

[0075] S12, the initial model is trained based on multiple second Chinese texts to obtain the first predicted summary corresponding to each second Chinese text and the second category corresponding to each first predicted summary. For each second category, the second category represents the topic of the first predicted summary.

[0076] S13, For each first predicted summary, determine each first keyword in the first predicted summary based on the first predicted summary. For each first keyword, the first keyword is a word that has a degree of relevance to each word in the first predicted summary.

[0077] Optionally, for each first predicted summary, based on the first predicted summary, determine each first keyword in the first predicted summary, including:

[0078] S21, For each first predicted summary, extract each word from the first predicted summary;

[0079] S22, For each first predicted summary, determine each target word based on each word and the preset part of speech. For each target word, the target word is the word among the words that satisfies the preset part of speech.

[0080] S23, For each first predicted summary, construct a candidate keyword graph. The candidate keyword graph contains multiple vertices, multiple edge points, and edges between each vertex and each edge point. Multiple vertices form a vertex set, and multiple edge points form an edge set.

[0081] For each vertex, the vertex represents a target word; for each edge point, the edge point represents a target word other than the target word represented by the vertex; and for each edge, the edge represents the grammatical relationship between the vertex and the edge point corresponding to that edge.

[0082] S24, for each first predicted summary, determine the score corresponding to each vertex based on the vertex set, edge set and each edge. For each score, the score represents the sum of the scores of the correlation between the target word corresponding to the vertex and each edge point in the edge set corresponding to the vertex.

[0083] S25. For each first predicted summary, sort the scores from smallest to largest, starting from the largest score, and select a preset number of scores as target scores, and use the target words corresponding to each target score as the first keywords.

[0084] Optionally, not every word in the first predicted summary is representative. For example, some adjectives, even if deleted, will not affect the overall content of the summary. Only the object modified by the adjective needs to be used as the target word. Therefore, such adjectives are not representative words in the first predicted summary. Based on this, it is necessary to set the preset parts of speech in advance to select words that meet the preset parts of speech as target words.

[0085] Optionally, for the candidate keyword graph, the grammatical relationship between vertices and edge points can be clearly indicated. For example, if the word corresponding to the vertex is 'a' and the word corresponding to the edge point is 'b', then the edge between 'a' and 'b' corresponds to the grammatical relationship between 'a' and 'b'. This grammatical relationship can be analyzed using the dependency relationship analysis of the LIP platform in the existing technology, and semantic features can be incorporated to obtain the relationship between the part of speech of 'a' and the part of speech of 'b'.

[0086] Optionally, for each first predicted summary, a score is determined for each vertex based on the vertex set, edge set, and each edge. For example, if the vertex set contains words a, b, and c, and when word a is a vertex, the edge set contains words b and c, then the score for word a is the sum of the scores for the degree of association between word a and word b and the scores for the degree of association between word a and word c.

[0087] Optional, the preset quantity can be selected according to the actual situation.

[0088] Optionally, for each first prediction summary, based on the vertex set, edge set, and individual edges, determine the score corresponding to each vertex, including:

[0089] For each first predicted summary, based on the vertex set, edge set, and each edge, the score corresponding to each vertex is determined using a first formula, where the first formula is:

[0090]

[0091] Among them, WS(V i ) represents the score corresponding to the i-th vertex, d represents the damping factor, and Out(V) j ) represents the vertex set, In(V i V represents the edge set. i V represents the i-th vertex. j V represents the j-th edge point. k w represents the edge between the i-th vertex and the j-th edge point. jk V represents k The corresponding preset weight, WS(V) j ) represents the score corresponding to the j-th edge point and the i-th vertex.

[0092] Optionally, in practical applications, the summaries generated by the GPT-2 model may face the OOV (Out of Context) problem, where some words in the summary cannot be found in the original text, resulting in low-quality summaries. Therefore, the first keywords in the first predicted summary may also not be found in the second Chinese text. Based on this, a pointer generation network is introduced. Essentially, it locates the first keywords in the first predicted summary to their corresponding positions in the second Chinese text, and then replaces the first keywords with words at those positions. Since the replaced words are words that are already in the second Chinese text, the OOV problem is solved.

[0093] Optionally, the method further includes:

[0094] For each first predicted summary, based on each first keyword, determine the second keyword in each first keyword. For each second keyword, the second keyword is a word that cannot be found in the second Chinese text through the GPT-2 model.

[0095] For each first predicted summary, each second keyword is input into the pointer generation network. The pointer generation network determines the third keyword corresponding to each second keyword. For each third keyword, the third keyword is a word queried in the second Chinese text through the GPT-2 model.

[0096] Optionally, for each first predicted summary, each second keyword is input into a pointer generation network. The pointer generation network then determines the third keyword corresponding to each second keyword, including:

[0097] S31, combine all the words in the second Chinese text with a dictionary to form an expanded dictionary;

[0098] S32, for each first predicted summary, based on each second keyword, determine the probability of each second keyword being found in the expanded dictionary using the second formula, where the second formula is:

[0099]

[0100] Among them, P vocab V', b' represents the probability of the second keyword in the expanded dictionary, and V', b' represents the preset first network parameters. s represents the vector corresponding to the first Chinese text. t This represents the hidden state of the decoder in the pointer generation network;

[0101] S33, for each first predicted summary, based on the probability of each second keyword in the expanded dictionary, the probability distribution of each second keyword in the second Chinese text is determined using the third formula, whereby:

[0102]

[0103] Where P(w) represents the probability distribution, t represents time, and w represents the set of all second keywords. i This represents the i-th second keyword. p represents the attention score obtained at time t corresponding to the i-th second keyword. gen This represents the value at time t that maps any word in the expanded dictionary to (0,1). b ptr This represents the preset second network parameters, where σ() is the sigmoid function, and x t This represents the t-th word in the extended dictionary;

[0104] S34. For each first predicted summary, based on each second keyword and the probability distribution corresponding to the second keyword, determine the position of the second keyword in the second Chinese text, and take the word at the position as the third keyword.

[0105] S14, for each first predicted summary, combine the first keywords into a third Chinese text.

[0106] Optionally, for each first predicted summary, the first keywords are combined to form a third Chinese text, including:

[0107] For each first predicted summary, the third Chinese text is composed of each second keyword and each third keyword, excluding the third keyword.

[0108] S15, input each third Chinese text into the initial model, train the initial model, determine the second predicted summary corresponding to the third Chinese text, and the third category corresponding to the second predicted summary. The third category represents the topic of the second predicted summary.

[0109] In this embodiment, the first predicted summary is converted into a third Chinese text in a semi-human, semi-artificial intelligence manner. Furthermore, the text volume of the first predicted summary is much smaller than that of the second Chinese text. Therefore, the workload of manually editing high-quality summaries for the training set of the initial model is greatly reduced.

[0110] S16, Based on each second prediction summary and each second summary, determine the loss value of the initial model, whereby the loss value characterizes the difference between each second prediction summary and each second summary.

[0111] In this embodiment, since the second predicted summary comes from the third Chinese text, and the third Chinese text comes from the first predicted summary generated by the original model, the second predicted summary contains the original shortcomings of the first predicted summary (such as inaccurate content, incorrect word selection, etc.). At this time, constructing the loss value of the initial model with the low-quality second predicted summary and the high-quality second summary enables the initial model to learn to distinguish between the low-quality summary (second predicted summary) and the high-quality summary (second summary), thereby improving the quality of the generated summary.

[0112] S17. If the loss value meets the preset termination condition, the initial model that meets the preset termination condition is used as the GPT-2 model. If the loss value does not meet the preset termination condition, the network parameters of the initial model are adjusted, and the initial model is retrained according to the adjusted network parameters until the first loss value of the initial model meets the preset termination condition.

[0113] Optionally, the first Chinese text is input into the GPT-2 model. After the GPT-2 model determines the first abstract corresponding to the first Chinese text and the first category corresponding to the first abstract, relevant information can be output on the Postman software. The relevant information includes the first keyword, the third keyword, the first abstract, the first category, and warning information.

[0114] like Figure 2 As shown, an embodiment of the present invention provides a Chinese abstract generation and early warning system, comprising:

[0115] The first acquisition module 202 is used to acquire the first Chinese text;

[0116] The second acquisition module 203 is used to input the first Chinese text into the GPT-2 model, and through the GPT-2 model, determine the first summary corresponding to the first Chinese text and the first category corresponding to the first summary. The first category represents the topic of the first summary.

[0117] The early warning module 204 is used to determine early warning information based on the first category, wherein the early warning information indicates whether the second summary contains sensitive information;

[0118] In the second acquisition unit 203, the GPT-2 model is obtained through training the first unit. The first unit is specifically used for:

[0119] Obtain multiple second Chinese texts, and the corresponding second summaries for each second Chinese text;

[0120] The initial model is trained based on multiple second Chinese texts to obtain the first predicted summary corresponding to each second Chinese text and the second category corresponding to each first predicted summary. For each second category, the second category represents the topic of the first predicted summary.

[0121] For each first predicted summary, based on the first predicted summary, determine each first keyword in the first predicted summary. For each first keyword, the first keyword is a word that has a degree of relevance to each word in the first predicted summary.

[0122] For each first predicted summary, the first keywords are combined to form the third Chinese text;

[0123] Each third Chinese text is input into the initial model, the initial model is trained, and the second predicted summary corresponding to the third Chinese text is determined, as well as the third category corresponding to the second predicted summary. The third category represents the topic of the second predicted summary.

[0124] Based on each second prediction summary and each second summary, the loss value of the initial model is determined, and the loss value characterizes the difference between each second prediction summary and each second summary;

[0125] If the loss value meets the preset termination condition, the initial model that meets the preset termination condition is used as the GPT-2 model. If the loss value does not meet the preset termination condition, the network parameters of the initial model are adjusted, and the initial model is retrained based on the adjusted network parameters until the first loss value of the initial model meets the preset termination condition.

[0126] Optionally, in the second acquisition unit 203, for each first predicted summary, based on the first predicted summary, through the second unit, each first keyword in the first predicted summary is determined, wherein the second unit is specifically used for:

[0127] For each first predicted summary, extract each word from the first predicted summary;

[0128] For each first predicted summary, each target word is determined based on each word and the preset part of speech. For each target word, the target word is the word that satisfies the preset part of speech among the words.

[0129] For each first predicted summary, a candidate keyword graph is constructed. The candidate keyword graph contains multiple vertices, multiple edge points, and edges between each vertex and each edge point. Multiple vertices form a vertex set, and multiple edge points form an edge set.

[0130] For each vertex, the vertex represents a target word; for each edge point, the edge point represents a target word other than the target word represented by the vertex; and for each edge, the edge represents the grammatical relationship between the vertex and the edge point corresponding to that edge.

[0131] For each first predicted summary, a score is determined for each vertex based on the vertex set, edge set, and each edge. For each score, the score represents the sum of the scores of the correlation between the target word corresponding to the vertex and each edge point in the edge set corresponding to the vertex.

[0132] For each first predicted summary, sort the scores from smallest to largest, starting with the largest score, and select a preset number of scores as target scores, and use the target words corresponding to each target score as the first keywords.

[0133] Optionally, in the second acquisition unit 203, for each first predicted summary, based on the vertex set, edge set, and each edge, a score corresponding to each vertex is determined through the third unit, wherein the third unit is specifically used for:

[0134] For each first predicted summary, based on the vertex set, edge set, and each edge, the score corresponding to each vertex is determined using a first formula, where the first formula is:

[0135]

[0136] Among them, WS(V i ) represents the score corresponding to the i-th vertex, d represents the damping factor, and Out(V) j ) represents the vertex set, In(V i V represents the edge set. i V represents the i-th vertex. j V represents the j-th edge point. k w represents the edge between the i-th vertex and the j-th edge point. jk V represents k The corresponding preset weight, WS(V) j ) represents the score corresponding to the j-th edge point and the i-th vertex.

[0137] Optionally, the system may also include:

[0138] The third acquisition module is used to determine the second keyword in each first predicted summary based on each first keyword. For each second keyword, the second keyword is a word that cannot be found in the second Chinese text through the GPT-2 model.

[0139] The fourth acquisition module is used to input each second keyword into the pointer generation network for each first predicted summary, and to determine the third keyword corresponding to each second keyword through the pointer generation network. For each third keyword, the third keyword is a word queried in the second Chinese text through the GPT-2 model.

[0140] Then, in the second acquisition module 203, for each first predicted summary, the third Chinese text is composed of each first keyword, and the module also includes:

[0141] For each first predicted summary, the third Chinese text is composed of each second keyword and each third keyword, excluding the third keyword.

[0142] Optionally, in the fourth acquisition module, for each first predicted summary, each second keyword is input into the pointer generation network. The pointer generation network, through the fourth unit, determines the third keyword corresponding to each second keyword. Specifically, the fourth unit is used for:

[0143] Combine all the words in the second Chinese text with a dictionary to form an expanded dictionary;

[0144] For each first predicted summary, based on each second keyword, the probability of each second keyword being found in the expanded dictionary is determined using a second formula, where the second formula is:

[0145]

[0146] Among them, P vocab V represents the probability of the second keyword in the expanded dictionary. ' ,V,b,b ' This represents the preset first network parameter. s represents the vector corresponding to the first Chinese text. t This represents the hidden state of the decoder in the pointer generation network;

[0147] For each first predicted summary, based on the probability of each second keyword in the expanded dictionary, the probability distribution of each second keyword in the second Chinese text is determined using a third formula, whereby:

[0148]

[0149] Where P(w) represents the probability distribution, and t represents time. Let w represent the attention score at time t (i-th time), and w represent the set of all second keywords. i p represents the i-th second keyword. gen This indicates a trigger switch, meaning that p is triggered once every time time t. gen Then, a word is extracted from the extended dictionary, and this process continues until all words in the extended dictionary have been extracted. b ptr This represents the preset second network parameters, where σ() is the sigmoid function, and x t This represents the t-th word in the extended dictionary;

[0150] For each first predicted summary, based on each second keyword and the probability distribution corresponding to the second keyword, the position of the second keyword in the second Chinese text is determined, and the word at that position is taken as the third keyword.

[0151] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements some or all of the steps of the above-described method for generating and issuing warnings of Chinese summaries.

[0152] The electronic device can be a computer, and its program is computer software. The parameters and steps of the electronic device of the present invention can be referred to the parameters and steps in the embodiment of the method for generating and warning Chinese abstracts above, and will not be repeated here.

[0153] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this disclosure can be embodied in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the invention can also be implemented as a computer program product contained in one or more computer-readable media, which contains computer-readable program code. Computer-readable storage media can be, for example, but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof.

[0154] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0155] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for generating and issuing early warnings for Chinese abstracts, characterized in that, Includes the following steps: Get the first Chinese text; The first Chinese text is input into the GPT-2 model. The GPT-2 model is used to determine the first summary corresponding to the first Chinese text and the first category corresponding to the first summary. The first category represents the topic of the first summary. Based on the first category, a warning message is determined, wherein the warning message indicates whether the second summary contains sensitive information; The GPT-2 model was trained in the following way: S11, obtain multiple second Chinese texts and the second summary corresponding to the second Chinese texts; S12, the initial model is trained based on multiple second Chinese texts to obtain a first predicted summary corresponding to each second Chinese text and a second category corresponding to each first predicted summary. For each second category, the second category represents the topic of the first predicted summary. S13, for each of the first predicted summaries, based on the first predicted summaries, determine each of the first keywords in the first predicted summaries. For each of the first keywords, the first keyword is a word that has a degree of relevance to each word in the first predicted summaries. S14, for each of the first predicted summaries, the first keywords are combined to form a third Chinese text; S15, input each of the third Chinese texts into the initial model, train the initial model, determine the second predicted summary corresponding to the third Chinese text, and the third category corresponding to the second predicted summary, wherein the third category represents the topic of the second predicted summary; S16, Based on each of the second prediction summaries and each of the second summaries, determine the loss value of the initial model, the loss value representing the difference between each of the second prediction summaries and each of the second summaries; S17. If the loss value meets the preset termination condition, the initial model that meets the preset termination condition is used as the GPT-2 model. If the loss value does not meet the preset termination condition, the network parameters of the initial model are adjusted, and the initial model is retrained according to the adjusted network parameters until the first loss value of the initial model meets the preset termination condition.

2. The method according to claim 1, characterized in that, For each of the first predicted summaries, determining each first keyword in the first predicted summary includes: S21, For each of the first prediction summaries, extract each word from the first prediction summary; S22, for each of the first predicted summaries, each target word is determined based on each of the words and the preset part of speech. For each target word, the target word is the word among the words that satisfies the preset part of speech. S23, for each of the first predicted summaries, construct a candidate keyword graph, which contains multiple vertices, multiple edge points, and edges between each vertex and each edge point. The multiple vertices form a vertex set, and the multiple edge points form an edge set. Wherein, for each vertex, the vertex represents a target word; for each edge point, the edge point represents a target word other than the target word represented by the vertex; and for each edge, the edge represents the grammatical relationship between the vertex and the edge point corresponding to the edge. S24, for each first predicted summary, a score corresponding to each vertex is determined based on the vertex set, the edge set, and each edge. For each score, the score represents the sum of the scores of the correlation between the target word corresponding to the vertex and each edge point in the edge set corresponding to the vertex. S25, for each of the first predicted summaries, sort the scores from smallest to largest, starting from the largest score, and sequentially select a preset number of scores as target scores, and use the target words corresponding to each target score as the first keywords.

3. The method according to claim 2, characterized in that, For each of the first predicted summaries, determining the score corresponding to each vertex based on the vertex set, the edge set, and each edge includes: For each of the first predicted summaries, based on the vertex set, the edge set, and each edge, a score corresponding to each vertex is determined using a first formula, wherein the first formula is: Among them, WS(V i ) represents the score corresponding to the i-th vertex, d represents the damping factor, and Out(V) j ) represents the vertex set, In(V i V represents the edge set. i V represents the i-th vertex. j V represents the j-th edge point. k w represents the edge between the i-th vertex and the j-th edge point. jk V represents k The corresponding preset weight, WS(V) j ) represents the score corresponding to the j-th edge point and the i-th vertex.

4. The method according to claim 3, characterized in that, The method further includes: For each first predicted summary, a second keyword is determined based on each first keyword. For each second keyword, the second keyword is a word that cannot be found in the second Chinese text by the GPT-2 model. For each first predicted summary, each second keyword is input into a pointer generation network. Through the pointer generation network, a third keyword corresponding to each second keyword is determined. For each third keyword, the third keyword is a word queried in the second Chinese text through the GPT-2 model. For each of the first predicted summaries, the third Chinese text is composed of the first keywords, including: For each of the first predicted summaries, the second keywords (excluding the third keyword) and the third keywords are combined to form a third Chinese text.

5. The method according to claim 4, characterized in that, For each first predicted summary, each second keyword is input into a pointer generation network, and the third keyword corresponding to each second keyword is determined through the pointer generation network, including: S31, combine all the words in the second Chinese text with a dictionary to form an expanded dictionary; S32, for each of the first predicted summaries, based on each of the second keywords, the probability of each of the second keywords being found in the extended dictionary is determined using a second formula, wherein the second formula is: Among them, P vocab V', b' represents the probability of the second keyword in the expanded dictionary, and V', b' represents the preset first network parameters. s represents the vector corresponding to the first Chinese text. t This represents the hidden state of the decoder in the pointer generation network; S33, for each of the first predicted summaries, based on the probabilities of each second keyword in the extended dictionary, the probability distribution of each second keyword in the second Chinese text is determined using a third formula, wherein the third formula is: Where P(w) represents the probability distribution, t represents time, and w represents the set of all second keywords. i This represents the i-th second keyword. p represents the attention score corresponding to the i-th second keyword at time t. gen This represents the value at time t that maps any word in the expanded dictionary to (0,1). b ptr This represents the preset second network parameters, where σ() is the sigmoid function, and x t This represents the t-th word in the extended dictionary; S34, for each of the first predicted summaries, based on each of the second keywords and the probability distribution corresponding to the second keywords, determine the position of the second keyword in the second Chinese text, and use the word at the position as the third keyword.

6. A Chinese abstract generation and early warning system, characterized in that, include: The first acquisition module is used to acquire the first Chinese text. The second acquisition module is used to input the first Chinese text into the GPT-2 model, and through the GPT-2 model, determine the first summary corresponding to the first Chinese text and the first category corresponding to the first summary, wherein the first category represents the topic of the first summary; The early warning module is used to determine early warning information based on the first category, wherein the early warning information indicates whether the second summary contains sensitive information; In the second acquisition unit, the GPT-2 model is trained through the first unit, which is specifically used for: Obtain multiple second Chinese texts, and the second summaries corresponding to the second Chinese texts; Based on multiple second Chinese texts, the initial model is trained to obtain a first predicted summary corresponding to each second Chinese text and a second category corresponding to each first predicted summary. For each second category, the second category represents the topic of the first predicted summary. For each of the first predicted summaries, each of the first keywords in the first predicted summaries is determined based on the first predicted summaries. For each of the first keywords, the first keyword is a word that has a degree of relevance to each word in the first predicted summaries. For each of the first predicted summaries, the first keywords are combined to form the third Chinese text; Each of the third Chinese texts is input into the initial model, the initial model is trained, and the second predicted summary corresponding to the third Chinese text is determined, as well as the third category corresponding to the second predicted summary, wherein the third category represents the topic of the second predicted summary; Based on each of the second prediction summaries and each of the second summaries, a loss value for the initial model is determined, the loss value representing the difference between each of the second prediction summaries and each of the second summaries; If the loss value meets the preset termination condition, the initial model that meets the preset termination condition is used as the GPT-2 model. If the loss value does not meet the preset termination condition, the network parameters of the initial model are adjusted, and the initial model is retrained based on the adjusted network parameters until the first loss value of the initial model meets the preset termination condition.

7. An electronic device comprising a memory, a processor, and a program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for generating and issuing early warnings of Chinese summaries as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the steps of a Chinese summary generation and early warning method as described in any one of claims 1 to 5.

Images