News text generation method based on text style
By generating fake news using a seq2seq model and the pre-trained language model XLNet, and combining the Attention mechanism and BiLSTM detection, the problem of insufficient extraction of text style features and latent intent in fake news detection is solved, achieving more reliable fake news detection and improved information transparency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEBEI UNIV OF ENG
- Filing Date
- 2024-01-31
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies struggle to effectively extract text style features and underlying intentions, resulting in insufficient performance in fake news detection.
A fake news detection system is constructed by generating fake news using a seq2seq model, performing feature analysis and detection using a pre-trained language model XLNet, assigning feature weights using an attention mechanism, and obtaining deep semantic information using BiLSTM.
It improves the training efficiency and model generalization ability of fake news detection, enhances the reliability of fake news detection systems, helps identify and understand the spread of false information, and improves information transparency and public trust.
Smart Images
Figure CN122021561A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of fake news text research technology, and in particular to a method for generating news text based on text style. Background Technology
[0002] With the diversification of social media and the development of internet technology, smartphones and social media platforms have grown exponentially. At the same time, misinformation permeates the entire network, causing adverse effects on individuals and society, making the fight against the spread of misinformation urgent. While some machine learning methods have shown good performance in discrimination, they are clearly insufficient in extracting text style features and underlying intentions.
[0003] A search revealed a Chinese patent application (CN202311235587.X) that discloses a method for simultaneously detecting malicious comments and fake news. This method involves selecting potentially effective sets of interpretable parameters and corresponding classification models from individual malicious comment or fake news detection datasets, combining them pairwise, and then training and validating the method on a union C consisting of existing malicious comment dataset A and fake news dataset B. The results are then filtered using an evaluation metric: accuracy + precision + recall + F1 score, resulting in an optimized combination of interpretable parameters and classification models. However, the detection method in this patent has the following shortcomings: it cannot effectively extract text style features and underlying intent, and therefore requires further improvement. Summary of the Invention
[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for generating news text based on text style.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] A method for generating news text based on text style includes:
[0007] S1: Using a seq2seq model, sentences highlighting real news are replaced with credible but false information through language loading techniques;
[0008] S2: Subsequently, a pre-trained language model is used to perform feature analysis and detection on the generated text;
[0009] In S1, the seq2seq model consists of two parts: an encoder and a decoder. Both the encoder and decoder are composed of RNN neural networks. The encoder's main function is to encode a variable-length input into a fixed-size feature vector c, and then pass the feature vector c to the decoder. The decoder's main function is to use the feature vector c to sequentially output a variable-length sequence. For the text generation task, given a text sequence X = [x1, x2, ..., Xm], generate another text sequence Y = [y1, y2, ..., ym]. The training objective of the text generation model based on the seq2seq framework is to maximize the probability of generating the text sequence Y given the input text sequence X, i.e., to maximize p(x1, x2, ..., Xm | y1, y2, ..., ym).
[0010] Preferably: the encoder uses an RNN neural network f enc The input text sequence X is encoded using (·), and the hidden layer output hm of the RNN neural network at time m is used as the feature vector c of the input sequence, specifically using the following formula:
[0011]
[0012] c = h m
[0013] Among them, f enc (·) represents an RNN neural network with parameters θ dec e x The word vector input at time t.
[0014] Preferably, the decoder uses another RNN neural network f dec The target text sequence Y is generated sequentially using a feature vector c and a feedforward neural network g(·); the decoder first initializes the neural network f with the feature vector c. dec (·), and then use the output value yt-1 at time t-1 to generate the output yt at time t; let st represent the network f dec The hidden layer output of (·), where ot(0,1)|v| is the posterior probability of all words in the vocabulary:
[0015] s0 = c
[0016] s t =f dec (s t-1 ,e yt-1 ,θ dec )
[0017] y t =arg maxO t
[0018] Among them, f dec (·) represents a decoding recurrent neural network, and g(·) is a feedforward neural network with the softmax activation function; θ dec And θ0 are f dec In (·) and g(·), ey-1 is the word vector of t-1; y0 and yn represent special symbols for the start. <sos>and the special symbol representing the end of i <eos>.
[0019] Preferably, in S2, the method for performing feature analysis and detection on the generated text is as follows: First, the pre-trained language model XLNet is used to obtain word vectors with propaganda techniques and style. Then, deep semantic information is obtained through the BiLSTM bidirectional gating unit. Finally, the Attention mechanism is used to assign different feature weights according to the importance of the features, and the authenticity of the text is detected.
[0020] Preferably, in step S2, the generated text is analyzed and detected using a fake news detection model. The fake news detection model includes a fully connected layer, an Attention layer, a BiLSTM layer, an XLNet layer, and an Input layer.
[0021] Preferably, the XLNet layer employs a two-stream self-attention mechanism to incorporate the positional information gθ into the objective function of the AR model, as detailed below:
[0022]
[0023] Preferably, the objective function of the AR model is updated using a "dual-stream" formula, where "dual-stream" refers to Querystream and Contentstream; Querystream displays the position information of the current word but not its content information, while Contentstream displays both the content information and the position information of the current word; the "dual-stream" update formula is as follows:
[0024]
[0025]
[0026] Where g is the query hidden state; h is the content hidden state; m is the XLNet layer number; Q is the query vector; K is the key vector; and V is the content vector. Q, K, and V are obtained as their corresponding matrices through Linear.
[0027] Preferred: Based on the fake news detection model, a fake news detection system is designed and implemented. The fake news detection system is developed locally under the Win10 operating system, using Python as the system development language; PyTorch 11.12.1 as the neural network framework; web crawlers are used to capture Weibo information; and EChart is used to visualize the classification results.
[0028] Preferably, the fake news detection system includes:
[0029] Data crawling module:
[0030] News data and news comment data from social media platforms are obtained through a web crawling framework, and then stored in a database. The news data includes: the news title, the publication time, the content, and the corresponding comments and their timestamps. Popular news items are also retrieved and displayed.
[0031] Fake News Detection Module:
[0032] Fake news detection is the main functional module of the system. It uses a pre-trained XLNet to obtain text features containing propaganda statements, then uses a bidirectional long short-term memory network to obtain higher-level text features, and finally uses a self-attention mechanism to assign different weights, and outputs the detection results.
[0033] Analysis results visualization module.
[0034] Preferably, the fake news detection system further includes:
[0035] User Management Submodule:
[0036] Users in the user management module are divided into administrator accounts and regular accounts; regular accounts can perform basic operations; administrator accounts can also perform the above operations; by setting the administrator account as an internal account, the administrator account can operate on all accounts; after a user registers, the user's relevant information is stored in a database table.
[0037] Detection record module:
[0038] This module mainly stores news detection records. The detection record information includes the news title, detection date, the time when the news was published, and the detection result.
[0039] The beneficial effects of this invention are as follows:
[0040] 1. This invention is based on the distinct writing styles and underlying intentions of real news. It generates new training corpora using seq2seq technology. These training examples replace sentences with credible but false information that highlight the text style and underlying intentions of real news. Based on this, a text style-based fake news detection method is proposed. This method uses a pre-trained language model to obtain the style features of fake news, thereby improving training efficiency and the generalization ability of the model. As a result, the reliability of the fake news detection system is guaranteed.
[0041] 2. The fake news detection technology of the present invention can help identify and understand the spread trends of false information, providing more reliable data support for decision-making; the application of fake news detection technology can improve the transparency and credibility of information, and enhance the public's trust in news media and information sources; this helps to establish a more open, fair and reliable information environment. Attached Figure Description
[0042] Figure 1 This is a roadmap for a fake news detection technology based on a text style-based news text generation method proposed in this invention;
[0043] Figure 2 Here is a seq2seq framework diagram of a news text generation method based on text style proposed in this invention;
[0044] Figure 3 This is a framework diagram of a fake news detection model based on a text style-based news text generation method proposed in this invention.
[0045] Figure 4 This is a framework diagram of a fake news detection system based on a text style-based news text generation method proposed in this invention.
[0046] Figure 5 This is a schematic diagram of Bart encoding and decoding in a text style-based news text generation method proposed in this invention. Detailed Implementation
[0047] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.
[0048] Example 1:
[0049] A text style-based news text generation method, primarily used for fake news analysis and detection, includes:
[0050] S1: Using a seq2seq model, sentences highlighting real news are replaced with credible but false information through language loading techniques;
[0051] S2: Subsequently, a pre-trained language model is used to perform feature analysis and detection on the generated text;
[0052] In S1, the seq2seq model consists of two parts: an encoder and a decoder. The encoder and decoder are generally composed of RNN neural networks. The encoder's main function is to encode a variable-length input into a fixed-size feature vector c, and then pass the feature vector c to the decoder. The decoder's main function is to use the feature vector c to sequentially output a variable-length sequence. For a text generation task, given a text sequence X = [x1, x2, ..., Xm], generate another text sequence Y = [y1, y2, ..., ym]. The training objective of the text generation model based on the seq2seq framework is to maximize the probability of generating the text sequence Y given the input text sequence X, i.e., to maximize p(x1, x2, ..., Xm | y1, y2, ..., ym).
[0053] The encoder uses an RNN neural network f. enc The input text sequence X is encoded using (·), and the hidden layer output hm of the RNN neural network at time m is used as the feature vector c of the input sequence, specifically using the following formula:
[0054]
[0055] c = h m
[0056] Among them, f enc (·) represents an RNN neural network with parameters θ dec e x The word vector input at time t.
[0057] The decoder uses another RNN neural network f. dec The target text sequence Y is generated sequentially using a feature vector c and a feedforward neural network g(·); the decoder first initializes the neural network f with the feature vector c. dec (·), and then use the output value yt-1 at time t-1 to generate the output yt at time t; let st represent the network f dec The hidden layer output of (·), where ot(0,1)|v| is the posterior probability of all words in the vocabulary:
[0058] s0 = c
[0059] s t =f dec (s t-1 ,e yt-1 ,θ dec )
[0060] y t =arg maxO t
[0061] Among them, f dec (·) represents a decoding recurrent neural network, and g(·) is a feedforward neural network with the softmax activation function; θ dec And θ0 are f dec In (·) and g(·), ey-1 is the word vector of t-1; y0 and yn are generally special symbols representing the start. <sos>and the special symbol representing the end of i <eos>.
[0062] In S2, the method for performing feature analysis and detection on the generated text is as follows: First, the pre-trained language model XLNet is used to obtain word vectors with propaganda techniques and style. Then, deep semantic information is obtained through BiLSTM bidirectional gating units. Finally, the Attention mechanism is used to assign different feature weights according to the importance of the features, and the authenticity of the text is detected.
[0063] In step S2, the generated text is analyzed and detected using a fake news detection model. The fake news detection model includes a fully connected layer, an Attention layer, a BiLSTM layer, an XLNet layer, and an Input layer.
[0064] The XLNet layer employs a two-stream self-attention mechanism to incorporate the location information gθ into the objective function of the AR model, as detailed below:
[0065]
[0066] The objective function of the AR model is updated using a "dual-stream" formula, where "dual-stream" refers to Querystream and Contentstream. Querystream displays the position information of the current word but not its content information, while Contentstream displays both the content information and the position information of the current word. The update formula for the "dual-stream" model is as follows:
[0067]
[0068]
[0069] Where g is the query hidden state; h is the content hidden state; m is the XLNet layer number; Q is the query vector; K is the key vector; and V is the content vector. Q, K, and V are obtained as their corresponding matrices through Linear.
[0070] Because BART has an autoregressive decoder, it can be directly fine-tuned for sequence generation tasks such as abstract question answering and summarizing. In both of these tasks, information is copied from the input but manipulated, which is closely related to the goal of denoising pre-training. Here, the encoder input is the input sequence, and the decoder autoregressively generates the output. BART incorporates features of BERT's bidirectional encoder and GPT's left-to-right decoder, building upon the standard seq2seq Transformer model, making it more suitable for text generation scenarios than BERT; compared to GPT, it also provides more bidirectional contextual information. While achieving progress in generation tasks, it can also achieve state-of-the-art (SOTA) results in some text understanding tasks.
[0071] The innovation of this invention lies in incorporating propaganda techniques to mimic how humans create false information and generate fake news; in particular, it uses two commonly used propaganda methods: appeals to authority and emotional tone in the generation of fake news.
[0072] In practical applications:
[0073] Given a real news article, it first adds interjections, then identifies a prominent sentence, and then replaces it with a seemingly reasonable but uninformative sentence that fits the context; finally, it generates a propaganda sentence that makes the article look like fake news written by a human.
[0074] Given a real news article, use an RNN in a seq2seq model to replace a prominent sentence with a seemingly reasonable but false message. A Bart pre-trained language model with a seq2seq structure generates corresponding filler words and false information with an authoritative release.
[0075] Example 2:
[0076] A method for generating news text based on text style, wherein a fake news detection system is designed and implemented according to the fake news detection model. The fake news detection system is developed locally under the Win10 operating system, using Python as the system development language; PyTorch 11.12.1 as the neural network framework; web crawlers are used to crawl Weibo information; and EChart is used to visualize the classification results.
[0077] The fake news detection system includes:
[0078] Data crawling module:
[0079] We use a web crawler framework to acquire news data and news comment data from social media platforms such as Weibo, and then store the data in a database. The news data includes: the news title, the publication time, the content, and the corresponding comments, including the comment timestamps. We also acquire and display trending news.
[0080] Fake News Detection Module:
[0081] Fake news detection is the main functional module of the system. It uses a pre-trained XLNet to obtain text features containing propaganda statements, then uses a bidirectional long short-term memory network to obtain higher-level text features, and finally uses a self-attention mechanism to assign different weights, and outputs the detection results.
[0082] Analysis results visualization module:
[0083] Visualization is an indispensable part of data analysis. Its main purpose is to make users' understanding of data more straightforward and efficient. For this purpose, the system presents the analysis results to users from multiple dimensions.
[0084] The fake news detection system also includes:
[0085] User Management Submodule:
[0086] Users in the user management module are divided into administrator accounts and regular accounts. Regular accounts can perform basic operations such as logging in and registering. Administrator accounts can also perform the above operations. By setting the administrator account as an internal account, the administrator account can operate on all accounts. After a user registers, the user's relevant information is stored in a database table.
[0087] Detection record module:
[0088] This module mainly stores news detection records. The detection record information includes the news title, detection date, the time when the news was published, and the detection result.
[0089] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.< / eos> < / sos> < / eos> < / sos>
Claims
1. A method for generating news text based on text style, characterized in that, include: S1: Using a seq2seq model, sentences highlighting real news are replaced with credible but false information through language loading techniques; S2: Subsequently, a pre-trained language model is used to perform feature analysis and detection on the generated text; In S1, the seq2seq model consists of two parts: an encoder and a decoder. Both the encoder and decoder are composed of RNN neural networks. The encoder's main function is to encode a variable-length input into a fixed-size feature vector c, and then pass the feature vector c to the decoder. The decoder's main function is to use the feature vector c to sequentially output a variable-length sequence. For the text generation task, given a text sequence X = [x1, x2, ..., Xm], generate another text sequence Y = [y1, y2, ..., ym]. The training objective of the text generation model based on the seq2seq framework is to maximize the probability of generating the text sequence Y given the input text sequence X, i.e., to maximize p(x1, x2, ..., Xm | y1, y2, ..., ym).
2. The method for generating news text based on text style according to claim 1, characterized in that, The encoder uses an RNN neural network f enc The input text sequence X is encoded using (·), and the hidden layer output hm of the RNN neural network at time m is used as the feature vector c of the input sequence, specifically using the following formula: c=h m Among them, f enc (·) represents an RNN neural network with parameters θ dec e x The word vector input at time t.
3. The method for generating news text based on text style according to claim 2, characterized in that, The decoder uses another RNN neural network f dec The target text sequence Y is generated sequentially using a feature vector c and a feedforward neural network g(·); the decoder first initializes the neural network f with the feature vector c. dec (·), and then use the output value yt-1 at time t-1 to generate the output yt at time t; let st represent the network f dec The hidden layer output of (·), where ot(0,1)|v| is the posterior probability of all words in the vocabulary: s0 = c s t =f dec (s t-1 ,e yt-1 ,i dec ) and t =arg maxO t Among them, f dec (·) represents a decoding recurrent neural network, and g(·) is a feedforward neural network with the softmax activation function; θ dec And θ0 are f dec In (·) and g(·), ey-1 is the word vector of t-1; y0 and yn represent special symbols for the start. <sos>and the special symbol representing the end of i <eos> 。< / eos> < / sos> 4. The method for generating news text based on text style according to claim 3, characterized in that, In S2, the method for performing feature analysis and detection on the generated text is as follows: First, the pre-trained language model XLNet is used to obtain word vectors with propaganda techniques and style. Then, deep semantic information is obtained through the BiLSTM bidirectional gating unit. Finally, the Attention mechanism is used to assign different feature weights according to the importance of the features, and the authenticity of the text is detected.
5. The method for generating news text based on text style according to claim 4, characterized in that, In S2, the generated text is analyzed and detected by a fake news detection model. The fake news detection model includes a fully connected layer, an Attention layer, a BiLSTM layer, an XLNet layer, and an Input layer.
6. The method for generating news text based on text style according to claim 5, characterized in that, The XLNet layer employs a two-stream self-attention mechanism to incorporate the location information gθ into the objective function of the AR model, as detailed below:
7. The method for generating news text based on text style according to claim 6, characterized in that, The objective function of the AR model is updated using a "dual-stream" formula, where "dual-stream" refers to Querystream and Contentstream. Querystream displays the position information of the current word but not its content information, while Contentstream displays both the content information and the position information of the current word. The update formula for the "dual-stream" model is as follows: Where g is the query hidden state; h is the content hidden state; m is the XLNet layer number; Q is the query vector; K is the key vector; and V is the content vector. Q, K, and V are obtained as their corresponding matrices through Linear.
8. A method for generating news text based on text style according to claim 7, characterized in that, Based on the fake news detection model, a fake news detection system was designed and implemented. The fake news detection system was developed locally under the Win10 operating system, using Python as the system development language and PyTorch 11.12.1 as the neural network framework.
9. A method for generating news text based on text style according to claim 8, characterized in that, The fake news detection system includes: Data crawling module: News data and news comment data from social media platforms are obtained through a web crawling framework, and then stored in a database. The news data includes: the news title, the publication time, the content, and the corresponding comments and their timestamps. Popular news items are also retrieved and displayed. Fake News Detection Module: Fake news detection is the main functional module of the system. It uses a pre-trained XLNet to obtain text features containing propaganda statements, then uses a bidirectional long short-term memory network to obtain higher-level text features, and finally uses a self-attention mechanism to assign different weights, and outputs the detection results. Analysis results visualization module.
10. A method for generating news text based on text style according to claim 8, characterized in that, The fake news detection system also includes: User Management Submodule: Users in the user management module are divided into administrator accounts and regular accounts; regular accounts can perform basic operations, and administrator accounts can perform the same operations. By setting the administrator account as an internal account, the administrator account can operate on all accounts; after a user registers, the user's relevant information is stored in a database table; Detection record module: This module mainly stores news detection records. The detection record information includes the news title, detection date, the time when the news was published, and the detection result.