False news detection method based on comment generation and multi-view comprehensive analysis

Abstract

The invention discloses a false news detection method based on comment generation and multi-view comprehensive analysis, which utilizes the text analysis capability of a large language model to make up the defect that parameters need to be continuously adjusted when a traditional model detects social news posts through external knowledge and text logic. According to the method, the text generation capability of a large language model is utilized, a self-adaptive comment aggregator and a comment emotion analyzer are designed at the same time, and the problems that at present, social media post user comments are not comprehensive enough, and the contained information amount is small can be solved. The text analysis and text generation capabilities of the large language model are combined with the image-text cross-modal feature comparison method of the traditional small language model, so that more comprehensive and more excellent detection efficiency can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a fake news detection method based on generating comments and multi-perspective comprehensive analysis. Background Art

[0002] Current social media posts usually have multiple interconnected modes. Therefore, it is no longer possible to distinguish true from false news by single-modal detection alone. For example, real images can be combined with false news, or real descriptive language can be combined with tampered images. Therefore, it is necessary to comprehensively analyze false news from multiple modal perspectives. At the same time, with the emergence of various large language models, the use of large language models for multimodal false news detection has also become an important direction. Large language models can effectively capture the parts of news content that are inconsistent with common sense and logic with their rich knowledge and keen observation, thereby improving the accuracy and robustness of false news. In addition, the powerful text generation ability of large language models can provide some information that cannot be obtained by conventional detection, such as comments from silent users. Since comment information can play a very important role in detecting fake news, and currently most of the comment data relies on crawling from traditional social platforms, a major problem faced is that the comment information is not comprehensive enough. Due to the influence of social psychology, confidentiality and other factors, most of the collected comments come from active users, and some special users are missing. Therefore, the model trained based on this comment information lacks generalization and robustness. The professional knowledge and psychological knowledge of the large language model can well simulate the comments made by these special users. Secondly, through the adaptive comment aggregator and comment sentiment analyzer designed by us, the important information of the generated comments can be used to make up for the missing information of the real comments, and the emotional differences between the two comments can be compared, which can greatly enhance the generalization ability of the model and improve the detection efficiency.

[0003] Current research methods still face two major challenges. First, although large language models have been used for fake news detection, they only use the ability of large language models to analyze news or generate text, and do not combine the two to improve the ability to detect fake news. The ability of large language models to process natural language is not only about understanding text, but also about analyzing and applying it. The potential of large language models needs to be further developed. Secondly, although many existing methods extract multimodal features, they ignore the semantic relevance between multimodal features from different angles. They will misjudge news as fake news simply because the features between the modalities are different, affecting the performance of the detection model. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention provides a fake news detection method based on generating comments and multi-perspective comprehensive analysis, which is used to solve the problems mentioned in the above background technology.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a fake news detection method based on generating comments and multi-perspective comprehensive analysis, further comprising the following steps: Step S1: construct news samples and perform data preprocessing on the news samples; Step S2: After data preprocessing, text feature data, image feature data, and real comment data are obtained and annotated with emotion tags; Step S3: Based on the prompt template, a large language model is used to generate several simulated user comments on the news sample through prompts, and the text content of the news sample and the generated comments are comprehensively analyzed to obtain text data for news content analysis; Step S4: Based on the prompt template, the second large language model is used to perform a comprehensive analysis on the text data obtained after the content analysis of the first large language model, the text content of the news sample and the comments generated by the large language model through prompts to obtain news principle feature data; Step S5: The multi-view feature extraction model extracts features from news principle feature data, text feature data, real comment feature data, generated comment and image feature data to obtain corresponding encoding feature vectors; Step S6: The adaptive comment aggregator fuses the generated comment encoding feature vector and the real comment encoding feature vector to obtain a comprehensive comment encoding feature vector, and then fuses the comprehensive comment encoding feature vector and the text encoding feature vector to obtain a text and comment feature fusion vector; Step S7: input the generated comment encoding feature vector and the real comment encoding feature vector into the comment sentiment analyzer to obtain the sentiment difference coefficient by calculating the discrete Gaussian distribution and KL divergence; Step S8: The multi-view feature fusion model fuses the text encoding feature vector, the news principle encoding feature vector, the comprehensive comment encoding feature vector, the image encoding feature vector, and the graphic modality encoding feature vector to obtain a fused feature vector; Step S9: Aggregate the coded feature vector, fused feature vector, text and comment feature fusion vector and sentiment difference coefficient to obtain the final aggregated coded feature vector. The classifier module predicts the true or false prediction label of the news sample corresponding to the final aggregated coded feature vector.

[0006] Furthermore, in step S1, the news sample is constructed and the process of data preprocessing on the news sample is specifically as follows: Step S11: crawling and preprocessing the public social media data set through web crawler technology to obtain the data set required for model training, where a single piece of data in the news data set is a news sample; Step S12: preprocess the data set, delete news samples that lack text content, lack image descriptions or have unclear text descriptions, and obtain a single news data sample including text feature data, image feature data and emotion labels; randomly divide the news data set into a training set and a test set according to preset rules.

[0007] Furthermore, in step S3, the process of using the No. 1 large language model to generate comments and analyze the content of the news text is specifically as follows: Step S31, using the Zhipu Qingyan open source large language model as the No. 1 large language model to simulate social platform users and generate comments based on news samples in the data set; specifically: First, a prompt template is given. The specific prompt template is: This is [News text content]. If you are a user of a social platform, you can randomly select user attributes from [Gender: Male, Female], [Age: Under 18, 18 to 29, 30 to 49, 50 to 64, 64 and above], [Education: No higher education, Currently receiving higher education, Completed higher education], [Standpoint on news content: Support, Oppose, Neutral], and then generate 8 corresponding user comments for this news based on the user attributes you selected, and the word count of each comment is limited to 50 words; Step S32: After obtaining the corresponding generated comments according to the prompt template of step S31, continue to use the Zhipu Qingyan open source large language model as the No. 1 large language model to analyze the news samples and generated comments. The prompt template for the analysis is: This is [news content] and the comments on the corresponding news [comment content]. Now, as an expert in true and false news analysis, please analyze this news based on the news content and the comment content. You can analyze it based on the source, time, background, logic or common sense of the news. You do not need to give a judgment result, just give your analysis. The word count is limited to 200 words.

[0008] Furthermore, in step S4, the second large language model is used to further analyze the analysis content, generated comments and news content of the first large language model to eliminate part of the hallucination process. Specifically, the process is as follows: Use LLaMa open source big language model as the second big language model to re-analyze the news text content, the comments generated by Zhipu Qingyan open source big language model, and the analysis of the news text content. The prompt template for reanalysis is:

[0009] This is the news content [news text content], comment content [generated comments], and the analysis results of other large language models [Zhipu Qingyan large language model news analysis content]. Now, as an expert in true and false news analysis, please make full use of your knowledge to analyze this news. The logic should be clear and in line with professional standards, and you should maintain your independent thinking. You can reflect on or supplement the analysis results of other large models, especially some illusion analysis. You do not need to judge the results, but only give your analysis. The word count is limited to 300 words to obtain the final large language model news text content analysis, that is, the news principle feature data.

[0010] Further, the multi-view feature extraction model in step S5 includes a parallel pre-trained RoBERTa model, a pre-trained CLIP model, and a pre-trained SwinT model; The specific steps are: Step S51: training the RoBERTa model to process the text feature data to obtain a text encoding feature vector; Text feature data is represented as ;in, Represents text feature data, Represents the first text feature data, Represents the second text feature data, Represents the nth text feature data; The text encoding feature vector is represented as ;in, represents the text encoding feature vector, represents the first text encoding feature vector, represents the second text encoding feature vector, Represents the nth text encoding feature vector; Step S52, the pre-trained RoBERTa model processes the news principle feature data to obtain a news principle encoding feature vector; The news principle feature data is expressed as ;in, Represents news principle feature data, Indicates the first news principle feature data, Indicates the second news principle feature data, Represents the nth news principle feature data; The news principle encoding feature vector is expressed as ;in, represents the news principle encoding feature vector, represents the first news principle encoding feature vector, represents the second news principle encoding feature vector, represents the nth news principle encoding feature vector; Step S53: The pre-trained SwinT model processes the image feature data to obtain an image coding feature vector; Among them, the image feature data is represented by Iϵ ; and are the height and width of the image feature data respectively; I= ]; I represents image feature data, Represents the first image feature, Represents the features of the second image, Represents the features of the nth image; The image encoding feature vector is expressed as ; represents the image encoding feature vector, represents the first image encoding feature vector, represents the second image encoding feature vector, represents the nth image encoding feature vector; Step S54: The pre-trained RoBERTa model is used to process the generated comment feature data to obtain a generated comment encoding feature vector; Generate comments Comment feature data is represented as ; Represents the first comment feature data, Represents the second comment feature data, Represents the feature data of the nth comment; Generate comments Comment encoding feature vector is represented as ; represents the first comment encoding feature vector, represents the second comment encoding feature vector, represents the nth comment encoding feature vector; Step S55: The pre-trained RoBERTa model processes the real comment feature data to obtain the real comment encoding feature vector; The real review feature data is represented as ; Represents the first comment feature data, Represents the second comment feature data, Represents the feature data of the nth comment;

[0011] The real review encoding feature vector is represented as ; represents the first comment encoding feature vector, represents the second comment encoding feature vector, represents the nth comment encoding feature vector; Step S56: The pre-trained CLIP model processes the text feature data and the image feature data to obtain a text-image modality encoding feature vector , image-text modality encoding feature vector Divided into image-text modal text encoding feature vector , Image encoding feature vector of text-image modality .

[0012] Furthermore, the specific processing flow of the adaptive comment aggregator in step S6 is as follows: Step S61: fusing the generated comment coding feature vector and the real comment coding feature vector to obtain a comprehensive comment coding feature vector; Step S62: The text encoding feature vector and the comprehensive comment encoding feature vector are fused to obtain a text and comment feature fusion vector: specifically:

[0013] Step S621: Calculate and generate comment encoding feature vector and the true review encoding feature vector The cosine similarity score between them is then multiplied to generate the comment encoding feature vector Get the comprehensive review feature vector , thereby evaluating the important information of generated reviews to make up for the missing information in real reviews; Step S622: Calculate text encoding feature vector and the comment encoding feature vector The correlation matrix between , thereby obtaining important information in text recognition comments and ensuring information transmission between text and comments; Step S623: Convert the correlation matrix as the key vector and the comment encoding feature vector As the value vector, calculate the aggregate information of the key vector and the value vector; use the nonlinear activation function again to normalize the aggregate information to obtain the corresponding attention weight coefficient; use the attention weight coefficient to perform weighted summation on the value vector V to obtain the comment aggregation information; Step S624: Suppressing comment aggregation information through fusion gating mechanism Obtain fusion gating features from negative comments in order to ensure that the generated negative comments do not interfere with the model detection results; Step S625: The gated features are fused through residual connection processing and concatenated with the text encoding feature vector to retain the original features in the text; and a text and comment feature fusion vector is obtained; The processing flow of step S62 is expressed as follows: ; ; ; ; ; ; In the formula, To generate the cosine similarity score between the comment feature vector and the real comment feature vector, is the modulus of the feature vector encoding the real review, To generate the modulus length of the comment encoding feature vector, Encode the feature vector for the comprehensive review, Indicates that the input part is transformed into a query vector, Indicates that the input part is changed into a key vector. Represents the weight matrix, represents the ReLU activation function, represents the fusion gated feature, represents a nonlinear activation function, d represents the preset dimension, T represents transposition, represents the fusion vector of text and comment features, represents element-wise multiplication, represents the product operation, Represents comment aggregation information.

[0014] Furthermore, step S7 is specifically as follows: Step S71: Generate comment sentiment labels and real review sentiment labels Calculate the discrete Gaussian distribution to get the sentiment distribution of the generated comment encoding feature vector and sentiment distribution of the encoded feature vector of the real review ; Step S72: Calculate the sentiment distribution of the generated comment encoding feature vector and the real comment encoding feature vector Divergence, to compare the sentiment difference between the generated comment encoding feature vector and the real comment encoding feature vector, and get the sentiment difference score ; Step S73: Overlay the sentiment difference scores, using Mechanism and nonlinear activation function to obtain the final sentiment difference coefficient between the generated comments and the real comments ; The processing flow of step S7 is expressed as follows: ; ; ; ; ; In the formula, To generate comment sentiment labels, is the real comment sentiment label, is the preset parameter, and To generate the sentiment distance between the comment encoding feature vector and the real comment encoding feature vector, for Divergence, The natural constants are expressed as The exponential function of express network, represents a nonlinear activation function, represents the concatenation operation, i represents the index, and n represents the total number of generated comments.

[0015] Furthermore, the multi-view feature fusion model in step S8 includes a co-attention layer and a mapping layer connected in series; the co-attention layer is composed of two connected attention blocks and a fully connected layer; Step S8 is specifically as follows: The text encoding feature vector and the news principle encoding feature vector are fused to obtain the text and principle feature fusion vector, the text encoding feature vector and the comprehensive comment encoding feature vector are fused to obtain the text and comment feature fusion vector, the text encoding feature vector and the image encoding feature vector are fused to obtain the text and image feature fusion vector, and the image and text modality encoding feature vector extracted by the pre-trained CLIP model is fused to obtain the image and text cross-modal feature fusion vector; the text and principle feature fusion vector, the text and comment feature fusion vector, the text and image feature fusion vector and the image and text cross-modal feature fusion vector are fused to obtain the cross-modal feature alignment vector; specifically: Step S81: The text coding feature vector and the news principle coding feature vector are fused to obtain a text and principle feature fusion vector, specifically: Step S811: Encode the text feature vector and news principle encoding feature vector The first two attention blocks are fed together into the co-attention layer; Step S812, in the first attention block, the text encoding feature vector As the query vector Q, the news principle encoding feature vector As the key vector K and the value vector V, the semantic similarity score is calculated; the attention score is normalized using a nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is then weighted and summed using the attention weight coefficient to obtain the text principle weight vector ; Step S813, in the second attention block, the news principle encoding feature vector As the query vector Q, the text encoding feature vector As the key vector K and the value vector V, the query vector Q and the key vector K are used to calculate the semantic similarity score; the attention score is normalized again using the nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is weighted and summed using the attention weight coefficient to obtain the principle text weight vector ; Step S814: text principle weight vector With the principle text weight vector After concatenation, the vector is input into the fully connected layer to obtain the fusion vector of text and principle features: The processing flow of step S811 to step S814 is expressed as follows: Q= × , K = × ,V= × ; ; ; ; In the formula, Indicates that the input part is turned into a query vector, Indicates that the input part is changed into a key vector. It means to change the input part into a value vector. Represents the execution process of an attention mechanism, represents a nonlinear activation function, d represents the preset dimension of the common attention layer, T represents transposition, Represents the fusion vector of text and principle features, represents the shared attention layer, Represents a splicing operation; Step S82: The text coding feature vector and the image coding feature vector are fused to obtain a text and image feature fusion vector: specifically: Step S821: Encode the text feature vector and the image encoding feature vector The first two attention blocks are fed together into the co-attention layer; Step S822, in the first attention block, the text encoding feature vector As the query vector Q, the image feature encoding feature vector As the key vector K and the value vector V, the semantic similarity score is calculated; the attention score is normalized using a nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is then weighted and summed using the attention weight coefficient to obtain the text image weight vector ; Step S823, in the second attention block, the image encoding feature vector As the query vector Q, the text encoding feature vector As the key vector K and the value vector V, the query vector Q and the key vector K are used to calculate the semantic similarity score; the attention score is normalized again using the nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is weighted and summed using the attention weight coefficient to obtain the image text weight vector ; Step S824: text image weight vector and image text weight vector After splicing, it is input into the fully connected layer to obtain the text and image feature fusion vector. The processing flow of step S821 to step S824 is expressed as follows: Q= × , K = × ,V= × ; ; ; ; In the formula, Indicates that the input part is transformed into a query vector, Indicates that the input part is changed into a key vector. Indicates that the input part is changed into a value vector, Represents the execution process of an attention mechanism, represents a nonlinear activation function, d represents the preset dimension of the common attention layer, T represents transposition, represents the fusion vector of text and image features, represents the shared attention layer, Represents a splicing operation;

[0016] Furthermore, step S8 also includes: Step S83: fusing the image-text modality encoding feature vectors extracted by the pre-trained CLIP model to obtain an image-text cross-modality feature fusion vector; specifically: Step S831: In the first attention operation, the text encoding feature vector of the image-text modality is As the query vector Q, the image encoding feature vector of the text-image modality As the key vector K and value vector V, the semantic connection between the text encoding feature vector of the text-to-image modality and the image encoding feature vector of the text-to-image modality is calculated to obtain the output representation of the first attention operation ; In the second attention operation, the image encoding feature vector of the text-image modality As the query vector Q, the text encoding feature vector of the image-text modality As the key vector K and value vector V, we get the output representation of the second attention operation ; Step S832: Perform average pooling operations on the outputs of the previous and next attention operations to obtain the feature representation after average pooling, and then cascade to obtain the CLIP feature fusion representation The processing flow of step S831 to step S832 is expressed as follows: ; ; In the formula, Represents the cross-modal feature fusion vector of images and texts; Step S84: The text and principle feature fusion vector, the text and comment feature fusion vector, the text and image feature fusion vector and the image-text cross-modal feature fusion vector are fused to obtain a cross-modal feature alignment vector; specifically: Step S841: Encode the text feature vector and the image encoding feature vector After being input into the fully connected layer, it is then input into the co-attention layer; Step S842, when performing the first attention operation, the text encoding feature vector As the query vector Q, the image encoding feature vector As the key vector K and value vector V, the semantic connection between the text encoding feature vector and the image encoding feature vector is calculated to obtain the output representation of the first attention operation ; In the second attention operation, the image encoding feature vector As the query vector, the text encoding feature vector is used as the key vector and the value vector to get the output representation of the second attention block. ; Step S843: Input the outputs of the two attention operations into the average pooling layer for average pooling operation, and then cascade to obtain the cross-modal feature fusion representation. ; The processing flow of step S841 to step S843 is expressed as: ;

[0017] ; In the formula, represents the average pooling process, Represents the operation process of the shared attention layer; Step S844: Fusing cross-modal features and image-text cross-modal feature fusion vector After cascading, the input mapping layer is fused to obtain a cross-modal mapping representation ; Step S845: using the cross-modal semantic similarity score to adjust the key weights between multiple modalities, and calculating the semantic similarity relationship between the text encoding feature vector of the graphic modality and the image encoding feature vector of the graphic modality to obtain the cross-modal semantic similarity score; Step S846: Represent the cross-modal mapping Multiply it with the cross-modal semantic similarity score to obtain the cross-modal feature alignment vector ; The processing flow of step S844 to step S846 is expressed as: ; ; ; In the formula, is the cross-modal mapping representation, MPP is the mapping layer, is the cross-modal semantic similarity score, represents the modulus length of the text encoding feature vector of the image-text modality, represents the modulus length of the image encoding feature vector of the text-to-image modality, Represents the transposed matrix of the encoded feature vector of the text-to-image modality image.

[0018] Furthermore, step S9 is specifically as follows: Aggregate text encoding feature vector, image encoding feature vector, comprehensive comment encoding feature vector, comment sentiment difference coefficient, news principle encoding feature vector, text and principle feature fusion vector, text and image feature fusion vector, text and comment feature fusion vector, image and text cross-modal feature fusion vector, cross-modal feature alignment vector to obtain the final aggregated encoding feature vector, input it into the classifier module, and output the true or false prediction label of the news sample; specifically; ; ; In the formula, represents the predicted label of the classifier module, represents the fully connected layer, represents the cross entropy loss function, Represents the true label of the sample.

[0019] Compared with the existing technology, the present invention has the following beneficial effects:

[0020] (1) The present invention is inspired by the year-end summary, final evaluation and other assessment methods in daily life, and introduces three angles for comprehensive analysis of true and false news content, namely personal detection, peer detection and comprehensive detection. Personal detection is that the No. 1 large language model analyzes the news content containing self-generated comments. Peer detection is that the No. 2 large language model re-analyzes the analysis results of the No. 1 large language model containing news content and generated comments to obtain the final news principle. Comprehensive detection is to jointly detect news content from multiple modes such as text, image, comment, news principle, etc. Multi-angle comprehensive detection has improved detection performance compared with traditional multimodal detection methods.

[0021] (2) The present invention uses the Zhipu Qingyan open source large language model to generate simulated user comments, which supplements the information of those users who remain silent on conventional social platforms. Due to the powerful text generation ability and knowledge accumulation of the large language model, it can well understand the psychological state of those users and the level of news comments. Therefore, the large language model can well generate comments of silent users for fake news detection through artificial prompts. Since the comment information is more comprehensive, the model increases generalization and robustness after extracting the comment features, and the detection efficiency is greatly improved.

[0022] (3) The present invention designs an adaptive comment aggregator to use the important information contained in the generated comments to make up for the missing information in the real comments due to the existence of silent users. At the same time, the aggregator can eliminate single comment information and integrate different and diverse comments together, which can improve the overall performance of the model and make better use of the text generation ability of the large model.

[0023] (4) The present invention designs a comment sentiment analyzer, which contains rich emotions in the comments and can be extracted and effectively applied to fake news detection. However, the generated comments are different from the real comments. It is a more reasonable perspective to compare the difference between the two at the emotional level through the sentiment analyzer. Together with other modalities such as images and texts, the ability to detect fake news can be improved from multiple angles.

[0024] (5) The present invention uses the LLaMa open source large language model to analyze the text content of news. The large language model has acquired a strong knowledge reserve and preliminary ability to distinguish true from false news through a large amount of data training and tens of billions of parameter adjustments. By analyzing the background, logic, common sense and contextual semantics of the news through the large language model, some very useful information can be obtained to facilitate subsequent detection.

[0025] (6) The present invention uses a powerful pre-trained RoBERTa model and a pre-trained SwinT model to extract text encoding feature vectors and image encoding feature vectors, which helps to enhance the model's feature recognition and semantic analysis capabilities for the text content and image content of news, thereby better capturing the semantic connections in news data samples, including using the CLIP pre-trained model to extract text features in the image-text modality and image features in the image-text modality. When the cross-modal features of the image-text are obtained, the comparative learning features of the text and image are also obtained, which significantly improves the feature comparison between the two main modalities.

[0026] (7) The present invention uses a mapping layer in the process of text-image fusion to extract useful information from the text features and image features of the news, while eliminating some useless information, so that the text features of the text-image modality and the image features of the text-image modality extracted by the CLIP pre-training model are mutually compatible, laying a good foundation for subsequent feature complementarity and semantic alignment.

[0027] (8) The present invention fully considers that a large language model can be used for multimodal fake news detection at the beginning of the design. It uses the text generation and news analysis capabilities of a large language model in combination with multi-perspective comprehensive detection, effectively utilizes various modal information, and eliminates some illusions of the large language model. This not only fills the gap in traditional multimodal fake news detection, but also provides new ideas for subsequent technical inventions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the overall framework of the present invention. DETAILED DESCRIPTION

[0029] like Figure 1 As shown, the fake news detection method based on generating comments and multi-perspective comprehensive analysis further includes the following steps: Step S1: construct news samples and perform data preprocessing on the news samples; Step S2: After data preprocessing, text feature data, image feature data, and real comment data are obtained and annotated with emotion tags; Step S3: Based on the prompt template, a large language model is used to generate several simulated user comments on the news sample through prompts, and the text content of the news sample and the generated comments are comprehensively analyzed to obtain text data for news content analysis; Step S4: Based on the prompt template, the second large language model is used to perform a comprehensive analysis on the text data obtained after the content analysis of the first large language model, the text content of the news sample and the comments generated by the large language model through prompts to obtain news principle feature data; Step S5: The multi-view feature extraction model extracts features from news principle feature data, text feature data, real comment feature data, generated comment and image feature data to obtain corresponding encoding feature vectors; Step S6: The adaptive comment aggregator fuses the generated comment encoding feature vector and the real comment encoding feature vector to obtain a comprehensive comment encoding feature vector, and then fuses the comprehensive comment encoding feature vector and the text encoding feature vector to obtain a text and comment feature fusion vector; Step S7: input the generated comment encoding feature vector and the real comment encoding feature vector into the comment sentiment analyzer to obtain the sentiment difference coefficient by calculating the discrete Gaussian distribution and KL divergence; Step S8: The multi-view feature fusion model fuses the text encoding feature vector, the news principle encoding feature vector, the comprehensive comment encoding feature vector, the image encoding feature vector, and the graphic modality encoding feature vector to obtain a fused feature vector; Step S9: Aggregate the coded feature vector, fused feature vector, text and comment feature fusion vector and sentiment difference coefficient to obtain the final aggregated coded feature vector. The classifier module predicts the true or false prediction label of the news sample corresponding to the final aggregated coded feature vector.

[0030] Furthermore, in step S1, the news sample is constructed and the process of data preprocessing on the news sample is specifically as follows: Step S11: crawling and preprocessing the public social media data set through web crawler technology to obtain the data set required for model training, where a single piece of data in the news data set is a news sample; Step S12: preprocess the data set, delete news samples that lack text content, lack image descriptions or have unclear text descriptions, and obtain a single news data sample including text feature data, image feature data and emotion labels; randomly divide the news data set into a training set and a test set according to preset rules.

[0031] Furthermore, in step S3, the process of using the No. 1 large language model to generate comments and analyze the content of the news text is specifically as follows: Step S31, using the Zhipu Qingyan open source large language model as the No. 1 large language model to simulate social platform users and generate comments based on news samples in the data set; specifically: First, a prompt template is given. The specific prompt template is: This is [News text content]. If you are a user of a social platform, you can randomly select user attributes from [Gender: Male, Female], [Age: Under 18, 18 to 29, 30 to 49, 50 to 64, 64 and above], [Education: No higher education, Currently receiving higher education, Completed higher education], [Standpoint on news content: Support, Oppose, Neutral], and then generate 8 corresponding user comments for this news based on the user attributes you selected, and the word count of each comment is limited to 50 words; Step S32: After obtaining the corresponding generated comments according to the prompt template of step S31, continue to use the Zhipu Qingyan open source large language model as the No. 1 large language model to analyze the news samples and generated comments. The prompt template for the analysis is: This is [news content] and the comments on the corresponding news [comment content]. Now, as an expert in true and false news analysis, please analyze this news based on the news content and the comment content. You can analyze it based on the source, time, background, logic or common sense of the news. You do not need to give a judgment result, just give your analysis. The word count is limited to 200 words.

[0032] Furthermore, in step S4, the second large language model is used to further analyze the analysis content, generated comments and news content of the first large language model to eliminate part of the hallucination process. Specifically, the process is as follows: Use LLaMa open source big language model as the second big language model to re-analyze the news text content, the comments generated by Zhipu Qingyan open source big language model, and the analysis of the news text content. The prompt template for reanalysis is:

[0033] This is the news content [news text content], comment content [generated comments], and the analysis results of other large language models [Zhipu Qingyan large language model news analysis content]. Now, as an expert in true and false news analysis, please make full use of your knowledge to analyze this news. The logic should be clear and in line with professional standards, and you should maintain your independent thinking. You can reflect on or supplement the analysis results of other large models, especially some illusion analysis. You do not need to judge the results, but only give your analysis. The word count is limited to 300 words to obtain the final large language model news text content analysis, that is, the news principle feature data.

[0034] Further, the multi-view feature extraction model in step S5 includes a parallel pre-trained RoBERTa model, a pre-trained CLIP model, and a pre-trained SwinT model; The specific steps are: Step S51: training the RoBERTa model to process the text feature data to obtain a text encoding feature vector; Text feature data is represented as ;in, Represents text feature data, Represents the first text feature data, Represents the second text feature data, Represents the nth text feature data; The text encoding feature vector is represented as ;in, represents the text encoding feature vector, represents the first text encoding feature vector, represents the second text encoding feature vector, Represents the nth text encoding feature vector; Step S52, the pre-trained RoBERTa model processes the news principle feature data to obtain a news principle encoding feature vector; The news principle feature data is expressed as ;in, Represents news principle feature data, Indicates the first news principle feature data, Indicates the second news principle feature data, Represents the nth news principle feature data; The news principle encoding feature vector is expressed as ;in, represents the news principle encoding feature vector, represents the first news principle encoding feature vector, represents the second news principle encoding feature vector, represents the nth news principle encoding feature vector; Step S53: The pre-trained SwinT model processes the image feature data to obtain an image coding feature vector; Among them, the image feature data is represented by Iϵ ; and are the height and width of the image feature data respectively; I= ]; I represents image feature data, Represents the first image feature, Represents the features of the second image, Represents the features of the nth image; The image encoding feature vector is expressed as ; represents the image encoding feature vector, represents the first image encoding feature vector, represents the second image encoding feature vector, represents the nth image encoding feature vector; Step S54: The pre-trained RoBERTa model is used to process the generated comment feature data to obtain a generated comment encoding feature vector; Generate comments Comment feature data is represented as ; Represents the first comment feature data, Represents the second comment feature data, Represents the feature data of the nth comment; Generate comments Comment encoding feature vector is represented as ; represents the first comment encoding feature vector, represents the second comment encoding feature vector, represents the nth comment encoding feature vector; Step S55: The pre-trained RoBERTa model processes the real comment feature data to obtain the real comment encoding feature vector; The real review feature data is represented as ; Represents the first comment feature data, Represents the second comment feature data, Represents the feature data of the nth comment;

[0035] The real review encoding feature vector is represented as ; represents the first comment encoding feature vector, represents the second comment encoding feature vector, represents the nth comment encoding feature vector; Step S56: The pre-trained CLIP model processes the text feature data and the image feature data to obtain a text-image modality encoding feature vector , image-text modality encoding feature vector Divided into image-text modal text encoding feature vector , Image encoding feature vector of text-image modality .

[0036] Furthermore, the specific processing flow of the adaptive comment aggregator in step S6 is as follows: Step S61: fusing the generated comment coding feature vector and the real comment coding feature vector to obtain a comprehensive comment coding feature vector; Step S62: The text encoding feature vector and the comprehensive comment encoding feature vector are fused to obtain a text and comment feature fusion vector: specifically:

[0037] Step S621: Calculate and generate comment encoding feature vector and the true review encoding feature vector The cosine similarity score between them is then multiplied to generate the comment encoding feature vector Get the comprehensive review feature vector , thereby evaluating the important information of generated reviews to make up for the missing information in real reviews; Step S622: Calculate text encoding feature vector and the comment encoding feature vector The correlation matrix between , thereby obtaining important information in text recognition comments and ensuring information transmission between text and comments; Step S623: Convert the correlation matrix as the key vector and the comment encoding feature vector As the value vector, calculate the aggregate information of the key vector and the value vector; use the nonlinear activation function again to normalize the aggregate information to obtain the corresponding attention weight coefficient; use the attention weight coefficient to perform weighted summation on the value vector V to obtain the comment aggregation information; Step S624: Suppressing comment aggregation information through fusion gating mechanism Obtain fusion gating features from negative comments in order to ensure that the generated negative comments do not interfere with the model detection results; Step S625: The gated features are fused through residual connection processing and concatenated with the text encoding feature vector to retain the original features in the text; and a text and comment feature fusion vector is obtained; The processing flow of step S62 is expressed as follows: ; ; ; ; ; ; In the formula, To generate the cosine similarity score between the comment feature vector and the real comment feature vector, is the modulus of the feature vector encoding the real review, To generate the modulus length of the comment encoding feature vector, Encode the feature vector for the comprehensive review, Indicates that the input part is transformed into a query vector, Indicates that the input part is changed into a key vector. Represents the weight matrix, represents the ReLU activation function, represents the fusion gated feature, represents a nonlinear activation function, d represents the preset dimension, T represents transposition, represents the fusion vector of text and comment features, represents element-wise multiplication, represents the product operation, Represents comment aggregation information.

[0038] Furthermore, step S7 is specifically as follows: Step S71: Generate comment sentiment labels and real review sentiment labels Calculate the discrete Gaussian distribution to get the sentiment distribution of the generated comment encoding feature vector and sentiment distribution of the encoded feature vector of the real review ; Step S72: Calculate the sentiment distribution of the generated comment encoding feature vector and the real comment encoding feature vector Divergence, to compare the sentiment difference between the generated comment encoding feature vector and the real comment encoding feature vector, and get the sentiment difference score ; Step S73: Overlay the sentiment difference scores, using Mechanism and nonlinear activation function to obtain the final sentiment difference coefficient between the generated comments and the real comments ; The processing flow of step S7 is expressed as follows: ; ; ; ; ; In the formula, To generate comment sentiment labels, is the real comment sentiment label, is the preset parameter, and To generate the sentiment distance between the comment encoding feature vector and the real comment encoding feature vector, for Divergence, The natural constants are expressed as The exponential function of express network, represents a nonlinear activation function, represents the concatenation operation, i represents the index, and n represents the total number of generated comments.

[0039] Furthermore, the multi-view feature fusion model in step S8 includes a co-attention layer and a mapping layer connected in series; the co-attention layer is composed of two connected attention blocks and a fully connected layer; Step S8 is specifically as follows: The text encoding feature vector and the news principle encoding feature vector are fused to obtain the text and principle feature fusion vector, the text encoding feature vector and the comprehensive comment encoding feature vector are fused to obtain the text and comment feature fusion vector, the text encoding feature vector and the image encoding feature vector are fused to obtain the text and image feature fusion vector, and the image and text modality encoding feature vector extracted by the pre-trained CLIP model is fused to obtain the image and text cross-modal feature fusion vector; the text and principle feature fusion vector, the text and comment feature fusion vector, the text and image feature fusion vector and the image and text cross-modal feature fusion vector are fused to obtain the cross-modal feature alignment vector; specifically: Step S81: The text coding feature vector and the news principle coding feature vector are fused to obtain a text and principle feature fusion vector, specifically: Step S811: Encode the text feature vector and news principle encoding feature vector The first two attention blocks are fed together into the co-attention layer; Step S812, in the first attention block, the text encoding feature vector As the query vector Q, the news principle encoding feature vector As the key vector K and the value vector V, the semantic similarity score is calculated; the attention score is normalized using a nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is then weighted and summed using the attention weight coefficient to obtain the text principle weight vector ; Step S813, in the second attention block, the news principle encoding feature vector As the query vector Q, the text encoding feature vector As the key vector K and the value vector V, the query vector Q and the key vector K are used to calculate the semantic similarity score; the attention score is normalized again using the nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is weighted and summed using the attention weight coefficient to obtain the principle text weight vector ; Step S814: text principle weight vector With the principle text weight vector After concatenation, the vector is input into the fully connected layer to obtain the fusion vector of text and principle features: The processing flow of step S811 to step S814 is expressed as follows: Q= × , K = × ,V= × ; ; ; ; In the formula, Indicates that the input part is turned into a query vector, Indicates that the input part is changed into a key vector. It means to change the input part into a value vector. Represents the execution process of an attention mechanism, represents a nonlinear activation function, d represents the preset dimension of the common attention layer, T represents transposition, Represents the fusion vector of text and principle features, represents the shared attention layer, Represents a splicing operation; Step S82: The text coding feature vector and the image coding feature vector are fused to obtain a text and image feature fusion vector: specifically: Step S821: Encode the text feature vector and the image encoding feature vector The first two attention blocks are fed together into the co-attention layer; Step S822, in the first attention block, the text encoding feature vector As the query vector Q, the image feature encoding feature vector As the key vector K and the value vector V, the semantic similarity score is calculated; the attention score is normalized using a nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is then weighted and summed using the attention weight coefficient to obtain the text image weight vector ; Step S823, in the second attention block, the image encoding feature vector As the query vector Q, the text encoding feature vector As the key vector K and the value vector V, the query vector Q and the key vector K are used to calculate the semantic similarity score; the attention score is normalized again using the nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is weighted and summed using the attention weight coefficient to obtain the image text weight vector ; Step S824: text image weight vector and image text weight vector After splicing, it is input into the fully connected layer to obtain the text and image feature fusion vector. The processing flow of step S821 to step S824 is expressed as follows: Q= × , K = × ,V= × ; ; ; ; In the formula, Indicates that the input part is transformed into a query vector, Indicates that the input part is changed into a key vector. Indicates that the input part is changed into a value vector, Represents the execution process of an attention mechanism, represents a nonlinear activation function, d represents the preset dimension of the common attention layer, T represents transposition, represents the fusion vector of text and image features, represents the shared attention layer, Represents a splicing operation;

[0040] Furthermore, step S8 also includes: Step S83: fusing the image-text modality encoding feature vectors extracted by the pre-trained CLIP model to obtain an image-text cross-modality feature fusion vector; specifically: Step S831: In the first attention operation, the text encoding feature vector of the image-text modality is As the query vector Q, the image encoding feature vector of the text-image modality As the key vector K and value vector V, the semantic connection between the text encoding feature vector of the text-to-image modality and the image encoding feature vector of the text-to-image modality is calculated to obtain the output representation of the first attention operation ; In the second attention operation, the image encoding feature vector of the text-image modality As the query vector Q, the text encoding feature vector of the image-text modality As the key vector K and value vector V, we get the output representation of the second attention operation ; Step S832: Perform average pooling operations on the outputs of the previous and next attention operations to obtain the feature representation after average pooling, and then cascade to obtain the CLIP feature fusion representation The processing flow of step S831 to step S832 is expressed as follows: ; ; In the formula, Represents the cross-modal feature fusion vector of images and texts; Step S84: The text and principle feature fusion vector, the text and comment feature fusion vector, the text and image feature fusion vector and the image-text cross-modal feature fusion vector are fused to obtain a cross-modal feature alignment vector; specifically: Step S841: Encode the text feature vector and the image encoding feature vector After being input into the fully connected layer, it is then input into the co-attention layer; Step S842, when performing the first attention operation, the text encoding feature vector As the query vector Q, the image encoding feature vector As the key vector K and value vector V, the semantic connection between the text encoding feature vector and the image encoding feature vector is calculated to obtain the output representation of the first attention operation ; In the second attention operation, the image encoding feature vector As the query vector, the text encoding feature vector is used as the key vector and the value vector to get the output representation of the second attention block. ; Step S843: Input the outputs of the two attention operations into the average pooling layer for average pooling operation, and then cascade to obtain the cross-modal feature fusion representation. ; The processing flow of step S841 to step S843 is expressed as: ;

[0041] ; In the formula, represents the average pooling process, Represents the operation process of the shared attention layer; Step S844: Fusing cross-modal features and image-text cross-modal feature fusion vector After cascading, the input mapping layer is fused to obtain a cross-modal mapping representation ; Step S845: using the cross-modal semantic similarity score to adjust the key weights between multiple modalities, and calculating the semantic similarity relationship between the text encoding feature vector of the graphic modality and the image encoding feature vector of the graphic modality to obtain the cross-modal semantic similarity score; Step S846: Represent the cross-modal mapping Multiply it with the cross-modal semantic similarity score to obtain the cross-modal feature alignment vector ; The processing flow of step S844 to step S846 is expressed as: ; ; ; In the formula, is the cross-modal mapping representation, MPP is the mapping layer, is the cross-modal semantic similarity score, represents the modulus length of the text encoding feature vector of the image-text modality, represents the modulus length of the image encoding feature vector of the text-to-image modality, Represents the transposed matrix of the encoded feature vector of the text-to-image modality image.

[0042] Furthermore, step S9 is specifically as follows: Aggregate text encoding feature vector, image encoding feature vector, comprehensive comment encoding feature vector, comment sentiment difference coefficient, news principle encoding feature vector, text and principle feature fusion vector, text and image feature fusion vector, text and comment feature fusion vector, image and text cross-modal feature fusion vector, cross-modal feature alignment vector to obtain the final aggregated encoding feature vector, input it into the classifier module, and output the true or false prediction label of the news sample; specifically; ; ; In the formula, represents the predicted label of the classifier module, represents the fully connected layer, represents the cross entropy loss function, Represents the true label of the sample.

[0043] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fake news detection method based on generated comments and multi-perspective comprehensive analysis, characterized in that: The steps include: Step S1: construct news samples and perform data preprocessing on the news samples; Step S2: After data preprocessing, text feature data, image feature data, and real comment data are obtained and annotated with emotion tags; Step S3: Based on the prompt template, a large language model is used to generate several simulated user comments on the news sample through prompts, and the text content of the news sample and the generated comments are comprehensively analyzed to obtain text data for news content analysis; Step S4: Based on the prompt template, the second large language model is used to perform a comprehensive analysis on the text data obtained after the content analysis of the first large language model, the text content of the news sample and the comments generated by the large language model through prompts to obtain news principle feature data; Step S5: The multi-view feature extraction model extracts features from news principle feature data, text feature data, real comment feature data, generated comment and image feature data to obtain corresponding encoding feature vectors; Step S6: The adaptive comment aggregator fuses the generated comment encoding feature vector and the real comment encoding feature vector to obtain a comprehensive comment encoding feature vector, and then fuses the comprehensive comment encoding feature vector and the text encoding feature vector to obtain a text and comment feature fusion vector; Step S7: input the generated comment encoding feature vector and the real comment encoding feature vector into the comment sentiment analyzer to obtain the sentiment difference coefficient by calculating the discrete Gaussian distribution and KL divergence; Step S8: The multi-view feature fusion model fuses the text encoding feature vector, the news principle encoding feature vector, the comprehensive comment encoding feature vector, the image encoding feature vector, and the graphic modality encoding feature vector to obtain a fused feature vector; Step S9: Aggregate the coded feature vector, fused feature vector, text and comment feature fusion vector and sentiment difference coefficient to obtain the final aggregated coded feature vector. The classifier module predicts the true or false prediction label of the news sample corresponding to the final aggregated coded feature vector.

2. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 1, characterized in that: In step S1, a news sample is constructed and the process of data preprocessing of the news sample is specifically as follows: Step S11: crawling and preprocessing the public social media data set through web crawler technology to obtain the data set required for model training, where a single piece of data in the news data set is a news sample; Step S12: preprocess the data set, delete news samples that lack text content, lack image descriptions or have unclear text descriptions, and obtain a single news data sample including text feature data, image feature data and emotion labels; randomly divide the news data set into a training set and a test set according to preset rules.

3. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 2, characterized in that: The specific process of using the No. 1 large language model to generate comments and analyze the content of the news text in step S3 is as follows: Step S31, using the Zhipu Qingyan open source large language model as the No. 1 large language model to simulate social platform users and generate comments based on news samples in the data set; specifically: First, a prompt template is given. The specific prompt template is: This is [News text content]. If you are a user of a social platform, you can randomly select user attributes from [Gender: Male, Female], [Age: Under 18, 18 to 29, 30 to 49, 50 to 64, 64 and above], [Education: No higher education, Currently receiving higher education, Completed higher education], [Standpoint on news content: Support, Oppose, Neutral], and then generate 8 corresponding user comments for this news based on the user attributes you selected, and the word count of each comment is limited to 50 words; Step S32: After obtaining the corresponding generated comments according to the prompt template of step S31, continue to use the Zhipu Qingyan open source large language model as the No. 1 large language model to analyze the news samples and generated comments. The prompt template for the analysis is: This is [news content] and the comments on the corresponding news [comment content]. Now, as an expert in true and false news analysis, please analyze this news based on the news content and the comment content. You can analyze it based on the source, time, background, logic or common sense of the news. You do not need to give a judgment result, just give your analysis. The word count is limited to 200 words.

4. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 3, characterized in that: In step S4, the second large language model is used to further analyze the analysis content, generated comments and news content of the first large language model to eliminate part of the hallucination process. Specifically, the process is as follows: Use LLaMa open source big language model as the second big language model to re-analyze the news text content, the comments generated by Zhipu Qingyan open source big language model, and the analysis of the news text content. The prompt template for reanalysis is: This is the news content [news text content], comment content [generated comments], and the analysis results of other large language models [Zhipu Qingyan large language model news analysis content]. Now, as an expert in true and false news analysis, please make full use of your knowledge to analyze this news. The logic should be clear and in line with professional standards, and you should maintain your independent thinking. You can reflect on or supplement the analysis results of other large models, especially some illusion analysis. You do not need to judge the results, but only give your analysis. The word count is limited to 300 words to obtain the final large language model news text content analysis, that is, the news principle feature data.

5. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 4, characterized in that: The multi-view feature extraction model in step S5 includes a parallel pre-trained RoBERTa model, a pre-trained CLIP model, and a pre-trained SwinT model; The specific steps are: Step S51: training the RoBERTa model to process the text feature data to obtain a text encoding feature vector; Text feature data is represented as ;in, Represents text feature data, Represents the first text feature data, Represents the second text feature data, Represents the nth text feature data; The text encoding feature vector is represented as ;in, represents the text encoding feature vector, represents the first text encoding feature vector, represents the second text encoding feature vector, Represents the nth text encoding feature vector; Step S52, the pre-trained RoBERTa model processes the news principle feature data to obtain a news principle encoding feature vector; The news principle feature data is expressed as ;in, Represents news principle feature data, Indicates the first news principle feature data, Indicates the second news principle feature data, Represents the nth news principle feature data; The news principle encoding feature vector is expressed as ;in, represents the news principle encoding feature vector, represents the first news principle encoding feature vector, represents the second news principle encoding feature vector, represents the nth news principle encoding feature vector; Step S53: The pre-trained SwinT model processes the image feature data to obtain an image coding feature vector; Among them, the image feature data is represented by Iϵ ; and are the height and width of the image feature data respectively; I= ]; I represents image feature data, Represents the first image feature, Represents the features of the second image, Represents the features of the nth image; The image encoding feature vector is expressed as ; represents the image encoding feature vector, represents the first image encoding feature vector, represents the second image encoding feature vector, represents the nth image encoding feature vector; Step S54: The pre-trained RoBERTa model is used to process the generated comment feature data to obtain a generated comment encoding feature vector; Generate comments Comment feature data is represented as ; Represents the first comment feature data, Represents the second comment feature data, Represents the feature data of the nth comment; Generate comments Comment encoding feature vector is represented as ; represents the first comment encoding feature vector, represents the second comment encoding feature vector, represents the nth comment encoding feature vector; Step S55: The pre-trained RoBERTa model processes the real comment feature data to obtain the real comment encoding feature vector; The real review feature data is represented as ; Represents the first comment feature data, Represents the second comment feature data, Represents the feature data of the nth comment; The real review encoding feature vector is represented as ; represents the first comment encoding feature vector, represents the second comment encoding feature vector, represents the nth comment encoding feature vector; Step S56: The pre-trained CLIP model processes the text feature data and the image feature data to obtain a text-image modality encoding feature vector , image-text modality encoding feature vector Divided into image-text modal text encoding feature vector , Image encoding feature vector of text-image modality .

6. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 5, characterized in that: The specific processing flow of the adaptive comment aggregator in step S6 is as follows: Step S61: fusing the generated comment coding feature vector and the real comment coding feature vector to obtain a comprehensive comment coding feature vector; Step S62: The text encoding feature vector and the comprehensive comment encoding feature vector are fused to obtain a text and comment feature fusion vector: specifically: Step S621: Calculate and generate comment encoding feature vector and the true review encoding feature vector The cosine similarity score between them is then multiplied to generate the comment encoding feature vector Get the comprehensive review feature vector , thereby evaluating the important information of generated reviews to make up for the missing information in real reviews; Step S622: Calculate text encoding feature vector and the comment encoding feature vector The correlation matrix between , thereby obtaining important information in text recognition comments and ensuring information transmission between text and comments; Step S623: Convert the correlation matrix as the key vector and the comment encoding feature vector As the value vector, calculate the aggregate information of the key vector and the value vector; use the nonlinear activation function again to normalize the aggregate information to obtain the corresponding attention weight coefficient; use the attention weight coefficient to perform weighted summation on the value vector V to obtain the comment aggregation information; Step S624: Suppressing comment aggregation information through fusion gating mechanism Obtain fusion gating features from negative comments in order to ensure that the generated negative comments do not interfere with the model detection results; Step S625: The gated features are fused through residual connection processing and concatenated with the text encoding feature vector to retain the original features in the text; and a text and comment feature fusion vector is obtained; The processing flow of step S62 is expressed as follows: ； ； ； ； ； ； In the formula, To generate the cosine similarity score between the comment feature vector and the real comment feature vector, is the modulus of the feature vector encoding the real review, To generate the modulus length of the comment encoding feature vector, Encode the feature vector for the comprehensive review, Indicates that the input part is transformed into a query vector, Indicates that the input part is changed into a key vector. Represents the weight matrix, represents the ReLU activation function, represents the fusion gated feature, represents a nonlinear activation function, d represents the preset dimension, T represents transposition, represents the fusion vector of text and comment features, represents element-wise multiplication, represents the product operation, Represents comment aggregation information.

7. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 6, characterized in that: Step S7 is specifically as follows: Step S71: Generate comment sentiment labels and real review sentiment labels Calculate the discrete Gaussian distribution to get the sentiment distribution of the generated comment encoding feature vector and sentiment distribution of the encoded feature vector of the real review ; Step S72: Calculate the sentiment distribution of the generated comment encoding feature vector and the real comment encoding feature vector Divergence, to compare the sentiment difference between the generated comment encoding feature vector and the real comment encoding feature vector, and get the sentiment difference score ; Step S73: Overlay the sentiment difference scores, using Mechanism and nonlinear activation function to obtain the final sentiment difference coefficient between the generated comments and the real comments ; The processing flow of step S7 is expressed as follows: ； ； ； ； ； In the formula, To generate comment sentiment labels, is the real comment sentiment label, is the preset parameter, and To generate the sentiment distance between the comment encoding feature vector and the real comment encoding feature vector, for Divergence, The natural constants are expressed as The exponential function of express network, represents a nonlinear activation function, represents the concatenation operation, i represents the index, and n represents the total number of generated comments.

8. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 7, characterized in that: The multi-view feature fusion model in step S8 includes a co-attention layer and a mapping layer connected in series; the co-attention layer consists of two connected attention blocks and a fully connected layer; Step S8 is specifically as follows: The text encoding feature vector and the news principle encoding feature vector are fused to obtain the text and principle feature fusion vector, the text encoding feature vector and the comprehensive comment encoding feature vector are fused to obtain the text and comment feature fusion vector, the text encoding feature vector and the image encoding feature vector are fused to obtain the text and image feature fusion vector, and the image and text modality encoding feature vector extracted by the pre-trained CLIP model is fused to obtain the image and text cross-modal feature fusion vector; the text and principle feature fusion vector, the text and comment feature fusion vector, the text and image feature fusion vector and the image and text cross-modal feature fusion vector are fused to obtain the cross-modal feature alignment vector; specifically: Step S81: The text coding feature vector and the news principle coding feature vector are fused to obtain a text and principle feature fusion vector, specifically: Step S811: Encode the text feature vector and news principle encoding feature vector The first two attention blocks are fed together into the co-attention layer; Step S812, in the first attention block, the text encoding feature vector As the query vector Q, the news principle encoding feature vector As the key vector K and the value vector V, the semantic similarity score is calculated; the attention score is normalized using a nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is then weighted and summed using the attention weight coefficient to obtain the text principle weight vector ; Step S813, in the second attention block, the news principle encoding feature vector As the query vector Q, the text encoding feature vector As the key vector K and the value vector V, the query vector Q and the key vector K are used to calculate the semantic similarity score; the attention score is normalized again using the nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is weighted and summed using the attention weight coefficient to obtain the principle text weight vector ; Step S814: text principle weight vector With the principle text weight vector After concatenation, the vector is input into the fully connected layer to obtain the fusion vector of text and principle features: The processing flow of step S811 to step S814 is expressed as follows: Q= × ,K= × ,V= × ; ； ； ； In the formula, Indicates that the input part is turned into a query vector, Indicates that the input part is changed into a key vector. It means to change the input part into a value vector. Represents the execution process of an attention mechanism, represents a nonlinear activation function, d represents the preset dimension of the common attention layer, T represents transposition, Represents the fusion vector of text and principle features, represents the shared attention layer, Represents a splicing operation; Step S82: The text coding feature vector and the image coding feature vector are fused to obtain a text and image feature fusion vector: specifically: Step S821: Encode the text feature vector and the image encoding feature vector The first two attention blocks are fed together into the co-attention layer; Step S822, in the first attention block, the text encoding feature vector As the query vector Q, the image feature encoding feature vector As the key vector K and the value vector V, the semantic similarity score is calculated; the attention score is normalized using a nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is then weighted and summed using the attention weight coefficient to obtain the text image weight vector ; Step S823, in the second attention block, the image encoding feature vector As the query vector Q, the text encoding feature vector As the key vector K and the value vector V, the query vector Q and the key vector K are used to calculate the semantic similarity score; the attention score is normalized again using the nonlinear activation function to obtain the corresponding attention weight coefficient; the value vector V is weighted and summed using the attention weight coefficient to obtain the image text weight vector ; Step S824: text image weight vector and image text weight vector After splicing, it is input into the fully connected layer to obtain the text and image feature fusion vector. The processing flow of step S821 to step S824 is expressed as follows: Q= × ,K= × ,V= × ; ； ； ； In the formula, Indicates that the input part is transformed into a query vector, Indicates that the input part is changed into a key vector. Indicates that the input part is changed into a value vector, Represents the execution process of an attention mechanism, represents a nonlinear activation function, d represents the preset dimension of the common attention layer, T represents transposition, represents the fusion vector of text and image features, represents the shared attention layer, Represents a concatenation operation.

9. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 8, characterized in that: Step S8 also includes: Step S83: fusing the image-text modality encoding feature vectors extracted by the pre-trained CLIP model to obtain an image-text cross-modality feature fusion vector; specifically: Step S831: In the first attention operation, the text encoding feature vector of the image-text modality is As the query vector Q, the image encoding feature vector of the text-image modality As the key vector K and value vector V, the semantic connection between the text encoding feature vector of the text-to-image modality and the image encoding feature vector of the text-to-image modality is calculated to obtain the output representation of the first attention operation ; In the second attention operation, the image encoding feature vector of the text-image modality As the query vector Q, the text encoding feature vector of the image-text modality As the key vector K and value vector V, we get the output representation of the second attention operation ; Step S832: Perform average pooling operations on the outputs of the previous and next attention operations to obtain the feature representation after average pooling, and then cascade to obtain the CLIP feature fusion representation The processing flow of step S831 to step S832 is expressed as follows: ； ； In the formula, Represents the cross-modal feature fusion vector of images and texts; Step S84: The text and principle feature fusion vector, the text and comment feature fusion vector, the text and image feature fusion vector and the image-text cross-modal feature fusion vector are fused to obtain a cross-modal feature alignment vector; specifically: Step S841: Encode the text feature vector and the image encoding feature vector After being input into the fully connected layer, it is then input into the co-attention layer; Step S842, when performing the first attention operation, the text encoding feature vector As the query vector Q, the image encoding feature vector As the key vector K and value vector V, the semantic connection between the text encoding feature vector and the image encoding feature vector is calculated to obtain the output representation of the first attention operation ; In the second attention operation, the image encoding feature vector As the query vector, the text encoding feature vector is used as the key vector and the value vector to get the output representation of the second attention block. ; Step S843: Input the outputs of the two attention operations into the average pooling layer for average pooling operation, and then cascade to obtain the cross-modal feature fusion representation. ; The processing flow of step S841 to step S843 is expressed as: ； ； In the formula, represents the average pooling process, Represents the operation process of the shared attention layer; Step S844: Fusing cross-modal features and image-text cross-modal feature fusion vector After cascading, the input mapping layer is fused to obtain a cross-modal mapping representation ; Step S845: using the cross-modal semantic similarity score to adjust the key weights between multiple modalities, and calculating the semantic similarity relationship between the text encoding feature vector of the graphic modality and the image encoding feature vector of the graphic modality to obtain the cross-modal semantic similarity score; Step S846: Represent the cross-modal mapping Multiply it with the cross-modal semantic similarity score to obtain the cross-modal feature alignment vector ; The processing flow of step S844 to step S846 is expressed as: ； ； ； In the formula, is the cross-modal mapping representation, MPP is the mapping layer, is the cross-modal semantic similarity score, represents the modulus length of the text encoding feature vector of the image-text modality, represents the modulus length of the image encoding feature vector of the text-to-image modality, Represents the transposed matrix of the encoded feature vector of the text-to-image modality image.

10. The method for detecting fake news based on generating comments and comprehensive analysis from multiple perspectives according to claim 9, characterized in that: Step S9 is specifically as follows: Aggregate text encoding feature vector, image encoding feature vector, comprehensive comment encoding feature vector, comment sentiment difference coefficient, news principle encoding feature vector, text and principle feature fusion vector, text and image feature fusion vector, text and comment feature fusion vector, image and text cross-modal feature fusion vector, cross-modal feature alignment vector to obtain the final aggregated encoding feature vector, input it into the classifier module, and output the true or false prediction label of the news sample; specifically; ； ； In the formula, represents the predicted label of the classifier module, represents the fully connected layer, represents the cross entropy loss function, Represents the true label of the sample.

Images