A fake news oriented detection method and related device

Abstract

The application provides a fake news detection method and related equipment, and belongs to the technical field of natural language processing and information authenticity detection. The method comprises the following steps: preprocessing and segmenting an original input text to obtain a sentence set and score a candidate claim set obtained; converting the sentence into a single sentence declarative expression to obtain a normalized claim, and structurally constructing the normalized claim; simultaneously performing information retrieval according to the structural construction mode to obtain a candidate evidence, performing multidimensional consistency verification on the structured claim and the candidate evidence, and scoring to obtain a total verification score; and correcting the total verification score by using the matching degree of a news site and an IP address, and outputting a verification result. The application prepositions noise processing and fact positioning, adopts a structured claim for fine-grained verification, and corrects through cross-information source matching of an IP address and a news site, so that the accuracy, interpretability and identification ability for numerical / time tampering of fake news detection are improved.

Description

Technical Field

[0001] This application relates to the fields of natural language processing and information authenticity detection technology, and in particular to a method and related equipment for detecting fake news. Background Technology

[0002] With the widespread use of the internet and social media, the rapid spread of fake news has posed a serious threat to social order and public perception. To address this issue, academia and industry have proposed various methods for detecting fake news.

[0003] Existing fake news detection methods mostly employ coarse-grained text classification strategies, directly inputting news articles, social media posts, and image-text composites into deep learning models, which then output a "true / false" binary classification result or a probability of authenticity. However, this type of method has the following significant drawbacks in practical applications: (1) Poor interpretability: The model only provides the overall judgment result, and cannot locate the specific erroneous factual claims, nor can it distinguish specific error types such as numerical errors, time errors, relational errors or inconsistent statistical standards, making it difficult for users to trust the judgment result.

[0004] (2) Noise sensitivity: Social media texts contain a large amount of redundant information that is irrelevant to fact-checking, such as usernames, links, hashtags, OCR errors, emotional expressions, and calls to action. This noise seriously interferes with the model's judgment, leading to an increased false positive rate.

[0005] (3) Weak ability to detect numerical and time-series misinformation: In reality, misinformation often misleads people by exaggerating quantities, tampering with time, confusing statistical standards, and changing comparison benchmarks. Existing methods are unable to effectively identify such misleading information due to a lack of fine-grained numerical comparison and time interval analysis capabilities.

[0006] (4) Unstructured claims: Existing methods typically only output free text when extracting factual claims, without explicitly breaking down key elements such as subject, predicate, object, quantity, time, location, and comparison relationship. When comparing evidence, information extraction needs to be repeated, which is not only computationally inefficient but also prone to error accumulation. Summary of the Invention

[0007] The main objective of this application is to propose a method, electronic device, storage medium, and program product for detecting fake news, so as to at least partially solve the technical problems in the prior art, such as poor interpretability, sensitivity to noise, weak ability to detect numerical and temporal false information, and unstructured claim representation.

[0008] To achieve the above objectives, one aspect of this application proposes a method for detecting fake news, the method comprising: S1: Preprocess and segment the original input text to obtain a set of sentences. Perform verifiability scoring on each sentence in the set of sentences and select the sentences with the highest scores as a set of candidate claims. S2: Normalize each sentence in the candidate claim set, remove its non-factual information, and transform it into a single declarative expression to obtain a normalized claim; S3: Construct each of the above-described normalized claims into a structured claim that includes subject, predicate, object, quantity, time, place, relation, and modifiers and scope constraints; S4: Perform a retrieval query based on at least one key field in the structured claim to obtain a candidate evidence set; S5: Perform multi-dimensional consistency verification between the structured claim and each candidate evidence in the candidate evidence set to obtain a single claim authenticity score. The multi-dimensional consistency verification includes at least one of numerical consistency verification and temporal consistency verification. S6: Combine the authenticity scores of multiple single claims corresponding to multiple structured claims obtained from multiple normalized claims to obtain the total verification score; S7: Obtain the news location extracted from the original input text and the IP address of the news webpage corresponding to the original input text. Based on the degree of matching between the geographical location of the news location and the IP address, correct the total verification score and output the verification result.

[0009] In some embodiments, in step S1: The preprocessing includes at least one of the following: duplicate segment removal, metadata cleaning, OCR / layout repair, and emotional expression filtering of the input text; The verifiability score is based on at least one of the factual score, verifiability score, and main information focus score for each sentence.

[0010] In some embodiments, step S2 employs a generative model trained based on joint loss for the normalization process. The joint loss includes sequence generation loss, assertion constraint loss, length constraint loss, and fact-keeping loss. Furthermore, constraint decoding is used during the generation process to filter social media spam, control the length, and detect sentence templates in the output text.

[0011] In some embodiments, the structured claim Cs in step S3 is represented as: Cs=(s,p,o,q,t,l,r,m), where s is the subject (or predicate), p is the predicate (or action), o is the object (or verb), q is the quantity or proportion, t is the time, l is the location, r is the relation, and m is the modification and scope constraint.

[0012] In some embodiments, in step S4: based on the key fields in the structured claim, candidate evidence is retrieved from news databases, open web pages, fact-checking databases, or local databases using a dual-channel retrieval method; and the candidate evidence set is determined based on semantic similarity, slot matching degree, and slot ranking score; the slot matching degree includes a measure of the consistency of at least one slot among subject, predicate, object, quantity, time, location, relationship, modification, and scope constraints.

[0013] In some embodiments, in step S5: The multi-dimensional consistency verification also includes at least one of semantic consistency verification, relational consistency verification and scope consistency verification; The authenticity score of a single claim is based on the sum of the product of the consistency verification results of each dimension and the corresponding weights; For the numerical consistency verification, when the units of quantity in the structured claim are inconsistent with those in the candidate evidence, a uniform unit conversion is used to calculate the proximity.

[0014] In some embodiments, in step S6: the authenticity scores of multiple individual claims are weighted and fused to obtain the total verification score; wherein, the weight corresponding to each authenticity score of an individual claim is related to the importance of the corresponding structured claim, and the importance is determined based on at least one of the following factors: whether it is a headline fact, whether it is located at the core of the main communication narrative, whether it contains quantity or time information, or whether it appears repeatedly in the original input text.

[0015] In some embodiments, in step S7: after correcting the total verification score according to the matching degree, the authenticity of the input text is determined, and a verification result including "true", "false" or "partially misleading" is output.

[0016] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0017] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0018] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0019] Compared with the prior art, the beneficial effects of this application are as follows: (1) This application adopts preprocessing, candidate claim set screening and normalized claim generation, which moves noise processing and fact location out of the fake news detection process, significantly reduces interference in subsequent information processing, and improves the processing speed and accuracy of the system.

[0020] (2) This application constructs a structured claim that includes subject, predicate, object, quantity, time, location, relationship, modification and scope constraints, transforms natural language facts into computable slot data, expands the scope of information retrieval, improves the accuracy and processing efficiency of information judgment during weighted fusion, and makes the detection results highly interpretable.

[0021] (3) This application achieves cross-information source combination of mathematical model algorithm and network server geographical location by matching and comparing IP address with news location, which further improves the accuracy of judging geographical location-related fake news. Attached Figure Description

[0022] Figure 1 This is a flowchart of a method for detecting fake news provided in an embodiment of this application; Figure 2 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0025] To address the technical problems existing in the prior art, this application provides a method, electronic device, storage medium, and program product for detecting fake news. The solution includes: preprocessing and segmenting the original input text to obtain a set of sentences and a set of candidate claims obtained through scoring; transforming the sentences into single-sentence declarative expressions to obtain standardized claims; structurally constructing the standardized claims; simultaneously retrieving information according to the structured construction method to obtain candidate evidence; comparing and scoring the structured claims with the candidate evidence to obtain a total verification score; and correcting the total verification score using the matching degree between news locations and IP addresses to obtain the output result. This solution employs preprocessing, a candidate claim set, and standardized claims, bringing noise processing and fact-finding forward from fake news detection. The segmentation method of structured claims expands the scope of information retrieval. By comparing IP addresses and news locations, it combines mathematical model algorithms with network server positioning, improving judgment accuracy.

[0026] The fake news detection method provided in this application relates to the fields of natural language processing and information authenticity detection technology. The fake news detection method provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the fake news detection method, but is not limited to the above forms.

[0027] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0028] like Figure 1 As shown, this embodiment provides a method for detecting fake news, specifically including the following steps: Step S1: Preprocess and segment the original input text to obtain a set of sentences, score the set of sentences, and obtain a set of candidate claims.

[0029] For example, the original input text can be any of the following: news article, social media post, text and image caption, or video description text. The input text is represented as:

[0030] in, For the first A token.

[0031] In one embodiment, preprocessing includes the following sub-steps: deduplication of duplicate segments, metadata cleaning, OCR / layout repair, and emotional expression filtering of the input text, as detailed below: 1) Removing duplicate segments Enter text x Segmented into collections of fragments by sentence or sliding window. Define fragment similarity:

[0032] in, and These represent the first and second parts of the input text after segmentation. The segment and the first A segment; and Let represent the weight coefficients of the literal similarity term and the semantic similarity term, respectively, and satisfy . ; For the intersection, union, and comparison of word sets; This is a semantic vector for the fragment.

[0033] like:

[0034] in, This represents the threshold for determining duplicate segments, used to judge whether two segments reach a level of redundancy. When the segment similarity is greater than this threshold, one of the segments is determined to be a duplicate segment, and one of the segments is deleted or merged according to a preset strategy; otherwise, the two segments are considered not to constitute a redundant duplicate relationship and are retained.

[0035] 2) Metadata Cleaning Using metadata filtering functions:

[0036] in Used to identify and remove: usernames, forwarding identifiers, timestamps; URLs, hashtags, platform button text; invalid placeholders, such as "None".

[0037] 3) OCR / Layout Repair Using functions:

[0038] in Used to handle: line breaks; abnormal all-caps sequences; misaligned concatenation of multiple OCR columns; and reconstruction of broken short sentences.

[0039] In multi-column OCR scenarios, the optimal row order can be determined by minimizing the language model loss:

[0040] in, The loss function of the language model is used to measure the linguistic fluency and semantic coherence of a text sequence given the order in which the segments are arranged. The smaller the loss, the more the arrangement conforms to the rules of natural language expression. Indicates in arrangement The next one is in the Text fragments at each location, That is, the original set of fragments. The sequence elements obtained after rearranging.

[0041] 4) Filtering of emotional expressions Utilizing an emotion / call to action rating system:

[0042] in Sentence segment The probability that it belongs to a non-factual expression.

[0043] like:

[0044] Then, the sentence segment will be removed or its weight reduced. Among them, This represents the threshold for filtering emotional expressions, used to determine whether a sentence or paragraph has an excessively strong tendency to express emotions, appeals, or non-factual information.

[0045] In one embodiment, segmentation and scoring include: Text after OCR / layout repair Segmented into a set of sentences:

[0046] For each sentence Calculate the verifiability score:

[0047] in Sentence It is a verifiable claim. This represents a conditional probability function. In a given sentence Under the given conditions, the probability that the sentence belongs to a verifiable claim.

[0048] Specifically, the score is obtained by a linear combination of the following three sub-scores:

[0049] in, , , These represent the weighting coefficients for the factual score, verifiability score, and main information focus score, respectively, used to control the contribution of the three types of sub-scores to the final verifiability score, and preferably satisfying the following conditions: ; Fact score: Used to determine whether a sentence is a statement of fact, rather than an expression of emotion, appeal, or attitude; Verifiability score: Used to determine whether a sentence contains content that can be supported or refuted by evidence; The main information focus score is used to determine whether a sentence is close to the core narrative of the entire text.

[0050] As one specific implementation method, It can be output by a binary classification model; It can be output by the claimcheck-worthiness model; It can be obtained from the similarity score between the sentence and the full text vector or the graph centrality score.

[0051] In one embodiment, top- is retained. The highest-scoring sentence:

[0052] in, This represents the total number of sentences obtained after segmentation. Indicates from the The number of candidate claims with the highest scores is selected from each sentence, and The two do not correspond.

[0053] If multiple candidate sentences are retained, then an inter-sentence graph is further constructed. The edge weight is defined as:

[0054] in: A graph representing the inter-sentence relationships constructed between candidate sentences. This represents a set of nodes, where each node corresponds to a candidate sentence. This represents a set of edges, where each edge represents a relationship between sentences. Represents a node With nodes The edge weights are used to measure the strength of the association between two candidate sentences; Let represent the weight coefficients of the semantic similarity terms, coreference terms, and temporal correlation terms in the sentence vectors, respectively, preferably satisfying the following conditions: ; Sentence With sentences The vector cosine similarity is used to measure the semantic similarity between two sentences; Sentence With sentences The strength of the coreference between sentences is used to reflect whether the two sentences involve the same subject, event, or referent. Sentence With sentences The strength of the temporal connection between two sentences reflects the degree to which they are sequential, overlapping, or refer to the same point in time or time period.

[0055] Then, graph aggregation is used to obtain a unified set of candidate claims. .

[0056] Step S2: Normalize each sentence in the candidate claim set, remove non-factual information, and transform it into a single-sentence declarative expression to obtain a normalized claim.

[0057] Given a set of candidate claims Using generative models Generate standardized claims During the training phase, the model is trained using joint loss. The joint loss includes sequence generation loss, claim constraint loss, length constraint loss, and fact-keeping loss. During the inference stage, constraint decoding is used to filter social media spam, control the length of the output text, and detect whether the sentence structure of the normalized claim conforms to the sentence template.

[0058] 1) Given a set of candidate claims Generate standardized claims:

[0059] in, This represents a claim normalization function or claim normalization model, used to unify the set of candidate claims. Transform into standardized claims that are single-sentence, clear, and verifiable; The parameters represent the normalized model.

[0060] 2) Joint loss defined by the training objective:

[0061] in: Sequence generation loss; Claim constraint loss ensures that the output is a verifiable claim; Length constraint loss to avoid generating excessively long summaries; This is to prevent the loss of factual accuracy and to avoid generating new facts that do not exist in the original text.

[0062] , and These represent the weighting coefficients for assertion constraint loss, length constraint loss, and fact-keeping loss, respectively, used to adjust the relative importance of each loss term in the overall training objective.

[0063] Joint losses Used for normalizing model parameters during the training phase. Optimize the model to ensure that it meets the requirements of readability, verifiability, simplicity, and factual accuracy when generating standardized claims.

[0064] 2.1) Sequence generation loss It can be written directly in the form of cross-entropy:

[0065] in, Indicates the position of the standardization claim. The target token; This represents the sequence of tokens that have already been generated; The normalized model is indicated by the condition The conditional probability of generating the target token.

[0066] 2.2) Loss due to assertive constraints It can be written as:

[0067] in, A truth label indicating whether the generated result is a verifiable claim; This indicates the assertion discriminator's judgment. The probability of a verifiable claim.

[0068] 2.3) Length constraint loss It can be written as:

[0069] in, This represents the maximum length threshold allowed by the normalization claim, used to control the generated results from not exceeding a preset length range, in order to avoid the output degenerating into a long summary or redundant rewriting.

[0070] 2.4) Loss due to failure to verify facts It can be written as:

[0071] in, Representing text For text The degree of implication is used to characterize the text. Does it semantically support or contain text? The main factual content, the preferred value is located in Interval.

[0072] 3) During the constraint decoding stage, template and length constraints are applied to the output:

[0073] in: Given a unified set of candidate claims Under the given conditions, the normalized model generates candidate output sequences. The conditional probability; , , : These represent the weight coefficients of the social media spam penalty, length penalty, and assertion sentence structure reward, respectively, used to adjust the contribution of each constraint to the decoding scoring function; Does the generated result contain social media spam? Is it too long? Does it conform to the assertion sentence template?

[0074] Step S3: Construct each normalized claim into a structured claim.

[0075] As one implementation method, structured advocacy Cs Represented as an 8-tuple: Cs =( s,p,o,q,t,l,r,m ),in s As the main body / subject, p For predicates / actions o For the object / subject q For quantity or proportion, t For time, l For location, r For relationships, m For modification and range constraints. Among them, q, t, r This can be further refined as follows:

[0076] in: : numerical value; :unit; Time granularity; : Relationship type; : Benchmark for comparison.

[0077] There are two construction methods: 1) The first method is: two-stage extraction; 1.1) The first stage involves sequence labeling to obtain the label sequence:

[0078] The tag collection contains:

[0079] in, Indicates the starting position (Begin) of a semantic slot segment. Indicates the internal position (Inside) of this semantic slot segment; Indicates the subject / main phrase slot. Indicates predicate / behavior slots. Indicates the object / verb slot. Indicates the time slot. Indicates the number of slots. Indicates the location slot. Indicates the relationship slot. This indicates the slot for modification and range constraints.

[0080] Training loss:

[0081] in, Used to convert conditional probabilities into log-likelihood form in order to construct the sequence labeling training loss; This indicates that, given the input, normalization is advocated. Under the conditions, the first The location was labeled. The conditional probability, which is output by the sequence labeling model.

[0082] 1.2) The second stage involves standardizing the extracted spans, for example: 2.3 million

[0083] 20 percent

[0084] "last year"

[0085] "higher than"

[0086] 2) The second method is: constraint generation, directly outputting the linearized slot sequence:

[0087] Corresponding loss:

[0088] in, The weighting coefficients representing the structural legality constraint loss are used to control the importance of slot order, type, and integrity constraints; The weighting coefficients representing the normalization constraint loss are used to adjust the impact of normalization errors in quantity, time, and relational fields on the overall loss.

[0089] The loss represents the slot sequence generation loss, used to constrain the model to correctly generate linearized structured slot sequences. Its expression is:

[0090] in, The length of the slot sequence; For position Target slot token; For position Previous slot token sequence; The standardization of input; Given a normalized claim and a preceding slot token, calculate the conditional probability of generating the current slot token.

[0091] The structural validity constraint loss is used to constrain the linearized slot sequence generated by the model to meet preset slot order, field type, and required field integrity requirements. Its expression is:

[0092] in, The number of slots in the preset slot template; For the first in the model output The actual sequential position of each slot; For the first in the preset template The standard sequential position of each slot; This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise. This is the set of slots that require type checking. Slot values ​​generated for the model; The actual type of the generated slot value; For the groove Target type; This is a set of required slots; Indicates slot Not generated or the generated result is empty. The above three items correspond to penalties for incorrect slot order, incorrect slot type, and missing required slots, respectively.

[0093] This represents the slot normalization constraint loss, used to constrain the quantity, time, and relational fields of the model output to be normalized and mapped to a unified representation. Its expression is:

[0094] in, Normalized values ​​for the quantity field output by the model, using a unified unit; This refers to the standardized numerical values ​​of the quantity field in the corresponding standard answer, within the same standard unit. and These are the normalized start and end times of the model output time field, respectively. and These are the start and end times of the corresponding time fields in the standard answer; The type of relation output by the model; This refers to the target relation type in the standard answer; This is an indicator function; it takes a value of 1 when the model's output relation type is inconsistent with the target relation type, and a value of 0 otherwise. The above three terms correspond to the quantity normalization error, time normalization error, and relation normalization error, respectively.

[0095] Structured examples, such as normalized claims: "Public transportation accounts for 60% of daily travel in a certain city." It can be represented as:

[0096] For example: "The amount of new energy power generation in a certain region has increased by 150% in the past five years." It can be represented as:

[0097] The subsequent system will no longer need to guess which words represent quantity, time, and comparison relationships from the free text, but can directly perform slot-level comparison.

[0098] Step S4: Perform a search query based on the key fields in the structured claims to obtain a set of candidate evidence.

[0099] Specifically, in step S4, a search query is performed based on the key fields in the structured claim to retrieve candidate evidence from news databases, open web pages, fact-checking databases, or local databases to obtain a candidate evidence set. During the search, a dual-channel search is used, and candidate evidence is determined based on semantic similarity, slot matching degree, and slot ranking score. Slot matching degree includes the degree of consistency of subject / verb, predicate / action, object / object, quantity, time, location, relationship, modification, and scope constraints.

[0100] 1) Candidate evidence Definition

[0101] : No. Candidate evidence text; Source identifiers, such as news sites, fact-checking databases, and knowledge base entries; Evidence corresponds to time information; Other metadata. It refers to a specific instance of evidence, which can be a news report, a fact-checking entry, a fragment from an encyclopedia / knowledge base, or a search result.

[0102] 2) Constructing queries based on structured claims:

[0103] in Based on the importance of fields, keyword queries, sparse search representations, and dense search representations can be combined.

[0104] 3) Dual-channel retrieval

[0105] in, Representing a structured claim With candidate evidence Initial search scores between; Indicates based on query With candidate evidence Sparse retrieval score; Representing a structured claim Dense semantic vector representation; Indicate candidate evidence Dense semantic vector representation; This represents the cosine similarity between the two vectors; and These represent the weight coefficients of the sparse and dense search channels, respectively, preferably satisfying the following conditions: .

[0106] 4) Take the top- Candidate evidence:

[0107] 5) Then, perform slot-level reordering and calculate the slot matching score:

[0108] in, Indicates the slot type to be compared, taken from the set. These correspond to the subject, predicate, object, quantity, time, place, relationship, and modification and scope constraints, respectively. Indicates slot Weighting coefficients; Representing a structured claim Middle slot The possible values ​​of ; Indicate candidate evidence Corresponding slot The possible values ​​of ; This represents the slot matching function, used to measure the relationship between structured claims and candidate evidence in slots. The degree of consistency.

[0109] 6) The final re-ranking score is obtained by combining the source credibility:

[0110] in, Indicates the initial search score; Indicates the slot matching score; Indicate candidate evidence Source credibility score; These represent the weight coefficients of the initial retrieval score, slot matching score, and source credibility score, respectively, preferably satisfying the following conditions: .

[0111] Step S5: Perform multi-dimensional consistency verification between the structured claims and each candidate piece of evidence to obtain the authenticity score of each claim.

[0112] Specifically, in step S5, the multi-dimensional consistency verification includes semantic consistency verification, numerical consistency verification, temporal consistency verification, relational consistency verification, and scope consistency verification; the single claim authenticity score is based on the sum of the products of each consistency verification result and the corresponding weight; for numerical consistency, when the quantity in the structured claim is inconsistent with the quantity unit in each candidate evidence, a uniform unit conversion is used to calculate the proximity.

[0113] 1) Input and output representation methods: Input is structured proposition and candidate evidence The output is the verification score of the structured claim relative to the candidate evidence. .

[0114] 2) The verification score expression is:

[0115] in: For semantic consistency; For numerical consistency; For time consistency; For relationship consistency; For the sake of range consistency; These represent the weighting coefficients of semantic consistency, numerical consistency, temporal consistency, relational consistency, and range consistency in the overall verification score, respectively, preferably satisfying the following conditions. .

[0116] 3) Semantic consistency function expression:

[0117] in, This indicates that the natural language inference model supports the normalization claim. With candidate evidence The result of judging the semantic relationship between them; This indicates the probability that candidate evidence supports the normative claim; This indicates the probability that candidate evidence refutes the normative claim; The semantic consistency score represents the probability of support and the probability of rebuttal; the higher the probability of support and the lower the probability of rebuttal, the better. The larger.

[0118] 4) Numerical consistency expression: Let the number of structured claims be . The number of candidate pieces of evidence is If the units are different, perform a uniform conversion first:

[0119] in, This indicates the number of slots in a structured claim. Indicates the numerical value in the claim. Indicates the unit of quantity in a claim; This indicates the number of slots in the candidate evidence. Indicates the numerical value in the evidence. Indicates the unit of quantity in the evidence; Indicates the standardized unit after unified conversion; Indicates that the unit is numerical value Convert to standard units The following values; This represents the standardized numerical value of quantities in a structured claim after unit conversion. This represents the standardized numerical value of the quantity of candidate evidence after unit conversion.

[0120] Define numerical consistency:

[0121] in, This represents a very small positive number that prevents the denominator from being zero; This represents the absolute value of the numerical difference between the structured claim and the candidate evidence; This represents the normalized denominator, used to eliminate the influence of different orders of magnitude. This represents the numerical consistency score; the smaller the numerical difference, the better. The closer it is to 1.

[0122] If it is an interval expression, it is expanded into an interval proximity score.

[0123] 5) Time consistency expression: Let the time interval for structured claims be:

[0124] The time frame for candidate evidence is:

[0125] Define intersection-union ratio:

[0126] Define time consistency:

[0127] in, The time frame representing the structured claims. Indicates the start time of the structured claim time interval. Indicates the end time; Indicates the time interval of candidate evidence. Indicates the start time of the time interval for candidate evidence. Indicates the end time; The intersection-over-union ratio (IoU) measures the degree of overlap between structured claims and candidate evidence over time. This indicates the granularity of the structured claims over time. Indicates the granularity of the time frame for candidate evidence; This represents a time granularity consistency function, used to measure the degree of matching between two things at the time granularity level, such as year, month, day, and interval. The temporal consistency function measures the degree of consistency between claims and evidence in terms of time sequence, such as "before," "after," "simultaneously," and "after." These represent the weight coefficients for the time interval overlap term, the granularity consistency term, and the direction consistency term, respectively, preferably satisfying the following conditions: ; 6) Relational consistency expression: The set of relation types is:

[0128] in, Represents a set of relation types; This indicates a greater than relation. This indicates a less than relation (Less Than). Indicates equality (Equal); Indicates an increase relationship. Indicates a decrease in the relationship (Decrease); This indicates the first occurrence of the relationship or the initial sexual relationship; This indicates a relationship that has continued from a certain point in time to the present. This indicates the relationship between elements located within a certain interval. It represents the interval relationship from one point in time to another.

[0129] If the structured claim relationship is The candidate evidence relationship is ,but:

[0130] in, The relation consistency score measures the degree of consistency between the relation types in the structured claims and the relation types in the candidate evidence; a value of 1 is taken when both are completely consistent, otherwise it is determined based on the similarity function between relation types. Calculate the partial match score.

[0131] 7) Range Consistency Expression Define the scope of structured claims Scope of candidate evidence :

[0132] in, This indicates the modification and scope constraint information in a structured claim; This indicates the modification and scope constraint information in the candidate evidence; The intersection-union ratio (IUU) indicates the degree of overlap between the scope of claims and the scope of evidence, used to measure the extent of overlap in their scope representations. The scope inclusion function is used to determine whether the scope of candidate evidence can cover the scope constraints expressed by the structured claim. The scope consistency score is used to identify overgeneralization or scope mismatch issues where local facts are presented as whole facts, or whole facts are reduced to local facts.

[0133] This is used to determine whether there is an overgeneralization where "partial facts are presented as overall facts".

[0134] Multi-evidence fusion expression: For top- The evidence is calculated separately:

[0135] Then normalize the scores by reordering:

[0136] The overall truthfulness score of the claim is obtained as follows:

[0137] Claim-level labels can be obtained based on the threshold:

[0138] Step S6: Combine the single-claim authenticity scores of multiple structured claims to obtain the total verification score.

[0139] Specifically, in step S6, the multiple single claim authenticity scores are weighted and fused to obtain the total verification score; the weights corresponding to the weights are related to the importance of the structured claims.

[0140] Among these, relevant factors include whether it is a fact in the headline, whether it is at the core of the main communication narrative, whether it includes high-risk information in terms of quantity / time, and whether it appears repeatedly in the entire text.

[0141] Step S5 outputs the truth score of a single structured claim supported by multiple pieces of evidence. Step S6 will further aggregate multiple structured claims in a document / post to obtain the final verification score for the entire content.

[0142] Therefore, the relationship between the variables is:

[0143] in: Indicates the first One core proposition; This indicates the first [unclear] corresponding to the claim. One candidate piece of evidence; Indicates the first The overall truthfulness score of each claim; This indicates the final detection tag for the entire content.

[0144] Multiple claims converge as follows: Extract from a piece of content One structured proposition:

[0145] Each claim receives a validity score in step S5:

[0146] Combined with the weight of the importance of the claims The aggregated verification score is as follows:

[0147] in The following factors can determine whether it is a fact in the headline; whether it is at the core of the main communication narrative; whether it contains high-risk information such as quantity / time; and whether it appears repeatedly in the entire text.

[0148] For example:

[0149] in: : asserting significance; Does it contain quantity information? : Does it include time information?

[0150] , and These represent the weighting coefficients for the salience feature, quantitative information feature, and temporal information feature, respectively, used to adjust the contribution of each feature to the candidate claim importance score.

[0151] Step S7: Obtain the news location and IP address, adjust the verification score according to the matching degree, and output the verification result.

[0152] Specifically, in step S7, the total verification score is corrected based on the matching degree to determine the authenticity of the input text, and the verification result is output; the verification result includes Real and False. ), partially misleading ( When the matching degree θ is related to... V(x) The product is less than the first preset matching threshold When the matching degree θ is 0, the output result is false; when the matching degree θ is 0, the output result is false. V(x) The product is greater than the first preset matching threshold. And less than the second preset matching threshold When the matching degree θ is 0, the judgment result is partially misleading; when the matching degree θ is 0, the judgment result is partially misleading. V(x) The product is greater than the second preset matching threshold. When the time is right, the output result is true.

[0153] Output expression:

[0154] Simultaneously output: Key Issues and Claims Set ; Supporting / Conflicting Evidence Set ; Error field collection .

[0155] in, It is a quantity field used to indicate inconsistencies in numerical values ​​or proportions; It is a time field used to represent a point in time, a time interval, or an inconsistency in time granularity; This is a relational field used to indicate inconsistencies in comparison relationships, increment / decrease relationships, or time-series relationships. It is a range field used to indicate range constraints, modifier restrictions, or inconsistencies between whole-part expressions.

[0156] In summary, compared with the prior art, the method of this embodiment has at least the following advantages and beneficial effects: 1) By employing preprocessing, candidate claim set, and claim normalization, noise processing and fact-finding are moved forward from fake news detection, thus improving the speed of subsequent information processing; 2) The structured approach to information retrieval expands the scope of information retrieval and improves the accuracy and efficiency of information judgment during weighted fusion. 3) By matching and comparing IP addresses with news locations, the mathematical model algorithm and network server positioning were combined, further improving the accuracy of the judgment.

[0157] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0158] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0159] Please see Figure 2 , Figure 2 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 201 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 202 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 202 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 202 and is called and executed by the processor 201 using the methods described in the embodiments of this application. Input / output interface 203 is used to implement information input and output; The communication interface 204 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 205 transmits information between various components of the device (e.g., processor 201, memory 202, input / output interface 203, and communication interface 204); The processor 201, memory 202, input / output interface 203 and communication interface 204 are connected to each other within the device via bus 205.

[0160] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0161] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0162] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0163] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0164] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0165] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0166] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0169] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0170] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0171] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0172] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0173] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0174] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for detecting fake news, characterized in that, The method includes the following steps: S1: Preprocess and segment the original input text to obtain a set of sentences. Perform verifiability scoring on each sentence in the set of sentences and select the sentences with the highest scores as a set of candidate claims. S2: Normalize each sentence in the candidate claim set, remove its non-factual information, and transform it into a single declarative expression to obtain a normalized claim; S3: Construct each of the above-described normalized claims into a structured claim that includes subject, predicate, object, quantity, time, place, relation, and modifiers and scope constraints; S4: Perform a retrieval query based on at least one key field in the structured claim to obtain a candidate evidence set; S5: Perform multi-dimensional consistency verification between the structured claim and each candidate evidence in the candidate evidence set to obtain a single claim authenticity score. The multi-dimensional consistency verification includes at least one of numerical consistency verification and temporal consistency verification. S6: Combine the authenticity scores of multiple single claims corresponding to multiple structured claims obtained from multiple normalized claims to obtain the total verification score; S7: Obtain the news location extracted from the original input text and the IP address of the news webpage corresponding to the original input text. Based on the degree of matching between the geographical location of the news location and the IP address, correct the total verification score and output the verification result.

2. The method according to claim 1, characterized in that, In step S1: The preprocessing includes at least one of the following: duplicate segment removal, metadata cleaning, OCR / layout repair, and emotional expression filtering of the input text; The verifiability score is based on at least one of the factual score, verifiability score, and main information focus score for each sentence.

3. The method according to claim 1, characterized in that, In step S2, a generative model trained based on joint loss is used for the normalization process. The joint loss includes sequence generation loss, assertion constraint loss, length constraint loss and fact-keeping loss. Furthermore, constraint decoding is used during the generation process to filter social media spam, control the length, and detect sentence templates in the output text.

4. The method according to claim 1, characterized in that, The structured claim Cs mentioned in step S3 is represented as follows: Cs= (s,p,o,q,t,l,r,m), in s As the main body p For predicate, o For the object, q For quantity or proportion, t For time, l For location, r For relationships, m For modification and range constraints.

5. The method according to claim 1, characterized in that, In step S4: Based on the key fields in the structured claim, candidate evidence is retrieved from news databases, open web pages, fact-checking databases, or local databases using a dual-channel retrieval method; and the candidate evidence set is determined based on semantic similarity, slot matching degree, and slot ranking score; the slot matching degree includes a measure of the consistency of at least one slot among subject, predicate, object, quantity, time, location, relationship, modification, and scope constraints.

6. The method according to claim 1, characterized in that, In step S5: The multi-dimensional consistency verification also includes at least one of semantic consistency verification, relational consistency verification and scope consistency verification; The authenticity score of a single claim is based on the sum of the product of the consistency verification results of each dimension and the corresponding weights; For the numerical consistency verification, when the units of quantity in the structured claim are inconsistent with those in the candidate evidence, a uniform unit conversion is used to calculate the proximity.

7. The method according to claim 1, characterized in that, In step S6: the authenticity scores of multiple individual claims are weighted and fused to obtain the total verification score; wherein, the weight corresponding to each authenticity score of an individual claim is related to the importance of the corresponding structured claim, and the importance is determined based on at least one of the following factors: whether it is a headline fact, whether it is located at the core of the main communication narrative, whether it contains quantity or time information, and whether it appears repeatedly in the original input text.

8. The method according to claim 1, characterized in that, In step S7: After correcting the total verification score according to the matching degree, the authenticity of the input text is determined, and a verification result including "true", "false" or "partially misleading" is output.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images