Method for automatic recognition of false information

Abstract

This application provides an automatic method for identifying false information, belonging to the field of information identification technology. The method includes: acquiring a multimodal fusion feature set corresponding to multiple data sources; constructing a city-wide governance knowledge graph based on multiple entities corresponding to different entity types in city-wide governance government data and the relationships between these entities; performing an identification operation on each multimodal fusion feature in the multimodal fusion feature set: matching the geographical location information in the multimodal fusion feature with the coordinates of events contained in each node of the city-wide governance knowledge graph to obtain target nodes that match the multimodal fusion feature; weighting the matching degree between the multimodal fusion feature and the events corresponding to each target node based on the feature weights corresponding to each target node to obtain a comprehensive risk score; and determining that the information corresponding to the multimodal fusion feature is false information when the comprehensive risk score is greater than a preset risk threshold. This application can improve the efficiency of false information identification.

Description

Technical Field

[0001] This application belongs to the field of information identification technology, and more specifically, relates to a method for automatic identification of false information. Background Technology

[0002] In the digital transformation of urban social governance, the automated identification of false information has become a core requirement for maintaining public safety and social stability. Current false information, such as online fraud, exhibits multimodal, cross-platform, and high-frequency dissemination characteristics. Existing technologies lack a real-time integration mechanism for government data and other internet data, making it difficult to detect cross-domain false information and reducing the efficiency of false information identification. Summary of the Invention

[0003] The purpose of this application is to provide an automatic method for identifying false information, which identifies the multi-source fused data based on government data, thereby improving the efficiency of false information identification.

[0004] A first aspect of this application provides a method for automatically identifying false information, including:

[0005] Obtain the multimodal fusion feature set corresponding to multiple data sources; the multimodal fusion feature set is obtained by mapping text features and visual features from multiple data sources to a multidimensional joint vector space using a visual-text joint embedding model;

[0006] A knowledge graph of urban governance is constructed based on multiple entities corresponding to different entity types in the municipal governance government data and the relationships between these entities; each entity corresponds one-to-one with each node in the knowledge graph of urban governance, and each node in the knowledge graph of urban governance contains the event corresponding to that node;

[0007] For each multimodal fusion feature in the multimodal fusion feature set, perform the following recognition operation:

[0008] The geographic location information in the multimodal fusion feature is matched with the coordinates of events contained in each node of the urban governance knowledge graph to obtain the target node that matches the multimodal fusion feature.

[0009] The matching degree between the multimodal fusion feature and the event corresponding to each target node is weighted based on the feature weights corresponding to each target node to obtain the comprehensive risk score of the multimodal fusion feature;

[0010] When the comprehensive risk score is greater than the preset risk threshold, the information corresponding to the multimodal fusion feature is determined to be false information.

[0011] In one embodiment of this application, after performing a recognition operation on each multimodal fusion feature in the multimodal fusion feature set, the automatic identification method for false information further includes:

[0012] All false information corresponding to the multimodal fusion feature set is grouped; each group of false information corresponds to the same event.

[0013] A network for the spread of misinformation is constructed based on each set of misinformation.

[0014] In response to a request to trace the source of false information corresponding to a target event, the propagation network of false information corresponding to the target event is traversed based on a depth-first search algorithm to obtain the propagation path of the target event.

[0015] In one embodiment of this application, the method for automatically identifying false information further includes:

[0016] The spatial location and data source of the propagation nodes in the propagation path are divided to determine the propagation area and platform of the false information corresponding to the target event.

[0017] The dissemination platform based on data sources divides communities into multiple dissemination communities for the spread of false information.

[0018] In one embodiment of this application, the text features in the multiple data sources are obtained by processing the text data in the multiple data sources using a pre-trained language model;

[0019] After determining that the information corresponding to the multimodal fusion features is false information, the automatic false information identification method also includes:

[0020] False information is stored in a database containing various types of false information, and it is determined whether the type of false information belongs to the type already included in the database.

[0021] If the type of false information does not belong to the types already included in the information base, the type of false information is marked so that the parameters of the pre-trained language model can be updated when the number of marked false information reaches a preset threshold.

[0022] In one embodiment of this application, the types of false information are labeled to update the parameters of the pre-trained language model when the number of labeled false information reaches a preset threshold, including:

[0023] Determine the confidence level corresponding to false information;

[0024] If the confidence level is less than the first value, the type of false information is manually labeled, all false information in the information database, including manually labeled false information, is obtained, and the parameters of the pre-trained language model are updated based on all false information.

[0025] If the confidence level is greater than the second value, the type of false information is automatically labeled using a predetermined self-training method, all false information in the information database, including automatically labeled false information, is obtained, and the parameters of the pre-trained language model are updated based on all false information; if the first value is less than the second value.

[0026] In one embodiment of this application, updating the parameters of a pre-trained language model based on all false information includes:

[0027] All false information in the information database is used as training samples. The parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples to obtain the pre-trained language model with updated parameters.

[0028] In one embodiment of this application, the method for automatically identifying false information further includes:

[0029] The information database is expanded by using generative adversarial networks to obtain an updated information database;

[0030] Using all the fake information in the information database as training samples, the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples, including:

[0031] All the fake information in the updated information database is used as training samples, and the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples.

[0032] A second aspect of this application provides an automatic identification device for false information, comprising:

[0033] The knowledge graph construction unit is used to construct a knowledge graph of urban governance based on multiple entities corresponding to different entity types in the municipal governance government data and the relationships between these entities. Each entity corresponds one-to-one with each node in the knowledge graph of urban governance, and each node in the knowledge graph of urban governance contains the event corresponding to that node.

[0034] The recognition unit is used to perform the following recognition operation on each multimodal fusion feature in the multimodal fusion feature set:

[0035] The geographic location information in the multimodal fusion feature is matched with the coordinates of events contained in each node of the urban governance knowledge graph to obtain the target node that matches the multimodal fusion feature.

[0036] The matching degree between the multimodal fusion feature and the event corresponding to each target node is weighted based on the feature weights corresponding to each target node to obtain the comprehensive risk score of the multimodal fusion feature;

[0037] When the comprehensive risk score is greater than the preset risk threshold, the information corresponding to the multimodal fusion feature is determined to be false information.

[0038] In one embodiment of this application, the automatic identification device for false information further includes a propagation path calculation unit;

[0039] The propagation path calculation unit is used for:

[0040] All false information corresponding to the multimodal fusion feature set is grouped; each group of false information corresponds to the same event.

[0041] A network for the spread of misinformation is constructed based on each set of misinformation.

[0042] In response to a request to trace the source of false information corresponding to a target event, the propagation network of false information corresponding to the target event is traversed based on a depth-first search algorithm to obtain the propagation path of the target event.

[0043] In one embodiment of this application, the propagation path calculation unit is further configured to:

[0044] The spatial location and data source of the propagation nodes in the propagation path are divided to determine the propagation area and platform of the false information corresponding to the target event.

[0045] The dissemination platform based on data sources divides communities into multiple dissemination communities for the spread of false information.

[0046] In one embodiment of this application, the text features in the multiple data sources are obtained by processing the text data in the multiple data sources using a pre-trained language model;

[0047] The automatic identification device for false information also includes a parameter update unit. After determining that the information corresponding to the multimodal fusion feature is false information, the parameter update unit is used to:

[0048] False information is stored in a database containing various types of false information, and it is determined whether the type of false information belongs to the type already included in the database.

[0049] If the type of false information does not belong to the types already included in the information base, the type of false information is marked so that the parameters of the pre-trained language model can be updated when the number of marked false information reaches a preset threshold.

[0050] In one embodiment of this application, the parameter update unit is specifically used for:

[0051] Determine the confidence level corresponding to false information;

[0052] If the confidence level is less than the first value, the type of false information is manually labeled, all false information in the information database, including manually labeled false information, is obtained, and the parameters of the pre-trained language model are updated based on all false information.

[0053] If the confidence level is greater than the second value, the type of false information is automatically labeled using a predetermined self-training method, all false information in the information database, including automatically labeled false information, is obtained, and the parameters of the pre-trained language model are updated based on all false information; if the first value is less than the second value.

[0054] In one embodiment of this application, the parameter update unit is specifically used for:

[0055] All false information in the information database is used as training samples. The parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples to obtain the pre-trained language model with updated parameters.

[0056] In one embodiment of this application, the automatic identification device for false information further includes a data expansion unit;

[0057] The data expansion unit is used for:

[0058] The information database is expanded by using generative adversarial networks to obtain an updated information database;

[0059] The parameter update unit is specifically used for:

[0060] All the fake information in the updated information database is used as training samples, and the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples.

[0061] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described automatic identification method for false information.

[0062] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for automatically identifying false information.

[0063] The beneficial effects of the automatic identification method for false information provided in this application are as follows:

[0064] This application first obtains a multimodal fusion feature set through a visual-text joint embedding model, deeply fusing multiple data sources to overcome the lack of a real-time fusion mechanism in existing technologies, thus aiding in the detection of cross-domain misinformation. Secondly, by constructing a city-wide governance knowledge graph based on government data, and utilizing the correlations between events at each node in the knowledge graph, misinformation is identified for each multimodal fusion feature, enabling the detection of cross-domain misinformation and significantly improving the efficiency of misinformation identification. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A flowchart illustrating an embodiment of the automatic identification method for false information provided in this application;

[0067] Figure 2 A flowchart illustrating an embodiment of the automatic identification method for false information provided in this application;

[0068] Figure 3 A structural block diagram of an automatic false information identification device provided in an embodiment of this application;

[0069] Figure 4 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0072] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the automatic identification method for false information provided in this application. The method can be executed by an electronic device and may include steps S101-S103.

[0073] S101: Obtain the multimodal fusion feature set corresponding to multiple data sources.

[0074] The multimodal fusion feature set is obtained by mapping text features and visual features from multiple data sources to a multidimensional joint vector space using a visual-text joint embedding model.

[0075] The automatic identification method for false information in this embodiment can be applied to the field of urban social governance. In this field, to accurately identify false information, during the data acquisition phase, multiple data sources of various types from different platforms can be periodically acquired through a government data gateway and a distributed framework. For example, the government data gateway can connect to multiple platforms such as the Health Commission's data system and the market supervision and management database system to transmit data from these platforms to electronic devices for analysis and processing. Since different platforms have different interfaces—for example, the types of interfaces include a and b, etc.—the Health Commission's data system uses... a The interface connection to the market supervision and management database system uses the 'b' interface. Therefore, the government data gateway adopts an adapter pattern to design a standardized interface, facilitating the smooth collection of data from multiple platforms. In this embodiment, the government data gateway can also be combined with an internet information crawler engine, achieving parallel collection of 100,000 data entries per second through the crawler engine's message queue. After data collection, structured data needs to be parsed and unstructured data needs to be identified to obtain initial data. The initial data is then cleaned and standardized to obtain data for feature extraction. This avoids processing invalid data and improves the accuracy of identifying false information.

[0076] In this embodiment, the multimodal fusion feature set is obtained by mapping text features and visual features from multiple data sources to a multidimensional joint vector space using a visual-text joint embedding model. The visual-text joint embedding model is a cross-modal deep learning model that can map text features and visual features to the same high-dimensional vector space, making semantically related visual features and text features closer in space, thereby achieving cross-modal understanding, retrieval, and matching.

[0077] Text features can be obtained by processing text data from multiple data sources using a pre-trained language model. The pre-trained language model can be the BERT-Prompt model, which combines the Prompt function with the BERT model. This model can guide the model to generate expected outputs during task execution by designing specific prompt templates, without significantly modifying the original parameters of the BERT model. For example, in scenarios involving the spread of misinformation, inserting a Prompt template containing "Official Announcement [X]" can improve semantic discrimination. Another example is the information about "a surge in vegetable prices in a certain area." The Prompt template can be linked to local price monitoring data and logistics data; based on the data in the Prompt template, it can be determined whether the information about "a surge in vegetable prices in a certain area" is true.

[0078] The BERT-Prompt model can significantly improve the semantic distinction between different texts in text data. This embodiment uses a visual encoder to transform image data / video frames into a series of visual vectors, i.e., visual features, that represent visual information.

[0079] Existing technologies primarily rely on single-modal analysis, lacking real-time semantic understanding of multi-source data, resulting in low accuracy in identifying misinformation. This embodiment utilizes a visual-text joint embedding model to map textual and visual features from multiple data sources to a multi-dimensional joint vector space, obtaining a multimodal fusion feature set. This breaks down the barriers between different modalities, achieving deep fusion. For example, visual features related to "supermarket panic buying video" are aligned with the textual feature "supply shortage" to detect the consistency between the scene and the text. This visual-text joint embedding model can be either a CPIL model or an ALBEF model. The above method makes the information representation of the multimodal fusion feature set more comprehensive and richer, fully exploring the hidden correlations in multiple data sources, which is beneficial to improving the accuracy of misinformation identification.

[0080] S102: Construct a knowledge graph of urban governance based on multiple entities corresponding to different entity types in urban governance government data and the relationships between these entities.

[0081] Each entity corresponds one-to-one with each node in the city governance knowledge graph, and each node in the city governance knowledge graph contains the events corresponding to that node.

[0082] In this embodiment, urban governance refers to the process of coordinating, managing, and handling public services, emergency management, and social governance within the scope of the city's administrative area, by integrating various governance resources such as government data, departmental resources, and social resources. The urban governance domain includes government data, environmental data, and public transportation data. Government data refers to various types of data generated or acquired by government departments in the course of fulfilling their duties, including entity data such as policy documents, geographic information, event records, and institutional information, as well as the relationships between these entities. Each entity is treated as a node in the urban governance knowledge graph, and the relationships between entities are treated as edges, thus constructing the urban governance knowledge graph. Each node in the urban governance knowledge graph contains the corresponding event. For example, if the "Regulations on the Management of Garbage Classification in Residential Communities of City A" is a policy document node, then "Applicable Area" and "BB Community has not implemented garbage classification management" can be considered the events corresponding to that node.

[0083] S103: Perform a recognition operation on each multimodal fusion feature in the multimodal fusion feature set.

[0084] refer to Figure 2 The identification operation includes steps S1031-S1033.

[0085] S1031: Match the geographic location information in the multimodal fusion feature with the coordinates of events contained in each node of the urban governance knowledge graph to obtain the target node that matches the multimodal fusion feature.

[0086] In this embodiment, the multimodal fusion feature includes geographic location information. By matching the geographic location information in the multimodal fusion feature with the coordinates of events contained in each node of the urban governance knowledge graph, the association between the multimodal fusion feature and real governance events can be established, providing a basis for judging false information.

[0087] The geographic location information in the multimodal fusion features is matched with the coordinates of events contained in each node of the urban governance knowledge graph to obtain the target nodes that match the multimodal fusion features, including:

[0088] Geographic location information is extracted from multimodal fusion features to construct dual-source geographic features based on textual and visual geographic features. The textual geographic features include a four-level hierarchical encoding of administrative regions, while the visual geographic features include latitude and longitude coordinates and scene feature vectors of geographic locations from the multimodal fusion features.

[0089] Extract the administrative region code of the event, the latitude and longitude information of the location where the event occurred, and the scene feature vector of each node;

[0090] By filtering the administrative region codes of each node that match the fourth-level hierarchical codes in the text's geographic features, multiple first nodes are obtained.

[0091] Based on multiple first nodes, calculate the first similarity between the latitude and longitude of each first node and the latitude and longitude in the multimodal fusion features, and calculate the second similarity between the scene feature vector of each first node and the scene feature vector in the multimodal fusion features.

[0092] The target node is obtained by filtering from multiple first nodes based on the first similarity and the second similarity.

[0093] In this embodiment, if the text feature in the multimodal fusion feature is "A large area of ​​collapse has occurred in the municipal park of AAA Street, AAA District, City A, and it has been fenced off," and the visual feature is an image, the administrative code in the text feature is extracted as C1, the latitude and longitude coordinates in the visual feature are extracted as D1, and the scene feature vector is extracted as E1. The geographical features of each node event in the knowledge graph are extracted. For example, the administrative region code, latitude and longitude information of the occurrence location, and scene feature vector of node 1 are (C2, D2, E2), the administrative region code, latitude and longitude information of the occurrence location, and scene feature vector of node 2 are (C3, D3, E3), and the administrative region code, latitude and longitude information of the occurrence location, and scene feature vector of node 3 are (C4, E4, D4). If node 1 and node 2 are matched by matching the administrative region code, then the first similarity between node 1 and node 2 and the latitude and longitude in the multimodal fusion feature is calculated, and the second similarity between node 1 and node 2 and the scene feature vector in the multimodal fusion feature is calculated. The first node with a first similarity greater than 0.8 and a second similarity greater than 0.6 is taken as the target node, and the first node with a first similarity less than 0.5 and a second similarity less than 0.4 is excluded from the target node list.

[0094] S1032: Based on the feature weights corresponding to each target node, the matching degree between the multimodal fusion feature and the event corresponding to each target node is calculated by weighting, and the comprehensive risk score of the multimodal fusion feature is obtained.

[0095] In this embodiment, assuming that two target nodes are obtained after matching, the feature weight of target node 1 is 0.7 and the matching degree with multimodal information is 0.9, and the feature weight of target node 2 is 0.3 and the matching degree is 0.5, then the comprehensive risk score is: 0.7×0.9+0.3×0.5=0.78.

[0096] S1033: When the comprehensive risk score is greater than the preset risk threshold, the information corresponding to the multimodal fusion feature is determined to be false information.

[0097] In this embodiment, when the comprehensive risk score exceeds a preset risk threshold, it indicates that the information corresponding to the multimodal fusion feature has a high degree of falsehood and should be identified as false information. The preset risk threshold can be determined based on historical experience or dynamically adjusted based on recent policy changes.

[0098] As can be seen from the above, the embodiments of this application first obtain a multimodal fusion feature set through a visual-text joint embedding model, deeply fusing multiple data sources, thus overcoming the deficiency of existing technologies in lacking a real-time fusion mechanism and helping to detect cross-domain false information. Secondly, by constructing a knowledge graph of urban governance based on government data, and utilizing the correlation of events at each node in the knowledge graph to identify false information for each multimodal fusion feature, cross-domain false information can be detected, significantly improving the efficiency of false information identification.

[0099] In one embodiment of this application, after performing a recognition operation on each multimodal fusion feature in the multimodal fusion feature set, the automatic identification method for false information further includes:

[0100] All false information corresponding to the multimodal fusion feature set is grouped; each group of false information corresponds to the same event.

[0101] A network for the spread of misinformation is constructed based on each set of misinformation.

[0102] In response to a request to trace the source of false information corresponding to a target event, the propagation network of false information corresponding to the target event is traversed based on a depth-first search algorithm to obtain the propagation path of the target event.

[0103] In this embodiment, the multimodal features in the multimodal fusion feature set are determined to be false information using the method described in this application embodiment. The false information is then stored in an information database. Multiple pieces of false information corresponding to the same event are grouped together, and a false information propagation network is constructed based on each group of false information. The nodes of the false information propagation network represent the propagation participants or information carriers, and the edges represent the propagation relationships of the information, such as publishing, forwarding, or reposting, which can intuitively present the propagation chain of false information for each event.

[0104] In response to a received request to trace the source of misinformation related to a target event, the propagation path of the target event can be determined based on the misinformation propagation network. The target event is a specific event for which the propagation path of misinformation needs to be traced, and the corresponding request to trace the source of misinformation is initiated by municipal governance staff or other relevant personnel.

[0105] In this embodiment, the propagation path of the target event can be obtained by traversing the false information propagation network corresponding to the target event using a depth-first search algorithm. The depth-first search algorithm is a graph traversal algorithm that starts from the starting node, prioritizes exploring all branch paths of the current node to the endpoint, then backtracks to explore untraversed paths until the entire network is traversed. This allows for the layer-by-layer mining of the relationships between propagation nodes in the false information propagation network.

[0106] In this embodiment, after receiving a request to trace the source of false information corresponding to a target event, the electronic device first matches the corresponding false information propagation network and locates the initial publishing node in the network as the starting point for traversal. Then, starting from the starting point, it prioritizes traversing a propagation path to the terminal node, and then backtracks to explore untraversed branches until all propagation links are covered. Finally, the traversal results are sorted by propagation time to generate a propagation path from the source to the intermediate nodes to the terminal nodes, and this path is fed back to the requester. This embodiment of the application, through false information tracing, can help relevant government departments actively combat the spread of false information and maintain good social order.

[0107] In one embodiment of this application, the method for automatically identifying false information further includes:

[0108] The spatial location and data source of the propagation nodes in the propagation path are divided to determine the propagation area and platform of the false information corresponding to the target event.

[0109] The dissemination platform based on data sources divides communities into multiple dissemination communities for the spread of false information.

[0110] In this embodiment, "propagation nodes" refer to various entities or carriers involved in the transmission of information along the path of false information dissemination, including user accounts that publish or forward false information, dissemination platforms that carry information, and terminal devices that publish information. Spatial location refers to the geographical location of administrative divisions in urban governance. Dividing propagation nodes along the path into spatial locations and data sources can be understood as extracting spatial location information and data source information from each propagation node. Spatial location information can be obtained through the user account's real-name authentication address, the geographical coordinates corresponding to the login device's IP address, or regional keywords mentioned in the information content; data source information can be directly determined based on the platform identifier to which the propagation node belongs. Then, the extracted spatial location information is matched with the geographical location of administrative divisions in urban governance, grouping propagation nodes within the same geographical area into one category. All involved geographical areas are then summarized as the propagation area of ​​the false information. Simultaneously, propagation nodes are classified according to the platform type to which they belong, and the number of propagation nodes and the frequency of information dissemination within each platform are statistically analyzed. Platforms through which the false information corresponding to the target event flows are then identified as propagation platforms.

[0111] In this embodiment, the community division based on the data source propagation platform yields multiple propagation communities for the spread of false information. This can be understood as follows: assuming there are two propagation platforms for the target event, a fraud group uses n different social media accounts (with different IP addresses) on one propagation platform to publish false investment information at different times, and uses m different social media accounts on the other propagation platform to publish false investment information at different times, thus forming two propagation communities.

[0112] In this embodiment, by identifying the community in which the target event spreads, it is possible to avoid missing any links in the chain of dissemination of false information, accurately locate all false information published by the entity corresponding to the event, and impose sanctions on that entity.

[0113] In one embodiment of this application, the method for automatically identifying false information further includes:

[0114] The propagation nodes in the propagation path are spatially divided and the data sources are divided to determine the propagation area and platform of the false information corresponding to the target event; based on the propagation area and platform, a false information propagation impact assessment model is constructed.

[0115] The strategy for governing misinformation is selected based on the output of the model for assessing the impact of misinformation dissemination.

[0116] In this embodiment, the output of the false information dissemination influence assessment model is the dissemination influence index I, and the formula for calculating the dissemination influence index I is as follows:

[0117]

[0118] Where N represents the total number of entities spreading false information, cov(P) represents the spatial coverage of the spread area P, with a value range of [0,1], Q represents the set of spread platforms, and ω q The weight coefficient of platform q is defined as follows: s q This indicates the speed at which false information spreads on platform q, measured in messages per hour. α, β, and γ represent weighting coefficients.

[0119] In this embodiment, when the dissemination influence index I is greater than the preset dissemination influence index, a governance strategy that processes the issue within a first time period is selected. When the dissemination influence index I is less than or equal to the preset dissemination influence index, a governance strategy that processes the issue within a time period greater than the first time period but less than the second time period is selected. This embodiment sets differentiated governance strategies based on the dissemination influence index I, which helps to shift governance decisions from experience-driven to data-driven, reducing subjective bias.

[0120] In one embodiment of this application, the text features in the multiple data sources are obtained by processing the text data in the multiple data sources using a pre-trained language model;

[0121] After determining that the information corresponding to the multimodal fusion features is false information, the automatic false information identification method also includes:

[0122] False information is stored in a database containing various types of false information, and it is determined whether the type of false information belongs to the type already included in the database.

[0123] If the type of false information does not belong to the types already included in the information base, the type of false information is marked so that the parameters of the pre-trained language model can be updated when the number of marked false information reaches a preset threshold.

[0124] In this embodiment, after determining that the information corresponding to the multimodal fusion feature is false information, it is also necessary to determine whether the type of false information is a new type of false information. If it is a new type of false information, then in order to improve the active learning ability of the pre-trained language model, the parameters of the model need to be updated.

[0125] Within a sampling period, all multimodal features identified as fraudulent information are stored in a database. During this period, the type of fraudulent information is compared with the types of fraudulent information stored in previous sampling periods. If no matching type is found, the fraudulent information is identified as a new type, marked, and stored in the database. If a matching type is found, the fraudulent information is not marked, but it is still stored in the database. When the number of marked fraudulent information reaches a preset threshold within a sampling period, the parameters of the pre-trained language model need to be updated to improve the accuracy of identifying new types of fraudulent information.

[0126] This embodiment improves the accuracy of identifying new types of misinformation by labeling them and updating the parameters of the pre-trained language model based on the labeled misinformation.

[0127] In one embodiment of this application, the types of false information are labeled to update the parameters of the pre-trained language model when the number of labeled false information reaches a preset threshold, including:

[0128] Determine the confidence level corresponding to false information;

[0129] If the confidence level is less than the first value, the type of false information is manually labeled, all false information in the information database, including manually labeled false information, is obtained, and the parameters of the pre-trained language model are updated based on all false information.

[0130] If the confidence level is greater than the second value, the type of false information is automatically labeled using a predetermined self-training method. All false information in the information database, including automatically labeled false information, is then obtained, and the parameters of the pre-trained language model are updated based on all false information.

[0131] In this embodiment, when a new type of misinformation emerges, such as AI deepfake videos, existing technologies typically rely on manual labeling of this new type of misinformation, followed by updating the pre-trained language model based on the labeled information. This process has a long response cycle and easily creates a governance "window of opportunity." This embodiment, however, can automatically select a labeling method based on the confidence level corresponding to the misinformation.

[0132] The confidence level is calculated as follows:

[0133] 1. Feature Extraction: Simultaneously acquire the text features (NLP extraction), propagation features (backend behavior data), and source features (account history data) of the fake information to be detected, forming a feature vector.

[0134] 2. Basic confidence calculation: The feature vector is processed using a pre-trained language model (such as a BERT model trained based on historical true and false labels) to output an initial confidence score (e.g., 0.70, meaning 70% chance of being false).

[0135] 3. External evidence correction: If the core facts of the information are found to have been officially refuted, the confidence level is corrected to 0.95; if no direct evidence is found, but the dissemination path shows "online promoter characteristics", the confidence level is corrected to 0.70.

[0136] 4. Result Output and Threshold Judgment: Set a confidence threshold (e.g., 0.8 is "highly suspected false", 0.5-0.8 is "medium suspected false"), and finally output "Confidence level of false information: 75% (medium suspected false)" and key judgment criteria (e.g., "the spread has the characteristics of online promoters, and no authoritative source is matched to support it").

[0137] If the confidence level is less than the first value, for example, 70%, an active learning strategy is adopted to manually label the types of misinformation. If the confidence level is greater than the second value, for example, 95%, a semi-supervised learning method is adopted to automatically label the types of misinformation using a predetermined self-training method.

[0138] In this embodiment, all false information in the information database, including manually labeled false information, can be obtained, and the parameters of the pre-trained language model can be updated based on all false information; alternatively, all false information in the information database, including automatically labeled false information, can be obtained, and the parameters of the pre-trained language model can be updated based on all false information; furthermore, all false information in the information database, including both manually labeled and automatically labeled false information, can be obtained, and the parameters of the pre-trained language model can be updated based on all false information.

[0139] Using the above methods, when the confidence level of false information is less than the first value, manual labeling can be performed to improve the accuracy of identifying the type of false information; when the confidence level of false information is greater than the second value, automatic labeling can be performed using a self-training method to save manual costs and achieve automatic model updates.

[0140] In one embodiment of this application, updating the pre-trained language model based on all false information includes:

[0141] All false information in the information database is used as training samples, and the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples.

[0142] In this embodiment, all the fake information in the information database is used as training samples. A Model-Agnostic Meta-Learning (MAML) algorithm can be employed to update the parameters of the last n fully connected layers of the pre-trained language model based on these training samples. Since the pre-trained language model can be based on the BERT model, which has a multi-layer Transformer encoder architecture, the first four layers of the BERT model primarily learn basic language rules, such as lexical semantics, grammatical structure, and simple contextual relationships. These are the general underlying knowledge for semantic recognition. Freezing the parameters of the first four layers prevents MAML from destroying this general underlying knowledge during meta-training, ensuring that the BERT model can stably transfer the language understanding capabilities learned in the pre-training stage to new tasks and reducing the forgetting of general knowledge due to excessive parameter adjustments. The last n layers of the BERT model (e.g., the last four layers) focus more on capturing task-specific features, such as deception patterns in fake information and domain-specific expressions. The core of MAML is to optimize and quickly adapt the initial parameters to new tasks through meta-training. By only updating the parameters of the last few fully connected layers, the BERT model can learn during the meta-training stage how to quickly adapt to the specific features of new types of misinformation by fine-tuning a small number of parameters. This approach can significantly improve the convergence speed of the BERT model in scenarios with few samples. For example, when facing new types of online fraud tactics, only a small number of labeled samples are needed to adjust the top-level weights to quickly achieve high recognition accuracy.

[0143] This embodiment significantly reduces the computational load during the meta-training and model fine-tuning stages by freezing most of the parameters of the BERT model and updating only the parameters of the last n layers. This reduces the demand for hardware resources and improves the training speed of the model, allowing for faster acquisition of the updated model. It is suitable for situations where the types of fake information are updated rapidly and the amount of fake information forwarded is large.

[0144] In this embodiment, the MAML algorithm is used to update the parameters of the last n fully connected layers of the pre-trained language model based on training samples. This can be achieved through a Parameter Server architecture, with a single batch training time of ≤30 minutes. After the pre-trained language model is updated, it can be validated on 5% of real-time traffic. Once the accuracy meets the target (e.g., an improvement of ≥2%), it can be deployed to the entire system. This method, by first validating on low-traffic traffic before applying it to high-traffic traffic, avoids incalculable consequences due to model testing failures and ensures the reliability of the updated pre-trained language model.

[0145] In one embodiment of this application, the method for automatically identifying false information further includes:

[0146] The information database is expanded by using generative adversarial networks to obtain an updated information database;

[0147] Using all the fake information in the information database as training samples, the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples, including:

[0148] All the fake information in the updated information database is used as training samples, and the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples.

[0149] In this embodiment, although the MAML algorithm can update the parameters of the last n fully connected layers of the pre-trained language model based on training samples in scenarios with few samples, the more samples there are, the better the training effect of the model will be. Therefore, this embodiment expands the false information in the information base based on generative adversarial networks to obtain more types of false information to update the information base and train the pre-trained language model.

[0150] Generative Adversarial Networks (GANs) consist of a generator and a discriminator. The generator, based on the StyleGAN2 architecture, can generate new types of fake information such as forged documents and synthetic videos. The discriminator employs a multimodal fusion engine to distinguish between real and generated samples. By alternately optimizing the generator and discriminator, GANs can improve the robustness of pre-trained language models to unknown fake information. The training cycle of GANs can be preset, for example, 100 rounds.

[0151] In this embodiment, the updated pre-trained language model can be tested based on publicly collected test samples, which can be generated based on a DeepFake generator.

[0152] This embodiment expands the information database with false information based on generative adversarial networks, and uses the expanded false information to train the pre-trained language model, which can significantly improve the model's recognition rate of unknown false information.

[0153] Corresponding to the automatic identification method for false information in the above embodiments, Figure 3 This is a structural block diagram of an automatic false information identification device provided according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 3 The automatic identification device 20 for false information includes: a data acquisition unit 21, a knowledge graph construction unit 22, and an identification unit 23.

[0154] Among them, the data acquisition unit 21 is used to acquire the multimodal fusion feature set corresponding to multiple data sources; the multimodal fusion feature set is obtained by mapping the text features and visual features in multiple data sources to a multidimensional joint vector space using a visual-text joint embedding model;

[0155] The knowledge graph construction unit 22 is used to construct a knowledge graph of urban governance based on multiple entities corresponding to different entity types in the municipal governance government data and the relationships between these entities; each entity corresponds one-to-one with each node in the knowledge graph of urban governance, and each node in the knowledge graph of urban governance contains the event corresponding to that node;

[0156] The recognition unit 23 is used to perform the following recognition operation on each multimodal fusion feature in the multimodal fusion feature set:

[0157] The geographic location information in the multimodal fusion feature is matched with the coordinates of events contained in each node of the urban governance knowledge graph to obtain the target node that matches the multimodal fusion feature.

[0158] The matching degree between the multimodal fusion feature and the event corresponding to each target node is weighted based on the feature weights corresponding to each target node to obtain the comprehensive risk score of the multimodal fusion feature;

[0159] When the comprehensive risk score is greater than the preset risk threshold, the information corresponding to the multimodal fusion feature is determined to be false information.

[0160] In one embodiment of this application, the automatic identification device for false information further includes a propagation path calculation unit;

[0161] The propagation path calculation unit is used for:

[0162] All false information corresponding to the multimodal fusion feature set is grouped; each group of false information corresponds to the same event.

[0163] A network for the spread of misinformation is constructed based on each set of misinformation.

[0164] In response to a request to trace the source of false information corresponding to a target event, the propagation network of false information corresponding to the target event is traversed based on a depth-first search algorithm to obtain the propagation path of the target event.

[0165] In one embodiment of this application, the propagation path calculation unit is further configured to:

[0166] The spatial location and data source of the propagation nodes in the propagation path are divided to determine the propagation area and platform of the false information corresponding to the target event.

[0167] The dissemination platform based on data sources divides communities into multiple dissemination communities for the spread of false information.

[0168] In one embodiment of this application, the text features in the multiple data sources are obtained by processing the text data in the multiple data sources using a pre-trained language model;

[0169] The automatic identification device for false information also includes a parameter update unit. After determining that the information corresponding to the multimodal fusion feature is false information, the parameter update unit is used to:

[0170] False information is stored in a database containing various types of false information, and it is determined whether the type of false information belongs to the type already included in the database.

[0171] If the type of false information does not belong to the types already included in the information base, the type of false information is marked so that the parameters of the pre-trained language model can be updated when the number of marked false information reaches a preset threshold.

[0172] In one embodiment of this application, the parameter update unit is specifically used for:

[0173] Determine the confidence level corresponding to false information;

[0174] If the confidence level is less than the first value, the type of false information is manually labeled, all false information in the information database, including manually labeled false information, is obtained, and the parameters of the pre-trained language model are updated based on all false information.

[0175] If the confidence level is greater than the second value, the type of false information is automatically labeled using a predetermined self-training method, all false information in the information database, including automatically labeled false information, is obtained, and the parameters of the pre-trained language model are updated based on all false information; if the first value is less than the second value.

[0176] In one embodiment of this application, the parameter update unit is specifically used for:

[0177] All false information in the information database is used as training samples. The parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples to obtain the pre-trained language model with updated parameters.

[0178] In one embodiment of this application, the automatic identification device for false information further includes a data expansion unit;

[0179] The data expansion unit is used for:

[0180] The information database is expanded by using generative adversarial networks to obtain an updated information database;

[0181] Using all the fake information in the information database as training samples, the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples, including:

[0182] All the fake information in the updated information database is used as training samples, and the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples.

[0183] See Figure 4 , Figure 4 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 4 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the units in the above-described device embodiments, for example... Figure 3 The functions of the data acquisition unit 21, knowledge graph construction unit 22, and recognition unit 23 are shown.

[0184] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0185] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0186] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory.

[0187] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the automatic identification method for false information provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0188] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0189] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0190] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0193] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0194] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0195] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for automatically identifying false information, characterized in that, include: Obtain the multimodal fusion feature set corresponding to multiple data sources; the multimodal fusion feature set is obtained by mapping text features and visual features from multiple data sources to a multidimensional joint vector space using a visual-text joint embedding model; A knowledge graph for urban governance is constructed based on multiple entities corresponding to different entity types in the municipal governance government data and the relationships between these entities. Each entity corresponds one-to-one with each node in the knowledge graph for urban governance, and each node in the knowledge graph for urban governance contains the event corresponding to that node. For each multimodal fusion feature in the multimodal fusion feature set, the following recognition operation is performed: The geographic location information in the multimodal fusion feature is matched with the coordinates of events contained in each node of the urban governance knowledge graph to obtain the target node that matches the multimodal fusion feature. The matching degree between the multimodal fusion feature and the event corresponding to each target node is weighted based on the feature weights corresponding to each target node to obtain the comprehensive risk score of the multimodal fusion feature; When the comprehensive risk score is greater than a preset risk threshold, the information corresponding to the multimodal fusion feature is determined to be false information.

2. The method for automatically identifying false information as described in claim 1, characterized in that, After performing a recognition operation on each multimodal fusion feature in the multimodal fusion feature set, the method further includes: All false information corresponding to the multimodal fusion feature set is grouped; each group of false information corresponds to the same event. A network for the spread of misinformation is constructed based on each set of misinformation. In response to receiving a request to trace the source of false information corresponding to a target event, the propagation network of the false information corresponding to the target event is traversed based on a depth-first search algorithm to obtain the propagation path of the target event.

3. The method for automatically identifying false information as described in claim 2, characterized in that, Also includes: The propagation nodes in the propagation path are spatially and data sources are divided to determine the propagation area and platform of the false information corresponding to the target event. Based on the data source, the dissemination platform is divided into multiple dissemination communities for the spread of false information.

4. The method for automatically identifying false information as described in claim 1, characterized in that, The text features in the multiple data sources are obtained by processing the text data in the multiple data sources using a pre-trained language model; After determining that the information corresponding to the multimodal fusion feature is false information, the method further includes: The false information is stored in a database containing various types of false information, and it is determined whether the type of the false information belongs to a type already included in the database. If the type of false information does not belong to the types already included in the information base, the type of false information is marked so that the parameters of the pre-trained language model are updated when the number of marked false information reaches a preset threshold.

5. The method for automatically identifying false information as described in claim 4, characterized in that, The step of labeling the types of false information to update the parameters of the pre-trained language model when the number of labeled false information reaches a preset threshold includes: Determine the confidence level corresponding to the false information; If the confidence level is less than the first value, the type of the false information is manually labeled, all false information in the information database including the manually labeled false information is obtained, and the parameters of the pre-trained language model are updated based on all the false information. If the confidence level is greater than the second value, the type of the false information is automatically labeled using a predetermined self-training method, all false information in the information database, including the automatically labeled false information, is obtained, and the parameters of the pre-trained language model are updated based on all false information; the first value is less than the second value.

6. The method for automatically identifying false information as described in claim 5, characterized in that, The parameters of the pre-trained language model are updated based on all the false information, including: All the false information in the information database is used as training samples. The parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples to obtain the pre-trained language model with updated parameters.

7. The method for automatically identifying false information as described in claim 5 or 6, characterized in that, Also includes: The false information in the information database is expanded based on generative adversarial networks to obtain an updated information database; The step of using all the fake information in the information database as training samples and updating the parameters of the last n fully connected layers of the pre-trained language model based on the training samples includes: All the false information in the updated information database is used as training samples, and the parameters of the last n fully connected layers of the pre-trained language model are updated based on the training samples.

8. An automatic identification device for false information, characterized in that, include: The data acquisition unit is used to acquire multimodal fusion feature sets corresponding to multiple data sources; the multimodal fusion feature sets are obtained by mapping text features and visual features from multiple data sources to a multidimensional joint vector space using a visual-text joint embedding model; The knowledge graph construction unit is used to construct a knowledge graph of urban governance based on multiple entities corresponding to different entity types in the municipal governance government data and the relationships between these entities; each entity corresponds one-to-one with each node in the knowledge graph of urban governance, and each node in the knowledge graph of urban governance contains the event corresponding to that node. The recognition unit is configured to perform the following recognition operation on each multimodal fusion feature in the multimodal fusion feature set: The geographic location information in the multimodal fusion feature is matched with the coordinates of events contained in each node of the urban governance knowledge graph to obtain the target node that matches the multimodal fusion feature. The matching degree between the multimodal fusion feature and the event corresponding to each target node is weighted based on the feature weights corresponding to each target node to obtain the comprehensive risk score of the multimodal fusion feature; When the comprehensive risk score is greater than a preset risk threshold, the information corresponding to the multimodal fusion feature is determined to be false information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images