Event false information detection method and device based on prior knowledge guidance
By introducing an external knowledge base to optimize triple relationship characteristics and integrating graph structure features with multimodal features, the problem of insufficient interpretability in small sample event detection is solved, and the credibility and generalization ability of the detection results are improved.
Patent Information
- Application Number
- CN202411662939.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-20
- Publication Date
- 2025-03-14
- Estimated Expiration
- 2044-11-20
AI Technical Summary
The existing knowledge-guided false information detection methods lack effective prior knowledge guidance when facing small sample events, making it difficult for the model to clearly determine key text features or describe information, resulting in insufficient interpretability.
By obtaining the description text and associated pictures of the events to be detected, relationship triplets, text features and image features are extracted, heterogeneous graphs are constructed and node feature vectors are calculated, relationship features are processed using graph convolution networks, multimodal features are fused, and relationship features are optimized through pre-established relationship weight prediction models.
It improves the accuracy of the model's judgment of event relationships under small sample conditions, enhances the ability to express and judge multimodal information, and improves the credibility and reliability of the detection results.
Smart Images

Figure CN119622569A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of network information security technology, and in particular to a method and device for detecting false event information based on prior knowledge guidance. Background Art
[0002] In the field of network security technology, network information security is particularly important. The openness of the Internet leads to it being filled with all kinds of false information. The spread of such false information not only interferes with public judgment, but may also cause confusion in public opinion and have a profound negative impact on social stability and development.
[0003] In order to improve the credibility of information on social platforms, industry experts and scholars have begun to actively explore effective governance strategies to reduce the impact of false information. Knowledge guidance is one of the effective methods, which uses external knowledge sources to analyze the differences between the content of the event to be detected on the social platform and the real data to detect the authenticity of the event. These knowledge sources include text and pictures obtained from other web pages, as well as online encyclopedias and fact-checking databases. In terms of detection models, it includes hierarchical multi-head attention networks, which combine text and evidence attention for false event detection; the CompareNet model, which detects fake news by comparing event information with external knowledge; and a preference-aware false event recognition framework, which combines with external knowledge bases for pattern recognition and joint detection. These studies have shown that knowledge guidance can effectively improve the accuracy of false event detection.
[0004] Mainstream methods in the field of false event detection rely on deep learning technology to analyze the semantic content of events to make true or false judgments. However, due to the limited number of small sample events, the model lacks effective prior knowledge guidance during the training process, making it difficult for the model to clearly determine which text features or descriptive information have a key impact on decision-making, resulting in insufficient interpretability. The interpretability problem reduces the credibility and reliability of related research, thereby affecting the practical application and promotion of the model.
[0005] Therefore, how to effectively use external knowledge bases for fact-checking to improve the interpretability of the model's judgment when facing small sample events has become a key issue that needs to be urgently addressed in the current field of false event detection. Summary of the invention
[0006] The present invention provides a method and device for detecting false information of events based on prior knowledge guidance, so as to solve the problem that due to the limited prior knowledge of small sample events, the existing false information detection method based on knowledge guidance cannot obtain sufficient prior information, resulting in reduced credibility of the detection result.
[0007] The present invention is achieved through the following technical solutions:
[0008] A first aspect of the present invention provides a method for detecting false event information based on prior knowledge, comprising:
[0009] Get the description text and associated pictures of the event to be detected;
[0010] Extracting relation triples and text features from the description text, and extracting image features from the associated pictures;
[0011] A heterogeneous graph is constructed based on the relationship triples, entities are used as nodes of the heterogeneous graph, node feature vectors of each node are calculated, and the relationship triples are input into a pre-established relationship weight prediction model to obtain relationship weights; the relationship weight prediction model is obtained based on loss training between a predicted value and a priori values of a known knowledge base;
[0012] Processing the node feature vector and the relationship weight using a graph convolutional network to obtain the relationship features of the relationship triplet;
[0013] Fusing the relationship feature, text feature and image feature to obtain a fused feature;
[0014] The fused features are input into a pre-trained discriminator to obtain a detection result of the event to be detected.
[0015] The present invention optimizes the triple relationship features by introducing an external knowledge base, ensuring that the model is more accurate in judging event relationships under small sample conditions. At the same time, the graph structure features are integrated with multimodal features to achieve full expression of multimodal information and enhance the discrimination ability, further improving the model's ability to judge small sample events. In addition, since the present invention focuses on knowledge guidance and multimodal information fusion, the training of the model is not limited to data in a specific field, nor is it limited to small sample events, expanding the application of false information detection in multiple fields and scenarios.
[0016] In one embodiment, extracting text features from the description text includes: extracting global features and local features from the description text using a Bert model, wherein the text features include concatenated text local features and text global features.
[0017] In one implementation, extracting image features from the associated pictures includes: extracting global features and local features of the associated pictures using a VGG model, wherein the image features include spliced image local features and image global features.
[0018] In one implementation, the relationship feature, text feature and image feature are fused to obtain a fused feature, including:
[0019] Concatenating the text features and the image features to obtain multimodal joint features;
[0020] The multimodal joint feature and the relationship feature are fused to obtain a fused feature.
[0021] In one implementation, the multimodal joint feature and the relationship feature are fused to obtain a fused feature, including:
[0022] Performing dimensionality reduction processing on the multimodal joint features through a multi-layer perceptron to obtain a dimensionality-reduced joint feature;
[0023] Processing the relationship features through global mean pooling to obtain global semantic features;
[0024] The dimension reduction joint feature and the global semantic feature are fused to obtain a fused feature.
[0025] In one implementation, the node feature vector is calculated as:
[0026]
[0027] in, Represents entity node e i The node feature vector of Represents entity node e i a collection of indices of word positions in the text, Indicates the length of the set, S j The feature vector representing the corresponding position in the text.
[0028] In one embodiment, the relationship weight prediction model and the training are performed through a joint loss, and the joint loss includes a triple loss function and a cross entropy loss function; the triple loss function is used to evaluate the difference between the predicted relationship weights of the relationship triplets of the sample data and the relationship weights provided by the known knowledge base, and the cross entropy function is used to evaluate the difference between the predicted results of the sample data and the annotated labels.
[0029] A second aspect of the present invention provides an event false information detection system based on prior knowledge guidance, comprising:
[0030] A data acquisition module is used to obtain the description text and associated pictures of the event to be detected;
[0031] A feature extraction module, used to extract relation triples and text features from the description text, and to extract image features from the associated pictures; and also used to construct a heterogeneous graph based on the relation triples, taking entities as nodes of the heterogeneous graph, and calculating a node feature vector of each node;
[0032] A relationship weight prediction module, used to predict the relationship weight of the relationship triple;
[0033] The feature extraction module is also used to process the node feature vector and the relationship weight to obtain the relationship feature of the relationship triple;
[0034] A multimodal fusion module, used for fusing the relationship features, text features and image features to obtain fused features;
[0035] The discriminator module is used to obtain the detection result of the event to be detected according to the fusion feature.
[0036] According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting false event information based on prior knowledge guidance as described in any one of the first aspect of the present invention is implemented.
[0037] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting false event information based on prior knowledge guidance as described in any one of the first aspects of the present invention is implemented.
[0038] Compared with the prior art, the present invention has the following advantages and beneficial effects: a knowledge-guided event false information detection method is proposed, which adopts a heterogeneous graph model, external knowledge audit, and the optimization design of triple loss and cross entropy loss, effectively improving the generalization ability of the model in small sample scenarios and the accuracy of false information detection, improving the ability to integrate data of different modalities, and improving the credibility of detection results, which has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:
[0040] Figure 1 is a flow chart of a method for detecting false event information based on prior knowledge guidance according to an embodiment of the present invention;
[0041] Figure 2 It is a schematic diagram of a processing flow of a false information detection device according to an embodiment of the present invention;
[0042] Figure 3 It is a structural schematic diagram of a false information detection system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0043] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.
[0044] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to or inherent to other steps or units of the device.
[0045] The terms used in the various embodiments of the application are only used to describe the purpose of specific embodiments and are not intended to limit the various embodiments of the application. As used herein, the singular form is intended to also include the plural form, unless the context clearly indicates otherwise. Unless otherwise limited, all terms used here (including technical terms and scientific terms) have the same meaning as the meanings commonly understood by ordinary technicians in the field of the various embodiments of the application. The terms (such as the terms defined in the dictionary generally used) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning, unless clearly defined in the various embodiments of the application.
[0046] In view of the deficiencies in the prior art, the present invention proposes a method, device and system for detecting false information in events based on prior knowledge guidance, which is suitable for false message prediction based on prior information guidance, especially for small sample event detection, and is beneficial to improving the credibility and reliability of the detection results, as well as the generalization ability of the detection model.
[0047] See also Figure 1 , Figure 2 As shown, Figure 1 is a flow chart of a method for detecting false event information based on prior knowledge according to an embodiment of the present invention. Figure 2 The present invention is a schematic diagram of the processing flow of a false information detection device, which includes the following steps.
[0048] S1, obtain the description text and associated pictures of the event to be detected.
[0049] S2, extracts triple relation features, text features and image features.
[0050] When extracting triples from text, the Bert model is used to embed the text and extract the relationship triples (entity, relationship, entity). The entity is used as the node feature of the graph model, the node feature vector and relationship weight of each entity node are calculated, and then the node feature vector and relationship weight are processed using the graph convolutional network (GCN) to obtain the relationship features of the relationship triples.
[0051] Conventional feature extraction models may be used for text features and image features. Text features and image features may be local features, global features, concatenated features of local features and global features, or fused features in other ways.
[0052] In one embodiment, the Bert model is used to extract global features and local features of the description text to obtain text feature representations of different granularities, and the local features of the text and the global features of the text are concatenated to obtain the text features in S2.
[0053] For the input text content t, t={x 1 ,x 2 ...x m}, where m represents the number of words in the text, and the converted features are represented as S = {s 1 ,s 2 ...s m}, where s i is the value corresponding to x i The conversion features of the characters are calculated by the pre-trained model Bert to obtain the fine-grained local features of the text content. The calculation formula is expressed as:
[0054] S={s 1 ,s 2 ...s m}=BERT(t)
[0055] For the global semantic information output by Bert, the calculation formula is expressed as:
[0056] h CLS =BERT(t)
[0057] Among them, h CLS Represents the global semantic information of the CLS tag corresponding to the Bert output. This vector is usually regarded as the global representation of the entire text t. Therefore, the global semantic information t output by BERT g It can be expressed as:
[0058] t g =h CLS
[0059] In one embodiment, a VGG (Visual Geometry Group) model is used to extract global features and local features of the associated image, and the image features in S2 are obtained by concatenating the local features of the image and the global features of the image.
[0060] For the input image content I, I is represented as a pixel matrix, and the converted feature is represented as F = {f 1 ,f 2 ,…,f n}, n represents the total number of pixels in the image, and f corresponds to pixel p i The pixel features are calculated by the pre-trained model VGG, which extracts the fine-grained local features of the image content. The calculation formula is expressed as:
[0061] F={f 1 ,f 2 ,…,f n}=VGG(I)
[0062] For the global semantic information output by VGG, the calculation formula is expressed as:
[0063] h global =VGG(I)
[0064] Among them, h global Represents the global semantic information output from VGG. This vector is usually regarded as the global representation of the entire image I. Therefore, the global semantic information I output by VGG g It is expressed as:
[0065] I g =h global
[0066] S3, relational features of triples based on prior knowledge.
[0067] For triple relationship features, the entities in the relationship triples are used as nodes of the heterogeneous graph. The node features are obtained by calculating the mean of the feature vectors corresponding to the entity positions in the text. 1 and entity e 2 , where the word positions in the original text are indexed sets and Then the node characteristics of the entity are calculated as follows:
[0068]
[0069] in, and Represents the entity node e of the triple 1 and e 2 The characteristic vector of and They correspond to the length of the entity node word set, S j and S k They represent the feature vectors of the corresponding characters in the text, and j and k represent the indexes of the corresponding characters.
[0070] The relationship features of the triples are verified and optimized with the help of external prior knowledge to improve the accuracy of the model's judgment of event relationships. Specifically, a relationship weight prediction model is built based on prior knowledge. The relationship triples are first input into the pre-built relationship weight prediction network for processing to obtain the predicted relationship weights. The predicted relationship weights are then compared with the relationship weights in the known knowledge base to optimize the model's prediction of relationship weights, thereby optimizing the triple relationship features.
[0071] First, the node feature vectors are concatenated and input into a neural network to establish a relationship weight prediction model, and the relationship weight W is obtained. edge The expression is:
[0072]
[0073] in, represents the concatenation operation of feature vectors, W nn is the weight matrix of the neural network, σ is the activation function, which ensures that the output value remains between (0,1), and W edge Represents the predicted relationship weight.
[0074] The relation triplets extracted from S2 are input into the large-scale external knowledge base phi3, and its powerful knowledge graph and relational reasoning capabilities are used to evaluate the credibility of the relations between entities. The relational features in the triples are further optimized and adjusted by comparing the relation weights predicted by the model with the weights of the known relations in the knowledge base.
[0075] During the model training process, the feedback results of phi3 are used to update the relationship features by calculating the triple loss between the predicted relationship and the query relationship. Specifically, during the model training, the relationship features are optimized through back propagation, so that it can effectively learn the potential relationship patterns in the knowledge base, thereby significantly improving the model's understanding and discrimination ability of the relationship, and the relationship weight labels are obtained as follows:
[0076] R = {r 1 ,r 2 ,…,r k}
[0077] Among them, R represents the set of relationship weight labels, r i represents the weight label corresponding to each triplet, and k represents the number of triplets contained in the event content.
[0078] The graph convolutional network is used to process the node feature vector and the relationship weight to obtain the relationship feature of the triple. Specifically, let the node feature be n is the number of nodes, and the weight matrix is W edge , then the calculation formula of GCN can be expressed as:
[0079]
[0080] Among them, H (l) represents the node feature matrix of the lth layer, which is obtained by concatenating V. is a normalized adjacency matrix, which contains the connection relationship and weight information between nodes W edge , W (l) is the learnable weight of the lth layer, and σ is the activation function. After a layer of GCN, the implicit feature H of the node is obtained (1) .
[0081] S4, fuses the relational features, text features and image features of the triplet to obtain the fused features.
[0082] First, the text features are concatenated with the global features of the image to generate a multimodal joint feature representation, which is then combined with the triple relationship features to obtain the final fusion feature.
[0083] Furthermore, before fusion, the joint feature representation is reduced in dimension through a multi-layer perceptron to obtain a more compact feature representation, which is then fused with the triple relationship feature.
[0084] Specifically, the global feature t extracted by Bert g Global features extracted from VGG I g Concatenate them to form a joint feature vector C, expressed as:
[0085] C=[t g ;I g ]
[0086] In order to improve the effectiveness of feature representation and reduce computational complexity, a multi-layer perceptron (MLP) neural network is used to reduce the dimension of the concatenated joint feature c to obtain a more compact and discriminative feature representation c′, which can be expressed as:
[0087] c′=MLP(c)
[0088] Finally, the obtained feature representation C′ will be used in the subsequent event false information detection model to provide stronger support for the accuracy and robustness of the model.
[0089] For relational features, global mean pooling is performed to obtain global semantic features, which are then fused with the multimodal joint features after dimensionality reduction.
[0090] To extract global semantic information, the calculation formula is:
[0091] h global =global_mean_pool(H (l+1) )
[0092] Among them, h global Represents the global semantic information output by GCN, and global_mean_pool represents the global mean pooling process.
[0093] Next, the reduced joint feature representation C′ is combined with the global semantic information h extracted by GCN global Directly add to get the final feature representation:
[0094]
[0095] S5, inputting the fused features into the pre-trained discriminator to obtain the detection result of the event to be detected.
[0096] The discriminator is trained by event information that has been marked as false news. In the present invention, the discriminator uses a neural network model to perform binary classification.
[0097] In one embodiment, when the relationship weight prediction model and the discriminator are jointly trained, the loss function adopts the joint loss of the triplet loss function and the cross entropy loss function, wherein the triplet loss is used to optimize the model's prediction of the triplet relationship, and the cross entropy loss is used to constrain the judgment of the authenticity of the event. By introducing the joint loss of the triplet loss function and the cross entropy loss function, the model is optimized to improve the accuracy and performance of relationship prediction and true and false event discrimination, and finally a trained multimodal small sample event false information detection model is obtained. The content description text and associated images of the event to be detected are input into the trained small sample event false information detection model to obtain the small sample event false information detection result.
[0098] The detailed training process is described as follows.
[0099] S51, the relationship loss is obtained by calculating the difference between the relationship edge weight predicted by the edge weight prediction model and the relationship weight provided in the external knowledge base.
[0100] Specifically, given the relationship edge weight W predicted by the model edgeThe relationship weight label R provided in the external knowledge base is used to calculate the distance between these weights to obtain the relationship loss. The loss is calculated in the following way: for each sample, if its label weight is greater than 0.7, it is considered a positive sample, and samples with label weights less than or equal to 0.7 are considered negative samples. For each pair of positive and negative samples, the distance between their weights is calculated and evaluated using the ternary loss formula to ensure that the distance of the positive sample is less than the distance of the negative sample, and a margin is added. Here, the margin is 1.0 to enhance the model's ability to distinguish and improve the accuracy of the prediction:
[0101] loss=max(0,d pos -d neg +margin)
[0102] Among them, d pos Represents the distance between the positive sample and its label weight, d neg Represents the distance between the negative sample and its label weight. Finally, by taking the average of all sample losses, the final relationship loss is obtained, which is expressed as:
[0103]
[0104] This relation loss is used to optimize the model parameters, where N represents the number of samples, to improve the accuracy and robustness of the model in the task of relation weight prediction.
[0105] S52, by comparing the prediction results of true and false events with the annotation labels, the cross entropy loss is calculated.
[0106] The content authenticity classification loss is calculated based on the false information detection results of small sample events and the true labels. The calculation formula is expressed as:
[0107]
[0108] Where N represents the number of samples, represents the input joint feature representation, y i is the i-th predicted value output by the discriminator.
[0109] S53, perform weighted summation of the relationship loss and the cross entropy loss to obtain the final loss function.
[0110] Specifically, the relation loss measures the difference between the edge weights predicted by the model and the relation weights provided in the external knowledge base, ensuring that the model can more accurately characterize the relationship between events. The cross entropy loss is used to measure the accuracy of the model in judging the authenticity of events, ensuring that the model can effectively distinguish between false and real events under the fusion of multimodal information.
[0111] The final loss function is a weighted sum of the two losses with a certain weight coefficient, and its expression is:
[0112]
[0113] Where λ represents the adjustment coefficient, which is a hyperparameter for adjusting the loss term trade-off.
[0114] An embodiment of the present invention further provides an event false information detection device based on prior knowledge guidance, which is used to execute the event false information detection method based on prior knowledge guidance of the above embodiment, and the device includes:
[0115] A data acquisition module is used to obtain the description text and associated pictures of the event to be detected;
[0116] A feature extraction module, used to extract relation triples and text features from the description text, and to extract image features from the associated pictures; and also used to construct a heterogeneous graph based on the relation triples, taking entities as nodes of the heterogeneous graph, and calculating a node feature vector of each node;
[0117] A relationship weight prediction module, used to predict the relationship weight of the relationship triple;
[0118] The feature extraction module is also used to process the node feature vector and the relationship weight to obtain the relationship feature of the relationship triple;
[0119] The discriminator module is used to obtain the detection result of the event to be detected according to the fusion feature.
[0120] Furthermore, the present invention also provides an event false information detection system based on prior knowledge guidance, such as Figure 3 The figure shows a structural diagram of a false information detection system, including the above-mentioned detection device, a user input module and a detection result visualization module. The user input module receives data input by the user, including information of events to be detected, graphic data such as detection time training samples, and instructions input by the user, such as instructions for modifying system parameters, model training setting parameters, etc. The system executes the training of the above-mentioned event false information detection method or detection device guided by prior knowledge based on the acquired event information and instructions. The output results are displayed through a visualization module, and the detection results include event structure diagrams, relationship verification information, and final true and false judgments. Users can intuitively view how the system processes input data, how to build a relationship network, how to verify facts, and quickly understand the detection conclusions. At the same time, key information in the detection process, such as entities, relationships, image features, etc., is also presented through a graphical interface to help users gain a deeper understanding of the detection process.
[0121] An embodiment of the present invention further provides an electronic device, which includes a processor and a memory, and the number of processors may be one or more. The memory, as a computer-readable storage medium, can be used to store software programs, computer executable programs, and modules. The processor executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory, so as to implement the event false information detection method based on prior knowledge guidance of any of the above embodiments of the present invention.
[0122] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0123] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for detecting false event information based on prior knowledge guidance according to any embodiment of the present invention.
[0124] An embodiment of the present invention further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the event false information detection method based on prior knowledge guidance according to any of the above embodiments of the present invention.
[0125] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for detecting false event information based on prior knowledge, characterized in that: include: Get the description text and associated pictures of the event to be detected; Extracting relation triples and text features from the description text, and extracting image features from the associated pictures; A heterogeneous graph is constructed based on the relationship triples, entities are used as nodes of the heterogeneous graph, node feature vectors of each node are calculated, and the relationship triples are input into a pre-established relationship weight prediction model to obtain relationship weights; the relationship weight prediction model is obtained based on loss training between a predicted value and a priori values of a known knowledge base; Processing the node feature vector and the relationship weight using a graph convolutional network to obtain the relationship features of the relationship triplet; Fusing the relationship feature, text feature and image feature to obtain a fused feature; The fused features are input into a pre-trained discriminator to obtain a detection result of the event to be detected.
2. The method for detecting false event information based on prior knowledge according to claim 1 is characterized in that: Extracting text features from the description text includes: using a Bert model to extract global features and local features from the description text, wherein the text features include concatenated text local features and text global features.
3. The method for detecting false event information based on prior knowledge guidance according to claim 1 is characterized in that: Extracting image features from the associated pictures includes: using a VGG model to extract global features and local features of the associated pictures, wherein the image features include spliced image local features and image global features.
4. The method for detecting false event information based on prior knowledge according to claim 1, characterized in that: The relationship features, text features and image features are fused to obtain fused features, including: Concatenating the text features and the image features to obtain multimodal joint features; The multimodal joint feature and the relationship feature are fused to obtain a fused feature.
5. The method for detecting false event information based on prior knowledge guidance according to claim 4 is characterized in that: The multimodal joint feature and the relationship feature are fused to obtain a fused feature, including: Performing dimensionality reduction processing on the multimodal joint features through a multi-layer perceptron to obtain a dimensionality-reduced joint feature; Processing the relationship features through global mean pooling to obtain global semantic features; The dimension reduction joint feature and the global semantic feature are fused to obtain a fused feature.
6. The method for detecting false event information based on prior knowledge according to claim 1, characterized in that: The node feature vector is calculated as: in, Represents entity node e i The node feature vector of Represents entity node e i a collection of indices of word positions in the text, Indicates the length of the set, S j The feature vector representing the corresponding position in the text.
7. The method for detecting false event information based on prior knowledge guidance according to claim 1, characterized in that: The relationship weight prediction model and the discriminator are trained through a joint loss, and the joint loss includes a triple loss function and a cross entropy loss function; the triple loss function is used to evaluate the difference between the predicted relationship weights of the relationship triples of the sample data and the relationship weights provided by the known knowledge base, and the cross entropy loss function is used to evaluate the difference between the predicted results of the sample data and the annotated labels.
8. An event false information detection device based on prior knowledge guidance, characterized in that: include: A data acquisition module is used to obtain the description text and associated pictures of the event to be detected; A feature extraction module, used to extract relation triples and text features from the description text, and to extract image features from the associated pictures; and also used to construct a heterogeneous graph based on the relation triples, taking entities as nodes of the heterogeneous graph, and calculating a node feature vector of each node; A relationship weight prediction module, used to predict the relationship weight of the relationship triple; The feature extraction module is also used to process the node feature vector and the relationship weight to obtain the relationship feature of the relationship triple; A multimodal fusion module, used for fusing the relationship features, text features and image features to obtain fused features; The discriminator module is used to obtain the detection result of the event to be detected according to the fusion feature.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for detecting false event information based on prior knowledge guidance according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting false event information based on prior knowledge guidance according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Cross-modal retrieval model training method and device
CN117435763A
Practical training knowledge graph completion method and system based on structure and text joint representation
CN117725224A
Small sample event false information detection method based on bias decoupling
CN117951299A
Multi-modal knowledge completion method based on image enhancement and multilevel residual network
CN118170922A
Multi-modal false news detection method based on factual knowledge fusion
CN118606789A