Information detection method, storage medium and electronic equipment

Abstract

The invention discloses an information detection method, a storage medium and electronic equipment. Relates to the field of artificial intelligence, and the method comprises the steps: obtaining to-be-detected information which comprises text information and image information; based on the similarity between the text information and the image information, the target weight of the image information is determined, and the target weight is used for reflecting the importance degree of the image information as supplementary description of the text information; performing text enhancement on the text information based on the image information and the target weight to obtain enhanced information; the to-be-detected information is detected based on the enhanced information, a detection result is obtained, and the detection result is used for representing whether the to-be-detected information is information describing a real event or not. Through the information detection method and device, the problem of low information detection accuracy in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to an information detection method, a storage medium, and an electronic device. Background Art

[0002] In the new media era, social media platforms, as a key channel for information dissemination, have greatly facilitated the rapid spread and wide coverage of news, but at the same time have also given rise to the proliferation of false news, posing a serious threat to social and economic stability.

[0003] Traditional fake news detection technology mainly focuses on text analysis. Although it can identify some false information based on text features, it ignores the important supplementary role of visual information such as pictures in social media news. In related technologies, image information and text information are fused for fake news detection, but because it only simply fuses image information and text information, it does not fully consider the specific correlation between image information and text information, resulting in limited effect when fusing multimodal information. In particular, when dealing with inconsistent images and text or low correlation, the model may be misled, which leads to low accuracy of information detection in related technologies.

[0004] Currently, no effective solution has been proposed to address the problem of low accuracy of information detection in related technologies. Summary of the invention

[0005] The main purpose of the present application is to provide an information detection method, a storage medium and an electronic device to solve the problem of low information detection accuracy in related technologies.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, an information detection method is provided. The method comprises: obtaining information to be detected, wherein the information to be detected includes text information and image information; determining the target weight of the image information based on the similarity between the text information and the image information, wherein the target weight is used to reflect the importance of the image information as a supplementary description of the text information; performing text enhancement on the text information based on the image information and the target weight to obtain enhanced information; and detecting the information to be detected based on the enhanced information to obtain a detection result, wherein the detection result is used to indicate whether the information to be detected is information describing a real event.

[0007] Furthermore, text enhancement is performed on the text information based on the image information and the target weight to obtain enhanced information, including: inputting the image information into the information generation model, using the information generation model to generate description information corresponding to the image information; and fusing the description information and the text information based on the target weight to obtain enhanced information.

[0008] Furthermore, the description information and the text information are fused based on the target weight to obtain enhanced information, including: based on the similarity between the text information and the description information, the target weight is corrected to obtain a corrected weight; based on the corrected weight, the description information and the text information are fused to obtain enhanced information.

[0009] Furthermore, the description information and the text information are fused based on the correction weight to obtain enhanced information, including: inputting the correction weight into a fusion model, and using the fusion model to perform weighted fusion on the description information and the text information to form enhanced information, wherein weighted fusion is used to indicate that the correction weight is used as an adjustment parameter in the process of fusing the description information and the text information, so that the target proportion is correlated with the correction weight, and the target proportion is used to indicate the ratio of the description information to the enhanced information.

[0010] Further, the information to be detected is detected based on the enhanced information to obtain a detection result, including: fusing the enhanced information, text information and image information to obtain fused information; and the information to be detected is detected based on the fused information to obtain a detection result.

[0011] Furthermore, the method also includes: extracting features from the text information and the image information respectively to obtain a text feature vector and an image feature vector; and determining the similarity based on the cosine value between the text feature vector and the image feature vector.

[0012] Furthermore, based on the cosine value between the text feature vector and the image feature vector, the similarity is determined, including: based on the cosine value between the text feature vector and the image feature vector, the initial similarity is determined; the first entity in the text information and the second entity in the image information are detected; based on the knowledge graph corresponding to the field of the information to be detected, the association between the first entity and the second entity is determined, wherein the knowledge graph is used to describe different entities in the field and the association relationship between different entities; the initial similarity is adjusted based on the association to obtain the similarity.

[0013] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an information detection device is provided. The device includes: an acquisition module, used to acquire information to be detected, wherein the information to be detected includes text information and image information; a first determination module, used to determine the target weight of the image information based on the similarity between the text information and the image information, wherein the target weight is used to reflect the importance of the image information as a supplementary description of the text information; an enhancement module, used to perform text enhancement on the text information based on the image information and the target weight to obtain enhanced information; a detection module, used to detect the information to be detected based on the enhanced information to obtain a detection result, wherein the detection result is used to indicate whether the information to be detected is information describing a real event.

[0014] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the above-mentioned information detection method.

[0015] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an electronic device is provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the above-mentioned information detection method is executed when the program is running.

[0016] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer program product is provided, including a computer program, and the computer program implements the above-mentioned information detection method when executed by a processor.

[0017] In an embodiment of the present application, firstly, the information to be detected including text information and image information is obtained; then, the target weight of the image information is determined by calculating the similarity between the text information and the image information; then, the text information is enhanced based on the image information and the target weight to obtain enhanced information; finally, the information to be detected is detected based on the enhanced information to obtain a detection result. It is easy to notice that the present application obtains the text information and the image information, and determines the target weight of the image information by calculating the similarity between the two, obtains the importance of the image information as a supplement to the text information, and then performs text enhancement on the text information according to the target weight to obtain enhanced information. In the above process, by analyzing and calculating the importance of the image information relative to the text information, the correlation between the image information and the text information is clarified, so that the image information can be used to effectively enhance the text information, so as to achieve the purpose of effectively integrating the image information and the text information and improving the accuracy of information detection, thereby solving the technical problem of low accuracy of information detection in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A hardware structure block diagram of a computer terminal for implementing an information detection method is shown;

[0020] Figure 2 is a flow chart of an information detection method according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of similarity calculation according to an embodiment of the present invention;

[0022] Figure 4 is a BLIP-2 model framework diagram according to an embodiment of the present invention;

[0023] Figure 5 is a query-type converter learning framework diagram according to an embodiment of the present invention;

[0024] Figure 6 is a flow chart of an optional information detection method according to an embodiment of the present invention;

[0025] Figure 7 is a schematic diagram of an information monitoring device according to an embodiment of the present invention;

[0026] Figure 8 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0030] Contrastive Language-Image Pre-training (CLIP): a pre-training model for learning the association between images and text;

[0031] Bootstrapping Large Image-Text Pretrained Models with Human Pseudo-Captions and Web Images (BLIP-2): BLIP-2 is a pre-trained model that uses human pseudo-labels and a large number of web images to generate accurate text descriptions that are closely related to the image content.

[0032] Querying Transformer (Q-former): It is an architecture designed to solve the problem of spatial misalignment between visual features and language features. In multimodal tasks, Q-Former uses a set of trainable query vectors (queries, referred to as Q) that interact with image features in the transformer architecture to extract image information associated with text descriptions.

[0033] Visual Transformer (ViT): A model that applies the Transformer architecture to image recognition tasks.

[0034] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0035] Example 1

[0036] According to an embodiment of the present application, an information detection method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing an information detection method. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0038] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the information detection method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned information detection method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0042] Under the above operating environment, this application provides Figure 2 The information detection method shown. Figure 2 FIG. 1 is a flow chart of an information detection method according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0043] Step S201, obtaining information to be detected, wherein the information to be detected includes text information and image information.

[0044] The above-mentioned information to be detected may refer to news or articles collected from social media, news platforms or other information dissemination channels whose authenticity needs to be judged. The information to be detected may include but is not limited to text description information, image information, video information, and audio information, etc. The specific information to be detected needs to be determined according to actual conditions and is not limited here. The information to be detected may be the analysis and detection object of the information detection method.

[0045] The above-mentioned text information may refer to the main carrier of information content. The text information may include but is not limited to the news title, text, summary and other parts. The specific text information needs to be determined according to the actual information situation and is not limited here. The text information can be used to directly reflect the narrative content of the news and the author's point of view.

[0046] The above-mentioned image information may refer to pictures contained in the information to be detected. The types of image information may include but are not limited to photos of the event scene, charts, data visualization, news-related pictures of people or places, etc. The specific image information needs to be determined according to actual conditions and is not limited here. Image information can be used to provide visual evidence for text information and help verify the accuracy of news texts.

[0047] In an optional embodiment, when detecting false news, the text information and image information in the information to be detected are obtained at the same time and comprehensively analyzed, wherein the text information is used to understand the theme and details of the news, while the image information provides visual evidence. The combination of the two can more comprehensively evaluate the authenticity and reliability of the news. For example, the text describes the details of an accident, and the accompanying picture shows the scene of the accident. If the details described in the two match, the credibility of the news increases; conversely, if there is an inconsistency or the accompanying picture is irrelevant to the text, it may indicate that the news contains false elements. The above process, by obtaining the information to be detected including text information and image information, and comparing and verifying the two, can more effectively judge the authenticity of the news and prevent the spread of false information.

[0048] Step S202: determining a target weight of the image information based on the similarity between the text information and the image information, wherein the target weight is used to reflect the importance of the image information as a supplement to the text information.

[0049] The above-mentioned similarity may refer to a numerical indicator used to measure the degree of relevance between text information and image information in terms of content, semantics or visual representation. The types of similarity may include but are not limited to cosine similarity, image-text similarity, etc. The specific similarity type needs to be determined according to actual needs and is not limited here. The similarity can be used to determine the proportion of image features in the final news features, evaluate the consistency between image information and text information, determine whether the image supports the facts described in the text, etc. The similarity can be calculated by comparing the language-image pre-training model.

[0050] The above-mentioned target weight may refer to the relative importance given to image information when integrating text information and image information. The target weight may be used to adjust the impact of image information on the overall characteristics of the news, so that it can both serve as a supplement to text information and be reasonably considered based on its actual relevance to the text.

[0051] In an optional embodiment, by calculating the similarity between text information and image information, the correlation between image information and text information can be measured, and then the target weight of image information can be determined. When the similarity is close to 1, it means that the image and text are highly correlated, and the target weight of image information is large, which means that image features account for a large proportion of fusion features, which can significantly enhance text features and help the model more accurately judge the authenticity of news; when the similarity is close to 0, it means that the image and text are irrelevant or contradictory, and the target weight of image information is small or even 0, which means that the contribution of image features in fusion features is ignored, avoiding misleading or irrelevant images from having a negative impact on the judgment of news authenticity. By determining the similarity and target weight, the information detection method proposed in this application can more intelligently process multimodal news data and improve the accuracy and reliability of false news detection.

[0052] In an optional embodiment, Figure 3 is a schematic diagram of similarity calculation according to an embodiment of the present invention. Figure 3 As shown, Figure 3 The right side shows the calculation process of a single similarity. First, the text information is processed through Text0 to obtain T k At the same time, the image information is processed by Image0 to obtain I k Finally, the similarity between the two is calculated and recorded as I k T k The left side shows the similarity calculation process of the overall information. The text information is extracted through the text encoder Text0 to obtain T1, T2, T3, ..., T N , the image information is extracted through the image encoder Image0 to obtain I1, I2, I3, ..., I N ,Finally, the similarity between the two is calculated as shown in the middle table.

[0053] Step S203: Perform text enhancement on the text information based on the image information and the target weight to obtain enhanced information.

[0054] The above-mentioned text enhancement may refer to a technical process of supplementing and enriching the content and semantics of text information by fusing image information and target weights to generate enhanced information. Text enhancement may be used to improve the expressive power and information volume of text information, so that the detection model can make judgments based on more complete and accurate news features.

[0055] The above-mentioned enhanced information may refer to new text features generated after text enhancement processing. The enhanced information may be used to represent more accurate text information and achieve more accurate false news detection.

[0056] In an optional embodiment, the text information is enhanced by the target weight of the image information to obtain enhanced information that can better reflect the accuracy and completeness of the news. Through the above steps, it can be ensured that the text information is effectively enhanced during the fusion process, and the generated enhanced information can more accurately reflect the authenticity and completeness of the news, thereby improving the performance of the fake news detection model.

[0057] Step S204: Detect the information to be detected based on the enhanced information to obtain a detection result, wherein the detection result is used to indicate whether the information to be detected is information describing a real event.

[0058] In an optional embodiment, the information to be detected is detected based on the enhanced information to obtain a detection result, which directly reflects whether the information to be detected describes a real event. This detection method based on enhanced information combines the text description of text information with the intuitive evidence provided by image information, which can significantly improve the analysis ability of complex financial news, thereby improving the accuracy and credibility of false news identification as a whole.

[0059] In an embodiment of the present application, firstly, the information to be detected including text information and image information is obtained; then, the target weight of the image information is determined by calculating the similarity between the text information and the image information; then, the text information is enhanced based on the image information and the target weight to obtain enhanced information; finally, the information to be detected is detected based on the enhanced information to obtain a detection result. It is easy to notice that the present application obtains the text information and the image information, and determines the target weight of the image information by calculating the similarity between the two, obtains the importance of the image information as a supplement to the text information, and then performs text enhancement on the text information according to the target weight to obtain enhanced information. In the above process, by analyzing and calculating the importance of the image information relative to the text information, the correlation between the image information and the text information is clarified, so that the image information can be used to effectively enhance the text information, so as to achieve the purpose of effectively integrating the image information and the text information and improving the accuracy of information detection, thereby solving the technical problem of low accuracy of information detection in the related technology.

[0060] Optionally, in the information detection method provided in the embodiment of the present application, text enhancement is performed on text information based on image information and target weight to obtain enhanced information, including: inputting the image information into an information generation model, and using the information generation model to generate description information corresponding to the image information; and fusing the description information and text information based on the target weight to obtain enhanced information.

[0061] The above-mentioned information generation model may refer to an artificial intelligence model for generating new text descriptions or information. The types of information generation models may include but are not limited to artificial intelligence models such as guided large-scale image-text pre-training models. The specific information generation model needs to be determined according to actual needs and is not limited here. The information generation model can be used to extract meaningful information from image information and convert it into text form to facilitate model understanding and processing.

[0062] The above-mentioned description information may refer to a text description generated by an information generation model based on the input image information. The type of description information may include but is not limited to scene description information, task description information, event description information, etc. The specific description information needs to be determined based on the specific information of the image information and is not limited here. The description information can be used to supplement the text information and help the detection model better understand the context of the news.

[0063] In an optional embodiment, the description information generated by the information generation model is fused with the original text information through similarity correction, and the generated enhanced information not only contains all the original information of the news, but also incorporates the rich details of the image description, so that the detection model can make judgments based on more comprehensive and accurate news features, significantly improving the accuracy and robustness of false news detection. In addition, the fusion of description information and text information is controlled under the framework of target weights, ensuring that the contribution of image description to the overall characteristics of the news matches its authenticity and relevance, avoiding the misleading of irrelevant images, and further optimizing the performance of the detection model.

[0064] In an optional embodiment, Figure 4 is a BLIP-2 model framework diagram according to an embodiment of the present invention, such as Figure 4 As shown in the figure, the BLIP-2 model is mainly divided into two parts: the querying transformer (Q-former) and the large language model. First, the image information is input into the picture encoder, and then input into the query transformer after being processed by the picture encoder. At the same time, the trainable Q vector is input into the query transformer. The query transformer trains both of them to obtain image features containing visual information. Finally, the large language model is used to train them to obtain the picture and its description text.

[0065] In an optional embodiment, Figure 5 is a query-type converter learning framework diagram according to an embodiment of the present invention, such as Figure 5 As shown in the figure, the image information is first processed by the image encoder and then input into the query converter; the left part of the query converter trains the trainable Q vector through the self-attention layer, the cross-attention layer and the feedforward neural network in sequence, and finally obtains the image-text matching result; the right part of the query converter inputs the text information into the self-attention layer and the feedforward neural network in sequence for training to realize the text generation of the image; finally, the output results of the two parts are subjected to image-text comparison learning.

[0066] The left part of the query-type converter may refer to an image encoder, and the right part of the query-type converter may refer to a text encoder.

[0067] In an optional embodiment, an image encoder Image Transformer is used to generate image features. For a picture P to be generated, a ViT model trained by a CLIP model is used to generate the image features P ViT That is, the description information corresponding to the image information.

[0068] P ViT =ViT(P);

[0069] Where P ViT Represents the description information corresponding to the image information, ViT represents the training model, and P represents the picture.

[0070] In order to solve the problem of misalignment between visual features and text feature spaces, Image Transformer introduces a trainable Q vector to align the feature space. First, the dependency relationship between them is modeled through the Self-Attention network.

[0071] Q Self-att =Self-Attention (Q) ;

[0072] In the formula, Self-Attention represents the self-attention mechanism, Q Self-att Represents the self-attention calculation result of the Q vector.

[0073] Then use the cross-attention mechanism to model the Q vector and image feature P ViT The dependency relationship between them, the query matrix is ​​defined as:

[0074]

[0075] Where W Q , W K With W V Corresponding to the feature matrices that convert image features into query, key, and value vectors, P ViT represents the image features, Q represents the query vector, K represents the key vector, V represents the value vector, Q Self-att Represents the self-attention calculation result of the Q vector.

[0076] The cross-attention feature calculation formula is:

[0077]

[0078] In the formula, α Cross-att represents the cross attention feature, Q represents the query vector, K T represents the transposed matrix of the key vector, V represents the value vector, d represents the key vector dimension, and softmax represents the normalization function.

[0079] Finally, the visual feature α of the fusion learnable Queries vector generated by the image encoder Image Transformer is obtained through a feedforward neural network. ImageT :

[0080] α ImageT =FFN I (α Cross-att );

[0081] In the formula, α Cross-att represents the cross attention feature, α ImageT represents visual features, and FFN represents feed-forward neural network.

[0082] For the image P to be generated, the text description S is generated by the BLIP-2 model with set task parameters. P :

[0083] Using the text encoder Text Transformer, we generate text features. First, we use a Self-Attention network shared with ImageTransformer to obtain the self-attention features of the text:

[0084]

[0085] In the formula, S P represents text description, Self-Attention represents the self-attention mechanism, S Self-att Represents the self-attention features of text.

[0086] Finally, the text features of the image description are obtained through a feedforward neural network. The text encoder also acts as a text decoder when generating image descriptions. For the text description in the training set, S P , the text feature calculation formula is as follows:

[0087] α TextT =FFN2(S Self-Att );

[0088] Where FFN2 represents a feedforward neural network, S Self-att Represents the self-attention feature of the text, α TextT Represents text features.

[0089] Optionally, in the information detection method provided in the embodiment of the present application, the description information and the text information are fused based on the target weight to obtain enhanced information, including: based on the similarity between the text information and the description information, the target weight is corrected to obtain a corrected weight; based on the corrected weight, the description information and the text information are fused to obtain enhanced information.

[0090] The above-mentioned corrected weight may refer to the actual weight obtained after correcting the target weight according to the relative importance of different modal information in the fusion process. The role of the corrected weight may include but is not limited to intelligently fusing multimodal information, reducing the interference of irrelevant modal information, and improving the accuracy of the detection model.

[0091] In an optional embodiment, the target weight is modified based on the similarity between the text information and the description information to obtain the modified weight, and then the description information and the text information are fused based on the modified weight to obtain the fused information. The above fusion process based on the modified weight enables the model to generate enhanced information containing more comprehensive and accurate descriptions, providing a more solid foundation for subsequent false news detection. This fusion mechanism improves the intelligent analysis capability of the model, thereby ensuring the high quality and high credibility of the false news detection results.

[0092] In an optional embodiment, in order to fuse the features between modalities, the present application adopts an attention mechanism for fusion. For the text description S′ of the encoded text information S′ and the image information P′, P , use the TextEncoder trained by the CLIP model to extract features and After splicing, we can get the common features of news text features and image description features

[0093]

[0094] In the formula, represents the text encoder part in the CLIP model, S′ represents the encoded text information, S′ P Represents the text description of image information, Concat represents the concatenation operation, Represents the splicing feature.

[0095] Feature fusion is performed through the attention mechanism, and the query matrix is ​​defined as:

[0096]

[0097] In the formula,

[0098] represents the splicing feature, W Q , W K With W V They respectively represent the functions used to generate the query vector feature matrix, the key vector feature matrix and the value vector feature matrix, Q represents the query vector, K represents the key vector, and V represents the value vector.

[0099] For each feature sequence, q i , ki , v i It can be expressed as:

[0100]

[0101] In the formula, a i represents the embedding vector of the i-th element in the input sequence, W q , W k With W v They are used to generate the query vector feature matrix, key vector feature matrix and value vector feature matrix, respectively. i , k i With v i Represent the transformed query vector, key vector and value vector respectively.

[0102] Then the intermediate output in the attention network is expressed as:

[0103]

[0104] In the formula, q i represents the query vector at the i-th position, k j represents the key vector at the jth position, d represents the dimension, a i，j represents the attention weight between the query vector at the i-th position and the key vector at the j-th position, v j represents the value vector at the jth position, b i Represents the output representation of the element at position i in the sequence after being processed by the self-attention mechanism.

[0105] The final fusion feature β is:

[0106] β=∑b i ;

[0107] In the formula, β represents the fusion feature, b i It represents the output representation of the element at position i in the sequence after being processed by the self-attention mechanism, and ∑ represents the summation operation.

[0108] In addition, similarity is introduced into the feature fusion process, and the modal fusion method with similarity correction is obtained as follows:

[0109] When i=p or j=p:

[0110]

[0111] In the formula, ω represents the learnable parameter, q i represents the query vector at the i-th position, k j represents the key vector at the jth position, d represents the dimension, a i，jRepresents the attention weight between the query vector at the i-th position and the key vector at the j-th position. f(ω) is positively correlated with ω. Let f(ω) = ω τ , then the above formula can be modified to:

[0112]

[0113] Finally, the corrected classification feature β is obtained, and the softmax layer is used to get the prediction result of the news:

[0114]

[0115] In the formula, β represents the corrected classification feature, b β represents the bias vector of the fully connected layer, W β represents the weight matrix of the fully connected layer, Represents the prediction result of the model, and Softmax represents the normalization function.

[0116] Optionally, in the information detection method provided in the embodiment of the present application, the description information and the text information are fused based on the correction weight to obtain enhanced information, including: inputting the correction weight into the fusion model, and using the fusion model to perform weighted fusion on the description information and the text information to form enhanced information, wherein the weighted fusion is used to indicate that the correction weight is used as an adjustment parameter in the process of fusing the description information and the text information, so that the target proportion is correlated with the correction weight, and the target proportion is used to indicate the ratio of the description information to the enhanced information.

[0117] The above-mentioned fusion model may refer to a deep learning model for converting text information and image information into a unified representation. The types of fusion models may include but are not limited to fusion models based on attention mechanisms, graph neural network fusion models, etc. The specific fusion model needs to be determined according to actual needs and is not limited here. The fusion model can be used to integrate information from different modalities into a more comprehensive and coherent representation to facilitate more accurate decision-making or prediction.

[0118] The above-mentioned weighted fusion may refer to a method of fusing multimodal information. The types of weighted fusion may include but are not limited to linear weighted fusion, nonlinear weighted fusion, dynamic weighted fusion, etc. The specific type of weighted fusion needs to be determined according to actual needs and is not limited here. Weighted fusion can be used to balance the contribution of multimodal information, reduce interference from irrelevant modes, etc.

[0119] The above-mentioned target proportion may refer to the proportion that the descriptive information should account for in the final enhanced information. The target proportion may be used to reflect the contribution of the descriptive information to the overall news features.

[0120] In an optional embodiment, the fusion model performs weighted fusion of text information and image description information through an attention mechanism. During the fusion process, the correction weight is input into the model as an adjustment parameter, and the model weights the description information and text information according to the correction weight to form enhanced information. In this process, the target weight is positively correlated with the correction weight, ensuring that the proportion of the description information in the enhanced information matches its similarity and importance to the text information. For example, for news with high similarity between images and texts, the description information will be given a higher weight, and vice versa. This can effectively screen out the description information that is highly relevant to the news text, improve the accuracy and completeness of the news features, and thus improve the accuracy of false news detection.

[0121] Optionally, in the information detection method provided in the embodiment of the present application, the information to be detected is detected based on the enhanced information to obtain a detection result, including: fusing the enhanced information, text information and image information to obtain fused information; and detecting the information to be detected based on the fused information to obtain a detection result.

[0122] In an optional embodiment, by deeply analyzing and fusing the text information and image description information, more accurate and comprehensive fusion information is obtained, and then the information to be detected is detected based on the fusion information to obtain the final detection result. The above process is introduced through deep multimodal fusion, which realizes the comprehensive analysis and accurate judgment of news content, greatly improves the accuracy and efficiency of false news detection, and provides a strong technical guarantee for maintaining the security and stability of financial information.

[0123] In an optional embodiment, Figure 6 is a flow chart of an optional information detection method according to an embodiment of the present invention. Figure 6 As shown in the figure, firstly, the text information is feature extracted by CLIP text encoder, and the image information is similarity calculated by CLIP picture encoder, and the description information is generated by BLIP picture encoder through picture description, and then processed by BLIP text encoder to obtain image description information; then the image description information, text information and similarity are feature fused to obtain fused features; finally, the fused features are detected by classification network to obtain detection results.

[0124] Optionally, the information detection method provided in the embodiment of the present application also includes: performing feature extraction on the text information and the image information respectively to obtain a text feature vector and an image feature vector; and determining the similarity based on the cosine value between the text feature vector and the image feature vector.

[0125] The above-mentioned text feature vector may refer to a numerical vector extracted from text information that can represent multi-dimensional features such as the content, style, and emotion of the text. The text vector feature may be obtained by encoding the text information through a text encoder Text Transformer. The text feature vector may be used to represent the deep semantics and structural information of the news text.

[0126] The above-mentioned image feature vector may refer to a numerical vector extracted from image information that can reflect features such as image content, structure, and visual elements. The image feature vector may be obtained by encoding the image information through an image encoder Image Transformer. The image feature vector may be used to capture key visual information in news illustrations. By comparing it with text features, it helps the detection model understand the consistency between the image and text and identify false news where the image and text do not match.

[0127] The above cosine value may refer to a method for measuring the similarity between two vectors. The cosine value may be used to quantify the similarity between a text feature vector and an image feature vector. The cosine value ranges from -1 to 1. The closer the value is to 1, the more similar the two vectors are. The closer the value is to -1, the less similar the two vectors are.

[0128] In an optional embodiment, by performing feature extraction on text and image information respectively, the obtained text feature vector and image feature vector are further used to calculate similarity using cosine values. This process not only strengthens the multi-angle understanding of news content, but also provides an important basis for subsequent feature fusion and false news detection, significantly improving the accuracy and robustness of detection.

[0129] In an optional embodiment, for a given text information π={S0, S1, S2, ..., S n} and the corresponding image information P = {P0, P1, P2, ..., P n}, CLIP model uses Transformer network as text encoder to obtain text feature vector CLIP t , the calculation formula is as follows:

[0130] CLIP t ={S i ′}

[0131] S i ′=Transformer(S i );

[0132] In the formula, CLIP t represents the text feature vector, S i Represents the original text information, S i′ represents the encoded text information, and Transformer represents the text encoder.

[0133] The image information uses the visual pre-trained model (Visual Transformer, referred to as ViT) as the image encoder to obtain the image feature vector CLIP i :

[0134] CLIP i {P i ′}

[0135] P i ′=ViT(P i );

[0136] In the formula, CLIP i represents the image feature vector, P i Represents the original image information, P i ′ represents the encoded image information, and ViT represents the image encoder.

[0137] CLIP model Given a batch of N image and text pairs, to train a batch of N × N possible image feature vectors CLIP i and text feature vector CLIP t For a certain picture-text pair I i -T j , where 0≤i<N, 0≤j<N.

[0138] If i=j, then this sample pair is called a positive sample; if i≠j, then this sample pair is called a negative sample. The cosine similarity of each pair of samples is defined as:

[0139]

[0140] In the formula, S′ represents the encoded text information, P′ represents the encoded image information, and cos represents the cosine similarity calculation.

[0141] The task of contrastive learning is defined as:

[0142]

[0143] In the formula, S′ represents the encoded text information, P′ represents the encoded image information, Max represents the maximum cosine similarity of N positive samples, and Min represents N 2 -The cosine similarity of N negative samples, cos represents cosine similarity.

[0144] The trainable parameters are text feature and image feature encoders, described as:

[0145]

[0146] In the formula, Transformer represents the text encoder, ViT represents the image encoder, and S i Indicates text information, P i Represents image information, Max represents the maximum value, Min represents the minimum value, and cos represents cosine similarity.

[0147] The trained text encoder and image encoder are obtained through the contrast learning training phase. The features of the encoded text information and image information are extracted through the trained text encoder and image encoder, and then the cosine similarity is calculated to obtain the similarity between the text information to be detected and the image information:

[0148]

[0149] In the formula, ω represents similarity, S′ represents the encoded text information, P′ represents the encoded image information, and CLIP Encoder_t Represents the text encoder in the CLIP model, CLIP Encoder_i represents the image encoder in the CLIP model, and cos represents the cosine similarity function.

[0150] Optionally, in the information detection method provided in the embodiment of the present application, the similarity is determined based on the cosine value between the text feature vector and the image feature vector, including: determining the initial similarity based on the cosine value between the text feature vector and the image feature vector; detecting the first entity in the text information and the second entity in the image information; determining the association between the first entity and the second entity based on the knowledge graph corresponding to the field of the information to be detected, wherein the knowledge graph is used to describe the association relationship between different entities in the field and different entities; adjusting the initial similarity based on the association to obtain the similarity.

[0151] The above-mentioned initial similarity may refer to the similarity obtained by directly calculating the cosine value between the text feature vector and the image feature vector without considering the entity association. The initial similarity may be used to intuitively measure the similarity between text information and image information at the visual and text levels.

[0152] The above-mentioned first entity may refer to an entity appearing in the text information. The first entity may include but is not limited to a person, organization, place, event, time point, etc. The specific first entity needs to be determined based on the actual text information and is not limited here.

[0153] The above-mentioned second entity may refer to an entity identified in the image information. The second entity may include but is not limited to a person, organization, place, event, time point, etc. The specific second entity needs to be determined based on the actual image information and is not limited here.

[0154] In an optional embodiment, the initial similarity is determined by calculating the cosine value between the text feature vector and the image feature vector. Then, in order to more accurately evaluate the authenticity of the news, the consistency at the entity level needs to be further considered. Therefore, the present application detects the first entity in the news text and the second entity in the accompanying image, and uses the domain knowledge graph to determine the correlation between these entities, thereby intelligently adjusting the initial similarity. This process not only considers the direct similarity between text and image at the content level, but also goes deep into the logical correlation at the entity level, ensuring the depth and breadth of information detection.

[0155] The information detection method provided in the embodiment of the present application analyzes and calculates the importance of image information relative to text information, clarifies the correlation between image information and text information, and can thus effectively enhance text information using image information, thereby achieving the purpose of effectively integrating image information and text information and improving information detection accuracy, thereby solving the technical problem of low information detection accuracy in related technologies.

[0156] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0157] Example 2

[0158] The embodiment of the present application also provides an information detection device. It should be noted that the information detection device of the embodiment of the present application can be used to execute the information detection method provided in the embodiment of the present application. The information detection device provided in the embodiment of the present application is introduced below.

[0159] According to an embodiment of the present application, a device for implementing the above information detection method is also provided. Figure 7 is a schematic diagram of an information monitoring device according to an embodiment of the present invention. Figure 7 As shown, the device includes: an acquisition module 702, used to acquire information to be detected, wherein the information to be detected includes text information and image information; a first determination module 704, used to determine the target weight of the image information based on the similarity between the text information and the image information, wherein the target weight is used to reflect the importance of the image information as a supplement to the text information; an enhancement module 706, used to perform text enhancement on the text information based on the image information and the target weight to obtain enhanced information; a detection module 708, used to detect the information to be detected based on the enhanced information to obtain a detection result, wherein the detection result is used to indicate whether the information to be detected is information describing a real event.

[0160] The information detection device provided in the embodiment of the present application analyzes and calculates the importance of image information relative to text information, clarifies the correlation between image information and text information, and can thus use image information to effectively enhance text information, thereby achieving the purpose of effectively integrating image information and text information and improving information detection accuracy, thereby solving the technical problem of low information detection accuracy in related technologies.

[0161] Optionally, in the information detection device provided in the embodiment of the present application, the enhancement module includes: a generation unit, used to input the image information into the information generation model, and use the information generation model to generate description information corresponding to the image information; a first fusion unit, used to fuse the description information and text information based on the target weight to obtain enhanced information.

[0162] Optionally, in the information detection device provided in the embodiment of the present application, the first fusion unit includes: a correction subunit, used to correct the target weight based on the similarity between the text information and the description information to obtain a corrected weight; and a fusion subunit, used to fuse the description information and the text information based on the corrected weight to obtain enhanced information.

[0163] Optionally, in the information detection device provided in the embodiment of the present application, the fusion subunit includes: inputting the correction weight into the fusion model, and using the fusion model to perform weighted fusion on the description information and the text information to form enhanced information, wherein the weighted fusion is used to indicate that the correction weight is used as an adjustment parameter in the process of fusing the description information and the text information, so that the target proportion is correlated with the correction weight, and the target proportion is used to indicate the ratio of the description information to the enhanced information.

[0164] Optionally, in the information detection device provided in the embodiment of the present application, the detection module includes: a second fusion unit, used to fuse the enhanced information, text information and image information to obtain fused information; and a first detection unit, used to detect the information to be detected based on the fused information to obtain a detection result.

[0165] Optionally, the information detection device provided in the embodiment of the present application also includes: an extraction module, used to extract features from text information and image information respectively to obtain text feature vectors and image feature vectors; and a second determination module, used to determine the similarity based on the cosine value between the text feature vector and the image feature vector.

[0166] Optionally, in the information detection device provided in the embodiment of the present application, the second determination module includes: a first determination unit, used to determine the initial similarity based on the cosine value between the text feature vector and the image feature vector; a second detection unit, used to detect the first entity in the text information and the second entity in the image information; the second determination unit, used to determine the association between the first entity and the second entity based on the knowledge graph corresponding to the field of the information to be detected, wherein the knowledge graph is used to describe the association relationship between different entities in the field and different entities; an adjustment unit, used to adjust the initial similarity based on the association to obtain the similarity.

[0167] Example 3

[0168] An embodiment of the present application may provide an electronic device, Figure 8 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 8 As shown, the electronic device may include: one or more ( Figure 8 (only one is shown) processor 802, memory 804, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0169] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0170] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain the information to be detected, wherein the information to be detected includes text information and image information; determine the target weight of the image information based on the similarity between the text information and the image information, wherein the target weight is used to reflect the importance of the image information as a supplement to the text information; perform text enhancement on the text information based on the image information and the target weight to obtain enhanced information; perform detection on the information to be detected based on the enhanced information to obtain a detection result, wherein the detection result is used to indicate whether the information to be detected is information describing a real event.

[0171] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: input the image information into the information generation model, and use the information generation model to generate description information corresponding to the image information; based on the target weight, the description information and text information are integrated to obtain enhanced information.

[0172] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: based on the similarity between the text information and the description information, modify the target weight to obtain the modified weight; based on the modified weight, fuse the description information and the text information to obtain enhanced information.

[0173] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: input the correction weight into the fusion model, and use the fusion model to weightedly fuse the description information and text information to form enhanced information, wherein weighted fusion is used to indicate that the correction weight is used as an adjustment parameter in the process of fusing the description information and the text information, so that the target proportion is correlated with the correction weight, and the target proportion is used to indicate the ratio of the description information to the enhanced information.

[0174] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: fusing the enhanced information, text information and image information to obtain fused information; and detecting the information to be detected based on the fused information to obtain a detection result.

[0175] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: extract features from the text information and image information respectively to obtain text feature vectors and image feature vectors; determine the similarity based on the cosine value between the text feature vector and the image feature vector.

[0176] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: determine the initial similarity based on the cosine value between the text feature vector and the image feature vector; detect the first entity in the text information and the second entity in the image information; determine the association between the first entity and the second entity based on the knowledge graph of the field corresponding to the information to be detected, wherein the knowledge graph is used to describe different entities in the field and the association relationship between different entities; adjust the initial similarity based on the association to obtain the similarity.

[0177] The embodiment of the present application provides a scheme for an information detection method. By analyzing and calculating the importance of image information relative to text information, the correlation between image information and text information is clarified, so that the image information can be used to effectively enhance the text information, so as to achieve the purpose of effectively integrating the image information and text information and improving the accuracy of information detection, thereby solving the technical problem of low information detection accuracy in related technologies.

[0178] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 8 The structure of the electronic device is not limited. Figure 8 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 8 Different configurations shown.

[0179] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0180] Example 4

[0181] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the information detection method provided in the first embodiment.

[0182] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0183] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the program steps of the information monitoring method.

[0184] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0185] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0190] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An information detection method, characterized in that: include: Acquire information to be detected, wherein the information to be detected includes text information and image information; Determining a target weight of the image information based on the similarity between the text information and the image information, wherein the target weight is used to reflect the importance of the image information as a supplement to the text information; Performing text enhancement on the text information based on the image information and the target weight to obtain enhanced information; The information to be detected is detected based on the enhanced information to obtain a detection result, wherein the detection result is used to indicate whether the information to be detected is information describing a real event.

2. The method according to claim 1, characterized in that Performing text enhancement on the text information based on the image information and the target weight to obtain enhanced information includes: Inputting the image information into an information generation model, and using the information generation model to generate description information corresponding to the image information; The description information and the text information are fused based on the target weight to obtain the enhanced information.

3. The method according to claim 2, characterized in that The description information and the text information are fused based on the target weight to obtain the enhanced information, including: Based on the similarity between the text information and the description information, the target weight is modified to obtain a modified weight; The description information and the text information are fused based on the modified weight to obtain the enhanced information.

4. The method according to claim 3, characterized in that The description information and the text information are fused based on the modified weight to obtain the enhanced information, including: The correction weight is input into a fusion model, and the fusion model is used to perform weighted fusion on the description information and the text information to form the enhanced information, wherein the weighted fusion is used to indicate that the correction weight is used as an adjustment parameter in the process of fusing the description information and the text information, so that a target proportion is correlated with the correction weight, and the target proportion is used to indicate the ratio of the description information to the enhanced information.

5. The method according to claim 1, characterized in that Detecting the information to be detected based on the enhanced information to obtain the detection result includes: fusing the enhanced information, the text information and the image information to obtain fused information; The information to be detected is detected based on the fusion information to obtain the detection result.

6. The method according to claim 1, characterized in that The method further comprises: Extracting features from the text information and the image information respectively to obtain a text feature vector and an image feature vector; The similarity is determined based on a cosine value between the text feature vector and the image feature vector.

7. The method according to claim 6, characterized in that Determining the similarity based on a cosine value between the text feature vector and the image feature vector includes: Determining an initial similarity based on the cosine value between the text feature vector and the image feature vector; detecting a first entity in the text information and a second entity in the image information; Determine the degree of association between the first entity and the second entity based on a knowledge graph of the field corresponding to the information to be detected, wherein the knowledge graph is used to describe different entities in the field and the association relationship between the different entities; The initial similarity is adjusted based on the association degree to obtain the similarity.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

9. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 7 when running.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images