Computer program, authenticity verification device, and authenticity verification method

Abstract

To provide a computer program capable of verifying the authenticity of online social media information, an authentication verification device, and an authentication verification method.SOLUTION: A computer program causes a computer to perform processing for: acquiring text data; calculating a confidence factor for the authenticity of claims contained in the acquired text data for each of a plurality of unique information sources using the plurality of unique information sources; and calculating the credibility of the text data on the basis of the confidence factors calculated for each of the unique information sources.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a computer program, an authenticity verification device, and an authenticity verification method. [Background technology]

[0002] In recent years, online social media has been recognized as a new means of providing a platform for many users to participate in. However, seeking and disseminating information, as well as online deliberation and discussion, requires information literacy due to language barriers and differences in fields of expertise.

[0003] Patent Document 1 discloses a methodology that is based on cooperation between humans and conversational agents and that optimizes online discussions and deliberations by quantifying them. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Rafik Hadfi and Takayuki Ito, “In Proceeding of the 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS'22, page 1794-1798, Richland, SC, 2022. International Foundation for Autonomous Agents and Multiagent Systems.” Summary of the Invention [Problem to be solved by the invention]

[0005] In order to increase the reliability of the content of discussions and deliberations, it is often extremely important to have discussions and deliberations based on evidence of hypotheses and assumptions. However, the methodology of Patent Document 1 does not take into consideration a mechanism for verifying the truth of the discussions and deliberations. As a result, when fake news is posted on online social media, true and false information are mixed together, leading to a situation where users are misled.

[0006] The present invention has been made in view of the above circumstances, and aims to provide a computer program, an authenticity verification device, and an authenticity verification method that can verify the authenticity of information in online social media. [Means for solving the problem]

[0007] The present application includes multiple means for solving the above problem, and as one example, a computer program causes a computer to execute a process of acquiring text data, using multiple unique information sources, calculating a degree of certainty regarding the veracity of claims contained in the acquired text data for each of the multiple unique information sources, and calculating the credibility of the text data based on the degree of certainty calculated for each of the multiple unique information sources. [Effects of the Invention]

[0008] According to the present invention, the authenticity of information in online social media can be verified. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of the configuration of an authenticity verification system according to an embodiment of the present invention. [Figure 2] FIG. 10 illustrates an example of the functions of a main agent. [Figure 3] FIG. 10 is a diagram illustrating an example of a function of a first agent. [Figure 4] FIG. 10 is a diagram illustrating an example of processing by a claim content extraction unit. [Figure 5] FIG. 10 is a diagram illustrating an example of processing by a question generation unit. [Figure 6] FIG. 10 is a diagram illustrating an example of processing by a search unit. [Figure 7] FIG. 10 is a diagram illustrating an example of processing by a certainty factor calculation unit. [Figure 8] FIG. 10 is a diagram illustrating an example of processing by a main agent. [Figure 9] FIG. 10 is a diagram illustrating an example of weighted scores calculated by each software agent for multiple assertions. [Figure 10] FIG. 10 illustrates an example of weighted scores when different unique sources of information are used for a claim. [Figure 11] FIG. 10 is a diagram showing evaluation results of authenticity verification in a comparative example. [Figure 12] FIG. 10 is a diagram showing evaluation results of authenticity verification according to the present embodiment. [Figure 13] FIG. 1 illustrates an example of the configuration of an authenticity verification device. [Figure 14] FIG. 10 is a diagram illustrating an example of a processing procedure performed by the authenticity verification device. DETAILED DESCRIPTION OF THE INVENTION

[0010] The present invention will be described below with reference to the drawings showing embodiments thereof. Fig. 1 is a diagram showing an example of the configuration of an authenticity verification system according to this embodiment. The authenticity verification system includes an authenticity verification device 50. Connected to the authenticity verification device 50 via a communication network 1 are a social media (SNS) 10, a terminal device 20, a large-scale language model 30 as a learning model, an information source 81 for a first agent, an information source 82 for a second agent, and an information source 83 for a third agent.

[0011] Social media 10 is an internet media formed by users such as individuals and organizations exchanging information by transmitting, sharing, or spreading information, and users can post various information. Examples of social media 10 include Facebook, Mixi, LinkDon, Twitter, Skype, Instagram, YouTube, TikTok, and LINE (all of which are trademarks). When a user posts on social media 10, the social media 10 can transmit the posted content (text data) to the authenticity verification device 50. Alternatively, when a user posts on social media 10, the authenticity verification device 50 can acquire the posted content (text data).

[0012] The terminal device 20 can be configured as, for example, a personal computer, a smartphone, a tablet terminal, or the like, and can display the authenticity verification result obtained by the authenticity verification device 50.

[0013] The large-scale language model 30 is a natural language processing model trained using large amounts of text data. The large-scale language model 30 can be fine-tuned to adapt to various natural language processing tasks. The large-scale language model 30 is constructed, for example, to divide an input sentence into its smallest units (tokenization), calculate the relevance of each token in the prompt (context understanding), extract features (encode), predict the next token (decode), and calculate the probability of the next token in the input sentence. Given a large amount of text data, the large-scale language model 30 can learn the context and meaning of words from the tokens and output words and sentences that are considered to have a high probability of following a specific word. The large-scale language model 30 can be configured, for example, using Bidirectional Encoder Representations from Transformer (BERT), Generative Pre-trained Transformer (GPT-3), GPT-3.5, or GPT-4.

[0014] The authenticity verification device 50 can realize its main functions using a computer program described below, and includes functions such as a main agent 65, a first agent 61, a second agent 62, and a third agent 63. The main agent 65, the first agent 61, the second agent 62, and the third agent 63 are software agents that operate in an intermediary relationship with a user or other software and can act on their own judgment to achieve a given goal. The main agent 65, the first agent 61, the second agent 62, and the third agent 63 may be equipped with a learning model based on machine learning. The first agent 61, the second agent 62, and the third agent 63 are also referred to as multiple software agents. Details of the main agent 65, the first agent 61, the second agent 62, and the third agent 63 will be described later. The authenticity verification device 50 shown in FIG. 1 shows the functional configuration of the software.

[0015] The first agent information source 81 is a unique information source used by the first agent 61, the second agent information source 82 is a unique information source used by the second agent 62, and the third agent information source 83 is a unique information source used by the third agent 63. In other words, the first agent 61 uses the first agent information source 81 when executing its own processing, but does not use the second agent information source 82 or the third agent information source 83. The second agent 62 uses the second agent information source 82 when executing its own processing, but does not use the first agent information source 81 or the third agent information source 83. The third agent 63 uses the third agent information source 83 when executing its own processing, but does not use the first agent information source 81 or the second agent information source 82.

[0016] The information source 81 for the first agent, the information source 82 for the second agent, and the information source 83 for the third agent include, for example, Google (registered trademark), Wikipedia (registered trademark), other news sites, etc. Note that in the example of Fig. 1, three agents, the first agent 61, the second agent 62, and the third agent 63, are shown excluding the main agent 65, but the number of agents is not limited to three.

[0017] 1, the authenticity verification device 50 is configured to include a main agent 65, a first agent 61, a second agent 62, and a third agent 63, but the configuration is not limited to this. For example, the main agent 65, the first agent 61, the second agent 62, and the third agent 63 may be appropriately distributed and included in multiple devices. Furthermore, the agents may be integrated by including the functions of the first agent 61, the second agent 62, and the third agent 63 in the main agent 65.

[0018] 2 is a diagram showing an example of the functions of the main agent 65. The main agent 65 has the functions of a text data distribution unit 651, a confidence factor collection unit 652, and a credibility calculation unit 653.

[0019] The text data distribution unit 651 distributes text data (for example, content posted by a user) acquired from the social media 10 to the first agent 61, the second agent 62, and the third agent 63, respectively.

[0020] The certainty level collecting unit 652 collects the certainty levels output by the first agent 61, the second agent 62, and the third agent 63. The certainty levels are the degree of certainty regarding the truth or falsity of the assertion content included in the text data.

[0021] The credibility calculation unit 653 calculates the credibility of the text data based on the confidence levels output by the first agent 61, the second agent 62, and the third agent 63, respectively.

[0022] 3 is a diagram showing an example of the functions of the first agent 61. Since the first agent 61, second agent 62, and third agent 63 all have the same functions, the following will explain only the first agent 61. The first agent 61 has the functions of an assertion content extraction unit 611, a question generation unit 612, a search unit 613, and a confidence factor calculation unit 614.

[0023] The claim content extraction unit 611 extracts the claim content contained in the acquired text data. The claim content includes, for example, the user's argument or point of contention contained in the message posted by the user. Note that the number of claim content contained in the text data is not limited to one, and there may be multiple claim content.

[0024] The question generator 612 generates a question whose answer is the assertion contained in the text data.

[0025] The search unit 613 uses the question generated by the question generation unit 612 to search the first agent information source 81, which is an information source specific to the first agent 61, and outputs the search results.

[0026] The certainty factor calculation unit 614 compares the claim content extracted by the claim content extraction unit 611 with the search results, and calculates the certainty factor regarding the truth of the claim content.

[0027] 4 is a diagram showing an example of the processing of the claim content extraction unit 611. The claim content extraction unit 611 outputs the acquired text data to the large-scale language model 30. The large-scale language model 30 extracts the claim content contained in the input text data from the text data, and outputs the extracted claim content to the claim content extraction unit 611.

[0028] As described above, each of the multiple software agents (first agent 61, second agent 62, and third agent 63) can input the acquired text data into the large-scale language model 30 (learning model) and obtain the content of the claims made about the text data extracted by the large-scale language model 30.

[0029] As shown in the example of Figure 4, the content of the claim is something like, "Regarding OOOO, we can consider OOOO. There are three reasons: OOO, OOO, and OOO." In the example of Figure 4, for convenience, the content of the claim is a relatively short sentence, but the content of the claim is not limited to the example of Figure 4 and may actually be a long sentence.

[0030] 5 is a diagram showing an example of the processing of the question generation unit 612. The question generation unit 612 outputs the claim content acquired by the claim content extraction unit 611 to the large-scale language model 30. The large-scale language model 30 generates a question from the input claim content such that the claim content serves as an answer, and outputs the generated question to the question generation unit 612.

[0031] As described above, each of the multiple software agents can input the acquired assertion content into the large-scale language model 30 and obtain a question that the large-scale language model 30 generates, to which the assertion content serves as the answer.

[0032] As shown in the example of FIG. 5, the question is something like, "Please explain how you think about OOOO and give reasons." However, the question is not limited to the example of FIG.

[0033] 6 is a diagram showing an example of the processing of the search unit 613. The search unit 613 searches the first agent information source 81 using the question acquired by the question generation unit 612, and acquires the search results.

[0034] As described above, each of the multiple software agents can use the obtained question to search a unique information source and output search results.

[0035] As shown in the example of Figure 6, the search results may be something like "There are also opinions that OOOO is △△△△, but the opinion that XXXX is appropriate. The main reason is XXX." However, the search results are not limited to the example of Figure 6.

[0036] FIG. 7 is a diagram showing an example of the processing of the certainty calculation unit 614. The certainty calculation unit 614 outputs the claim content acquired by the claim content extraction unit 611 and the search results acquired by the search unit 613 to the large-scale language model 30. The large-scale language model 30 compares the input claim content with the search results, determines whether the search results are similar to the claim content, and outputs a certainty factor regarding the truth of the claim content. Here, the certainty factor regarding the truth of the claim content is the truth or falsity of the claim content and the certainty factor of said truth or falsity. For example, if the claim content is true, it is the "true" information and the "true" certainty factor, and if the claim content is false, it is the "false" information and the "false" certainty factor. The certainty factor can be expressed as a numerical value greater than or equal to 0 and less than or equal to 1.

[0037] As described above, each of the multiple software agents can input the acquired claim content and search results into the large-scale language model 30, and obtain and output the confidence level regarding the truth or falsity of the claim content calculated by the large-scale language model 30.

[0038] As shown in the example of Figure 7, if the content of the claim is "For OOOO, we can consider it as OOOO. There are three reasons: OOO, OOO, and OOO," and the search result is "There is also the opinion that OOOO is △△△△, but the opinion XXXX is more appropriate. The main reason is XXX," the large-scale language model 30 can determine the degree of agreement between the content of the claim and the search result and calculate the confidence level.

[0039] 8 is a diagram showing an example of the processing of the main agent 65. As described above, the main agent 65 distributes text data (e.g., user posts) acquired from the social media 10 to each of the first agent 61, second agent 62, and third agent 63. Also, as described above, each of the first agent 61, second agent 62, and third agent 63 calculates a degree of certainty regarding the truth of the assertion, and outputs the calculated degree of certainty to the main agent 65.

[0040] As described above, the main agent 65 outputs the acquired text data to each of the multiple software agents, and can obtain a degree of confidence regarding the truth or falsity of the claims contained in the text data calculated using the unique information sources output by each of the multiple software agents.

[0041] As shown in Figure 8, the confidence level regarding the truth or falsity of a claim can be a weighted score. The weighted score WS can be calculated as WS = N C, where N = 1 if the claim is true and N = -1 if the claim is false. The symbol C represents the confidence level and can be expressed as a number between 0 and 1.

[0042] FIG. 9 is a diagram showing an example of weighted scores calculated by each software agent for multiple claims. As shown in FIG. 9, the claims extracted from text data are denoted as a, b, c, .... The first agent 61, the second agent 62, and the third agent 63 determine the truth or falsity of the claims using their own information sources, i.e., a first agent information source 81, a second agent information source 82, and a third agent information source 83. The weighted scores calculated by the first agent 61 for the truth or falsity of the claims a, b, c, ... are denoted as 1a, 1b, 1c, .... The weighted scores calculated by the second agent 62 for the truth or falsity of the claims a, b, c, ... are denoted as 2a, 2b, 2c, .... The weighted scores calculated by the third agent 63 for the truth or falsity of the claims a, b, c, ... are denoted as 3a, 3b, 3c, ....

[0043] FIG. 10 is a diagram showing an example of weighted scores when different unique information sources are used for each claim. FIG. 10 shows the weighted scores shown in FIG. 9 categorized by claim content. That is, multiple software agents calculate weighted scores individually for each claim content. In the example of FIG. 10, for claim content a, the first agent 61, the second agent 62, and the third agent 63 calculate weighted scores 1a, 2a, and 3a, respectively. For claim content b, the first agent 61, the second agent 62, and the third agent 63 calculate weighted scores 1b, 2b, and 3b, respectively. For claim content c, the first agent 61, the second agent 62, and the third agent 63 calculate weighted scores 1c, 2c, and 3c, respectively.

[0044] Returning to FIG. 8, the main agent 65 calculates the credibility of the text data for each assertion based on the weighted scores calculated by the first agent 61, the second agent 62, and the third agent 63.

[0045] As described above, the authenticity verification device 50 acquires text data, and using a plurality of unique information sources, calculates the degree of certainty regarding the authenticity of the claims contained in the acquired text data for each of the plurality of unique information sources, and can calculate the credibility of the text data based on the degree of certainty calculated for each of the plurality of unique information sources.

[0046] This allows authenticity verification to be performed from a variety of perspectives by using multiple unique information sources, making it possible to verify the authenticity of information on online social media.

[0047] Next, a method for calculating the credibility will be described.

[0048] The accuracy of the assertion is represented by CS, and the inaccuracy of the assertion is represented by IS. Accuracy CS and inaccuracy IS can be expressed by equations (1) and (3). Equation (1) is conditional on equation (2), and equation (3) is conditional on equation (4).

[0049]

number

[0050] Here, C is a set of claims, which includes all claims extracted from the text data. i represents the i-th assertion. Symbol a represents the software agent. Assertion c i The set of software agents who judged the value to be "true" is called T i The content of the claim is c i The set of software agents who judged the value to be "false" is called F. i Let C(a) be the confidence calculated by software agent a. The log is used for normalization.

[0051] Equation (1) represents the average confidence of software agents who judged the assertion to be "true," and equation (3) represents the average confidence of software agents who judged the assertion to be "false."

[0052] The weighted score WS calculated by each software agent is calculated as WS = N·S. Here, if the assertion is true, N = 1, and if the assertion is false, N = -1. Symbol C is the confidence level, which can be expressed as a number between 0 and 1. Therefore, equations (5), (6), and (7) hold. Symbol A is the set of all software agents.

[0053]

number

[0054] Claim content c i Veracity score of (c i ) can be calculated using equation (8). If all software agents judge the assertion to be "true", then T i =A, and the truth score VS(c i ) reaches an upper limit. Similarly, if all software agents judge the assertion to be false, then Fi =A, and the truth score VS(c i ) reaches a lower limit. The truthfulness score VS(c i ) is a number greater than or equal to 0 and less than or equal to 1.

[0055] If the credibility of text data P is MAFC(P), the credibility MAFC(P) can be calculated using equation (9). The credibility MAFC(P) is a numerical value between 0 and 1.

[0056]

number

[0057] In equation (9), if C=1, the authenticity verification device 50 can calculate the credibility of the text data based on the average of the certainty levels when the claim content is true and the average of the certainty levels when the claim content is false.

[0058] In equation (9), if C≠1, when there are multiple claims, the authenticity verification device 50 calculates the average certainty for each claim based on the average certainty when the claim is true and the average certainty when the claim is false, and can calculate the credibility of the text data based on the average certainty for each claim.

[0059] Next, the evaluation results of the authenticity verification performed by the authenticity verification device 50 will be described.

[0060] FIG. 11 shows the evaluation results of authenticity verification in the comparative example. The comparative example shows a case where each software agent performs authenticity verification using its own unique information source. In the example of FIG. 11, a first agent 61, a second agent 62, and a third agent 63 each use their own information source for the first agent 81, information source for the second agent 82, and information source for the third agent 83. The F-measure (F-score) is one of the evaluation indices for binary classification tasks. It focuses on the trade-off between precision and recall and is the harmonic mean of the two values. The F-measure can be calculated using the formula: F-measure = 2 × precision × recall / (precision + recall). As shown in FIG. 11, the F-measures are 0.51, 0.52, and 0.53.

[0061] FIG. 12 is a diagram showing the evaluation results of authenticity verification in this embodiment. When the aforementioned credibility MAFC(P) is used, the F-score is 0.63. It can be seen that the accuracy of authenticity verification is higher than that of the comparative example shown in FIG. 11. Specifically, (1) MAFC is calculated for certain text data, and if the text data is determined to be "false," and the ground truth of the text data is "false," it is determined to be "correct." Otherwise, it is determined to be "incorrect." (2) MAFC is calculated for certain text data, and if the text data is determined to be "true," and the ground truth of the text data is "true," it is determined to be "correct." Otherwise, it is determined to be "incorrect." (3) The results of applying the above-mentioned (1) and (2) to all the data included in the dataset can be calculated, and the F-score can be calculated.

[0062] When using the voting method, the F-measure was 0.55, indicating that the accuracy of truth verification was slightly higher than that of the comparative example shown in Figure 11. The voting method can calculate the credibility of text data based on the confidence level of each unique information source, whichever is greater: the number of unique information sources (which may be the number of software agents) when the assertion is true, or the number of unique information sources (which may be the number of software agents) when the assertion is false. Specifically, (1) if two of three software agents judge a certain text data to be "false" by majority vote, and the ground truth of the text data is "false," the text data is deemed "correct." Otherwise, the text data is deemed "incorrect." (2) if two of three software agents judge a certain text data to be "true" by majority vote, and the ground truth of the text data is "true," the text data is deemed "correct." Otherwise, the text data is deemed "incorrect." (3) The F-measure can be calculated by applying the above (1) and (2) to all the data contained in the dataset.

[0063] Fig. 13 is a diagram showing an example of the configuration of an authenticity verification device 50. The authenticity verification device 50 includes a control unit 51 that controls the entire device, a communication unit 52, a memory 53, a storage unit 54, and a recording medium reading unit 55. The authenticity verification device 50 shown in Fig. 13 shows a hardware configuration.

[0064] The control unit 51 can be configured by combining hardware such as a CPU (e.g., a multi-processor having multiple processor cores), GPUs (Graphics Processing Units), DSPs (Digital Signal Processors), and FPGAs (Field-Programmable Gate Arrays).

[0065] The communication unit 52 is equipped with a communication module and is capable of communicating with the social media 10, the terminal device 20, the information source for the first agent 81, the information source for the second agent 82, and the information source for the third agent 83 via the communication network 1.

[0066] The storage unit 54 can be configured with, for example, a hard disk or semiconductor memory, and stores a computer program (program product) 60 and required information.

[0067] The computer program 60 is a computer program that runs on the authenticity verification device 50, and a recording medium M (for example, an optically readable disk storage medium such as a CD-ROM) on which the computer program 60 is recorded is read by the recording medium reading unit 55, and the read computer program 60 is stored in the storage unit 54. The computer program 60 may also be downloaded from an external device via the communication unit 52 and stored in the storage unit 54.

[0068] The computer program 60 can implement the functionality of each of the aforementioned main agent 65, first agent 61, second agent 62, and third agent 63. The computer program 60 can be deployed to run on a single computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0069] The memory 53 can be configured with semiconductor memory such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc. A computer program 60 can be loaded into the memory 53, and the control unit 51 can execute the computer program 60. The control unit 51 can execute processing defined by the computer program 60. In other words, processing by the control unit 51 is also processing by the computer program 60.

[0070] 14 is a diagram showing an example of a processing procedure by the authenticity verification device 50. The control unit 51 acquires text data to be verified (S11) and outputs the acquired text data to a plurality of agents (S12). The plurality of agents are, for example, a first agent 61, a second agent 62, and a third agent 63.

[0071] The control unit 51 inputs the text data into a learning model (e.g., large-scale language model 30) and acquires the claim content extracted by the learning model (S13). The control unit 51 inputs the acquired claim content into the learning model and acquires the question generated by the learning model (S14).

[0072] The control unit 51 searches for unique information sources using the acquired question and outputs the search results (S15), and inputs the claim content and the search results into a learning model to obtain the confidence level regarding the truth or falsity of the claim content calculated by the learning model (S16).

[0073] The control unit 51 calculates the weighted score (S17) and determines whether there are other claims (S18). If there are other claims (YES in S18), the control unit 51 repeats the processing from step S14 onwards. If there are no other claims (NO in S18), the control unit 51 calculates the credibility of the text data (S19) and ends the processing.

[0074] As mentioned above, when verifying authenticity based on a single source, (1) it is assumed that the source is authoritative, (2) as a result, there is a tendency to be overconfident in the results of the verification, and (3) the content (text data) posted on online social media is a mixture of rumors and false information, and the content is complex and contains errors, so it is important to detect the reliability of that false information. Therefore, a simple binary classification task is not sufficient to calculate the credibility of text data.

[0075] However, according to this embodiment, by using a plurality of unique information sources, authenticity verification can be performed from a variety of perspectives, making it possible to verify the authenticity of information in online social media.

[0076] Furthermore, according to this embodiment, the credibility of text data is calculated based on the MAFC value, rather than simply on a binary classification task, and therefore the accuracy of authenticity verification can be improved.

[0077] (Supplementary Note 1) The computer program causes a computer to execute a process of acquiring text data, calculating, using a plurality of unique information sources, a degree of certainty regarding the veracity of claims contained in the acquired text data for each of the plurality of unique information sources, and calculating the credibility of the text data based on the degree of certainty calculated for each of the plurality of unique information sources.

[0078] (Appendix 2) The computer program in Appendix 1 causes a computer to execute a process of outputting the acquired text data to each of a plurality of software agents, and acquiring a degree of certainty regarding the truth of the assertion contained in the text data calculated using the unique information source output by each of the plurality of software agents.

[0079] (Appendix 3) In the computer program of Appendix 2, each of the plurality of software agents causes a computer to execute a process of inputting acquired text data into a learning model and obtaining the content of claims made about the text data that the learning model extracts.

[0080] (Appendix 4) In the computer program of Appendix 3, each of the plurality of software agents causes a computer to execute a process of inputting the acquired content of a claim into the learning model, which generates a question to which the content of the claim is the answer.

[0081] (Supplementary Note 5) The computer program according to Supplementary Note 4 causes a computer to execute a process in which each of the plurality of software agents searches the unique information source using the question and outputs a search result.

[0082] (Appendix 6) In the computer program of Appendix 5, each of the plurality of software agents inputs the acquired content of the claim and the search results into the learning model, and obtains and outputs a degree of confidence regarding the truth of the content of the claim calculated by the learning model.

[0083] (Supplementary Note 7) In the computer program according to any one of Supplementary Note 3 to Supplementary Note 6, the learning model includes a large-scale language model.

[0084] (Appendix 8) The computer program causes a computer to execute a process in any one of Appendices 1 to 7 to calculate the credibility of the text data based on an average of the confidence levels when the content of the claim is true and an average of the confidence levels when the content of the claim is false.

[0085] (Appendix 9) The computer program in Appendix 8 causes a computer to execute a process to, when there are multiple claims, calculate an average degree of certainty for each claim based on an average degree of certainty when the claims are true and an average degree of certainty when the claims are false, and to calculate the credibility of the text data based on the average degree of certainty for each claim.

[0086] (Appendix 10) The computer program in any one of Appendices 1 to 9 causes a computer to execute a process of calculating the credibility of the text data based on the confidence level for each of the unique information sources for the larger of the number of the unique information sources when the content of the claim is true and the number of the unique information sources when the content of the claim is false.

[0087] (Appendix 11) The authenticity verification device includes a control unit that acquires text data, calculates a degree of certainty regarding the authenticity of claims contained in the acquired text data for each of the plurality of unique information sources using the plurality of unique information sources, and calculates the credibility of the text data based on the degree of certainty calculated for each of the plurality of unique information sources.

[0088] (Appendix 12) The authenticity verification method acquires text data, and calculates, using a plurality of unique information sources, a degree of certainty regarding the authenticity of claims contained in the acquired text data for each of the plurality of unique information sources, and calculates the credibility of the text data based on the degree of certainty calculated for each of the plurality of unique information sources.

[0089] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, although the claims use a format in which a claim references two or more other claims (multiple claim format), this is not limited to this format. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0090] 1. Communication Network 10. Social Media 20 Terminal equipment 30 Large-scale language models 50 Authenticity Verification Device 51 Control section 52 Communications Department 53 Memory 54 Memory section 55 Recording medium reading unit 60 Computer Programs 61 First Agent 611 Claim content extraction part 612 Question generation part 613 Search Department 614 Confidence calculation unit 62 Second Agent 63 Third Agent 65 Main Agent 651 Text Data Distribution Department 652 Belief Collection Department 653 Credibility Calculation Department 81 Primary Agent Source 82 Information Source for Secondary Agent 83 Sources of Information for Third Agents

Claims

1. Get the text data Using a plurality of unique information sources, a degree of certainty regarding the truth of the assertions contained in the acquired text data is calculated for each of the plurality of unique information sources; calculating the credibility of the text data based on the confidence level calculated for each of the plurality of unique information sources; A computer program that causes a computer to perform a process.

2. The acquired text data is output to each of the multiple software agents, obtaining a degree of certainty regarding the truth of the assertion contained in the text data calculated using the unique information source output by each of the plurality of software agents; 2. The computer program according to claim 1, which causes a computer to execute a process.

3. Each of the plurality of software agents inputs the acquired text data into a learning model to obtain the assertion content for the text data that is extracted by the learning model.

3. A computer program according to claim 2, which causes a computer to execute a process.

4. Each of the plurality of software agents inputs the acquired assertion content into the learning model, and acquires a question that the learning model generates, the question being answered by the assertion content.

4. A computer program according to claim 3, which causes a computer to execute a process.

5. each of the plurality of software agents searches the unique information source using the query and outputs a search result; 5. A computer program according to claim 4, which causes a computer to execute a process.

6. each of the plurality of software agents inputs the acquired content of the claim and the search results into the learning model, and acquires and outputs a degree of confidence regarding the truth of the content of the claim calculated by the learning model; 6. A computer program according to claim 5, which causes a computer to execute a process.

7. the training model includes a large-scale language model; A computer program according to any one of claims 3 to 6.

8. calculating the credibility of the text data based on an average of the confidence levels when the content of the claim is true and an average of the confidence levels when the content of the claim is false; 7. A computer program according to claim 1, which causes a computer to execute a process.

9. If there are multiple claims, calculate an average confidence level for each claim based on the average confidence level when the claim is true and the average confidence level when the claim is false; Calculating the credibility of the text data based on the average of the confidence levels for each of the claims.

9. A computer program according to claim 8, which causes a computer to execute a process.

10. Calculating the credibility of the text data based on the degree of certainty for each of the unique information sources, which is the larger of the number of the unique information sources when the content of the claim is true and the number of the unique information sources when the content of the claim is false.

7. A computer program according to claim 1, which causes a computer to execute a process.

11. A control unit is provided, The control unit Get the text data Using a plurality of unique information sources, a degree of certainty regarding the truth of the assertions contained in the acquired text data is calculated for each of the plurality of unique information sources; calculating the credibility of the text data based on the confidence level calculated for each of the plurality of unique information sources; Authenticity verification device.

12. Get the text data Using a plurality of unique information sources, a degree of certainty regarding the truth of the assertions contained in the acquired text data is calculated for each of the plurality of unique information sources; calculating the credibility of the text data based on the confidence level calculated for each of the plurality of unique information sources; Authenticity verification method.

Images