Reliability evaluation device and program

The reliability evaluation device on SNS assesses information credibility by analyzing citation relationships, addressing the inadequacies of conventional methods by quantifying reliability through independent source identification, thereby reducing the spread of misinformation.

JP2024168178APending Publication Date: 2024-12-05NIPPON HOSO KYOKAI
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023084627
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional techniques are inadequate for accurately evaluating the reliability of online information, particularly in social networking services (SNS), as they either rely on time-consuming manual verification of multiple sources or fail to provide a comprehensive measure of reliability.

Method used

A reliability evaluation device that analyzes the citation relationships of posted information on SNS, identifying similar and cited information to determine the number of independent sources, thereby evaluating the reliability of the information based on the 'two-source rule'.

Benefits of technology

The device provides a quick and reliable assessment of information credibility by identifying independent sources, helping users avoid the spread of misinformation and enhancing the accuracy of information dissemination.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024168178000001_ABST
    Figure 2024168178000001_ABST
Patent Text Reader

Abstract

To provide a technology to evaluate reliability of information.SOLUTION: The present disclosure is a reliability evaluation device 10 that can communicate with a terminal device 30 displaying information via a network, and includes: a posted information acquisition unit that acquires first information displayed by the terminal device, of a first information sequence where a relation that information cites citing information is repeated once or more; a similar information identification unit that identifies second information similar to the first information from a second information sequence where a relation that information cites citing information is repeated once or more; and a reliability evaluation unit that evaluates reliability of the first information on the basis of whether or not first citing information cited in the first information sequence with the first information as a starting point overlaps with second citing information cited in the second information sequence with the second information as a starting point.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a reliability evaluation device and a program. [Background technology]

[0002] Information about incidents, accidents, etc. may be distributed over the Internet (hereafter referred to as online information). An increasing number of users are obtaining online information from SNS (Social Networking Services) before television and other mass media. For example, there are many situations in which SNS users learn about an incident that occurred in a specific area, such as a fire, through a post from a poster living in that area before it is reported in the mass media. This is fine if the online information comes from a news organization that is recognized as being "highly reliable," but when unspecified users view online information from unknown anonymous posters, it is often not easy to determine whether the information is true. If users who view online information easily decide that the information is true and repost it, this could lead to the spread of fake news and the like.

[0003] Journalists at news organizations sometimes impose rules on themselves to determine whether information is trustworthy. For example, there is the "two source rule" which states that "information obtained from an anonymous source will not be published unless it can be corroborated by another source" (Non-Patent Document 1). However, even if such rules are considered effective, when users actually apply the rules, they have to track down multiple sources themselves and confirm that the sources are different, which is time-consuming.

[0004] A technology has been proposed for determining the actual situation based on information from multiple information sources (see, for example, Patent Document 1). Also, a technology has been proposed for extracting rumor information published on a network about a specific target (see, for example, Patent Document 2). Also, on SNS and the like, actions taken by users other than the poster regarding posted information (e.g., reposting, giving a "like", etc.) are sometimes displayed as numbers or the like (see, for example, Non-Patent Document 2). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-140644 A [Patent Document 2] JP 2008-165599 A [Non-patent literature]

[0006] [Non-Patent Document 1] Bill Kovach, Tom Rosenstiel, Nobuyuki Okumura, Intelligence Journalism, Minerva Publishing, 2015 [Non-Patent Document 2] https: / / twitter.com / Summary of the Invention [Problem to be solved by the invention]

[0007] However, conventional technologies are not necessarily suitable methods for evaluating the reliability of information. For example, the technology described in Patent Document 1 is a technology that uses the difference between information from multiple information sources when the information contains contradictory content to determine the state of a disaster, but is not a technology for evaluating the reliability of online information. The technology described in Patent Document 2 uses machine learning to determine whether the data extracted from the Internet is rumor information, and it is difficult to evaluate the reliability when the data is new. The technology described in Non-Patent Document 2 can determine the attention of online information, but the value does not indicate its reliability.

[0008] In view of the above problems, the present disclosure provides a technique for evaluating the reliability of information. [Means for solving the problem]

[0009] In view of the above problems, the present disclosure provides a reliability evaluation device capable of communicating with a terminal device that displays information via a network, the device comprising: a posted information acquisition unit that acquires first information displayed by the terminal device from a first information series in which information repeats a relationship in which information cites cited information one or more times; a similar information identification unit that identifies second information similar to the first information from a second information series in which information repeats a relationship in which information cites cited information one or more times; and a reliability evaluation unit that evaluates the reliability of the first information based on whether or not first cited information cited in the first information series starting from the first information overlaps with second cited information cited in the second information series starting from the second information. Effect of the Invention

[0010] The present disclosure can provide a technique for evaluating the reliability of information. [Brief description of the drawings]

[0011] [Figure 1] FIG. 13 is a diagram showing an example of a screen of an application for posting information. [Diagram 2] FIG. 2 is a diagram for explaining an outline of a process in which the reliability evaluation device evaluates the reliability of online information; [Diagram 3] FIG. 1 is a diagram illustrating an example of a system configuration of a reliability evaluation system. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a reliability evaluation device, an SNS server, and a terminal device. [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a reliability evaluation system, an SNS server, and a terminal device. [Figure 6] FIG. 2 is a diagram illustrating posted information posted to an SNS server. [Figure 7] FIG. 13 is a diagram for explaining an evaluation example in which reliability is evaluated as high. [Figure 8] FIG. 13 is a diagram illustrating an example of an evaluation in which reliability is evaluated as low. [Figure 9] FIG. 11 is a diagram for explaining detailed processing of generating a sequence and evaluating its reliability. [Figure 10] FIG. 13 is a diagram illustrating an example of a search for a sequence evaluated to be unreliable. [Figure 11] 11 is a sequence diagram illustrating an example of a process flow in which the reliability evaluation device evaluates the reliability of posted information. [Figure 12] FIG. 12 is an example of a flowchart illustrating the process of step S7 in FIG. 11 in detail. [Figure 13] FIG. 13 is a diagram showing an example of a posted information display screen displayed on a terminal device by a user. [Figure 14] FIG. 13 is a diagram showing an example of a message displayed when the reliability of posted information is evaluated as low. [Figure 15] 11 is a flowchart illustrating a process flow in which the reliability evaluation device evaluates the reliability of the posted information by narrowing down the number of posted information similar to the posted information displayed by the terminal device; and [Figure 16] FIG. 1 is a diagram for explaining a schematic diagram of distance in an SNS space. [Figure 17] FIG. 13 is a diagram for explaining the basis for determining the number of edges from a user to another user as the reliability. [Figure 18]FIG. 11 is a flowchart illustrating an example of a process flow in which the reliability evaluation device evaluates the reliability of posted information based on the distance between contributors who have posted information sources that are independent of each other. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] A reliability evaluation system and a reliability evaluation method performed by the reliability evaluation system will be described below as an example of an embodiment of the present invention.

[0013] <Outline of reliability evaluation> First, an example of an application for posting information will be described as an example of the spread of information on the Internet with reference to Figure 1. Such an application is used, for example, in SNS, and allows users to post text and images with simple operations.

[0014] 1 shows an example of a screen of an application for posting information. The user inputs a message 101 such as "I'm at ____." When the user presses the post button 103, the message 101 is sent to the SNS server.

[0015] A timeline 104 is displayed on the screen of FIG. 1. Posts posted by the user and posts posted by users the user follows are displayed on the timeline 104. Poster A, whom the user follows, reposts a post 102 posted by poster B on the timeline 104. Therefore, by following poster A, the user can view poster B's post 102. In this way, the application is excellent for sharing and spreading information.

[0016] There are two types of reposting: In this embodiment, both types are simply referred to as "reposting." Reposting other people's posts exactly as they are · Quote and repost other people's posts with your comments While such reposting functions are effective for sharing and spreading information, they are also known to contain fake news (fake news articles created to be widely shared online), and it is often not easy to determine the authenticity of information. Therefore, in this embodiment, we believe that the "two source rule" explained in Non-Patent Document 1 is effective, and evaluate the reliability of online information as follows.

[0017] FIG. 2 is a diagram for explaining an outline of a process in which the reliability evaluation device 10 of this embodiment evaluates the reliability of online information.

[0018] (1) User 9 posts information P on terminal device 30. A (An example of the first information) is displayed. A When is displayed, post information P A The reliability evaluation process is performed.

[0019] (2) The reliability evaluation device 10 evaluates the posted information P A Other posts with similar content to B (an example of the second information) is searched using an index such as inter-document similarity, and the posted information P B are judged to be "similar posts."

[0020] (3) The reliability evaluation device 10 evaluates the posted information P A Refer to the citation information (such as the original post ID and citation URL) contained in the post information P A Cited contribution information P cited by a1 (an example of first cited information). The reliability evaluation device 10 similarly identifies the cited posted information P a1 Refer to the citation information contained in the cited post information P a1 Cited contribution information P cited by a2 The reliability evaluation device 10 performs the process of tracing the cited posted information from the first posted information (hereinafter, the information source R A This is repeated until a sequence of posted information (an example of a first information sequence) is generated. Ais information that does not quote posted information (it may be posted information or a website, etc.). B Similarly, for the cited submission information P b1 , P b2 The process of tracing (an example of the second cited information) is performed using information source R. B This is repeated until a sequence of posted information (an example of a second information sequence) is generated. B Even if you trace it back, the post information P A The series of posted information cited by and posted information P B If there is no overlap of cited posted information in the series of posted information cited by the posted information P, the reliability evaluation device 10 A , P B Source of information A ,R B is different, and the post information P A is considered to be highly reliable.

[0021] (4) Posting information P A Or P B When cited submissions are duplicated in the process of tracing the cited submissions of (in the figure, b2 The quote is from source R A The reliability evaluation device 10 evaluates the posted information P A , P B Source of information A is the same, and the posted information P A is rated as having low reliability.

[0022] In this way, the reliability evaluation device 10 can present the reliability to the user 9 when the user 9 displays the posted information. The user 9 can use the reliability as a reference to evaluate how reliable the posted information is and use it as a reference when taking action. For example, it becomes possible to avoid actions such as reposting unreliable posted information.

[0023] <Terminology> When information cites a cited piece of information, it means that the information contains the fact that it cites the cited piece of information. It does not mean that the information simply contains part or all of the cited piece of information, or is similar to it. In addition, citation means that a piece of information uses other information.

[0024] Trustworthiness is the ability to fulfill the role expected of it. In the case of information reliability, it is a measure to judge whether the information is true or not. The degree of reliability quantified by some index is called the degree of reliability.

[0025] <System configuration example> Fig. 3 is an example of a system configuration diagram of the reliability evaluation system 100. The reliability evaluation system 100 includes a reliability evaluation device 10 and a terminal device 30. However, the terminal device 30 may be a general-purpose computer, and may not be included in the reliability evaluation system 100. Also, Fig. 3 shows an SNS server 50, which may or may not be included in the reliability evaluation system 100.

[0026] The reliability evaluation device 10, the terminal device 30, and the SNS server 50 are communicably connected via a wide area network N1 such as the Internet. It is assumed that the reliability evaluation device 10 and the SNS server 50 are installed in a data center or the like. The reliability evaluation device 10 and the SNS server 50 may be compatible with cloud computing. Cloud computing refers to a form of use in which resources on a network are used without being aware of specific hardware resources. Therefore, the reliability evaluation device 10 and the SNS server 50 do not need to be housed in a single housing or provided as a single integrated device.

[0027] The SNS server 50 is a server device of the SNS that processes posted information. The SNS server 50 accepts logins from users and provides the user with posted information from posters that the user follows as a timeline. The SNS server 50 also provides the posted information posted by the user to the timelines of followers who follow the user. This process is repeated to spread the posted information.

[0028] The trustworthiness evaluation device 10 communicates with the SNS server 50 via the network N1. When a user browses posted information, the trustworthiness evaluation device 10 evaluates the trustworthiness of the posted information and transmits it to the terminal device 30. Since the trustworthiness evaluation device 10 processes the posted information posted to the SNS server 50 in this manner, it may be integrated with the SNS server 50. Alternatively, the SNS server 50 may have the functions of the trustworthiness evaluation device 10.

[0029] The functions of the reliability evaluation device 10 may be distributed among multiple information processing devices, or each of the multiple information processing devices may have all the functions of the reliability evaluation device 10, and the information processing device that processes the data may be switched by load balancing or the like.

[0030] The reliability evaluation device 10 may be a Web server that returns processing results to the terminal device 30 in response to a request from the terminal device 30. A server is a computer or software that provides information and processing results in response to a request from a client.

[0031] The user may enter into a contract with the trustworthiness evaluation device 10 or the SNS server 50 to use the trustworthiness evaluation device 10. In other words, the user pays a usage fee for the trustworthiness evaluation device 10. It is advantageous for the user to know the trustworthiness of posted information, and the trustworthiness evaluation device 10 and the SNS server 50 can also allocate the usage fee to resources for evaluating trustworthiness. Therefore, the user may also log in to the trustworthiness evaluation device 10.

[0032] The terminal device 30 is a general-purpose information processing device used by a user who views and posts posted information. The terminal device 30 is connected to a network N2. The network N2 may be a LAN, Wi-Fi (registered trademark), wide area Ethernet (registered trademark), a mobile phone network such as 4G, 5G, or 6G, etc. Here, a user is someone who uses an SNS provided by the SNS server 50.

[0033] In the terminal device 30, a web browser and a dedicated native application run on the SNS server 50. When the terminal device 30 runs a web browser, the terminal device 30 and the SNS server 50 run a web application. A web application is an application that runs by cooperation between a program written in a programming language (e.g., JavaScript (registered trademark)) that runs on the web browser and a program on the web server (here, the trust evaluation device 10) side.

[0034] In contrast, an application that is not executed unless installed in the terminal device 30 is called a native application. In this embodiment, too, the application executed on the terminal device 30 may be a Web application or a native application. In this case, the native application has the screen configuration, but the information to be displayed is acquired by the native application from the SNS server 50 and the reliability evaluation device 10. Therefore, there is no significant difference in the process flow between a Web application and a native application.

[0035] The terminal device 30 is, for example, a desktop PC, a notebook PC, a smartphone, a PDA (Personal Digital Assistant), a tablet terminal, etc., used by a user. In addition, the terminal device 30 may be any device on which a web browser or a native application runs. The terminal device 30 may also be an electronic whiteboard, a car navigation system, a television receiver, etc.

[0036] <Hardware configuration example> The hardware configurations of the reliability evaluation device 10, the SNS server 50, and the terminal device 30 included in the reliability evaluation system 100 according to this embodiment will be described with reference to FIG.

[0037] <<Trustworthiness evaluation device, SNS server and terminal device>> Fig. 4 is a diagram showing an example of a hardware configuration of the reliability evaluation device 10, the SNS server 50, and the terminal device 30 according to this embodiment. As shown in Fig. 4, the reliability evaluation device 10, the SNS server 50, and the terminal device 30 are constructed by a computer 500, and include a CPU 501, a ROM 502, a RAM 503, a HD (Hard Disk) 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0038] Among these, the CPU 501 controls the operation of the entire computer 500. The ROM 502 stores programs such as IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls reading or writing of various data from the HD 504 under the control of the CPU 501. The display 506 displays various information such as a cursor, a menu, a window, a character, or an image. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using the network N2. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG. 4.

[0039] In addition, the keyboard 511 is a type of input means having a plurality of keys used for inputting characters, numerical values, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading or writing of various data with respect to the DVD-RW 513 as an example of a removable recording medium. Note that the DVD-RW drive 514 is not limited to DVD-RW and may be DVD-R or the like. The media I / F 516 controls reading or writing (storage) of data with respect to the recording medium 515 such as a flash memory.

[0040] <Regarding functions> Next, with reference to FIG. 5, the functional configuration of the reliability evaluation system 100 and the like according to the present embodiment will be described. FIG. 5 is a diagram showing an example of the functional configuration of the reliability evaluation system 100, the SNS server 50, and the terminal device 30 according to the present embodiment.

[0041] <<Functional configuration of SNS server>> The SNS server 50 has a post information processing unit 51 and an API 52. Each of these units is a function or means realized by operating according to an instruction from the CPU 501 in which any of the components shown in FIG. 4 is expanded in the RAM 503. Note that the SNS server 50 has various functions other than those shown, but the functions used in the present embodiment will be mainly described.

[0042] The post information processing unit 51 performs overall processing regarding post information posted by each poster on the SNS. The post information processing unit 51 provides the post information of the posters followed by a certain user to the user's terminal device 30 as a timeline, and provides the post information posted by the user to the timeline of the followers who follow this user.

[0043] The API 52 (Application Programming Interface) is an interface on the Internet that provides the posted information processed by the SNS server 50 to an external server or the like. That is, the reliability evaluation device 10 obtains the posted information and evaluates the reliability by requesting the posted information from the API 52 using a program having a predetermined description. In addition, the API 52 may have a function of returning posted information similar to posted information specified by a posting ID or the like, and citing posted information of posted information specified by a posting ID or the like.

[0044] <<Terminal Device>> The terminal device 30 is used by a user who browses and posts posted information. The terminal device 30 has a communication unit 31, a display control unit 32, and an operation reception unit 33. Each of these functional units is a function or means realized by the CPU 501 executing instructions included in one or more programs installed in the terminal device 30. Note that this program may be a Web application executed by a Web browser, or may be a dedicated native application.

[0045] The communication unit 31 transmits and receives various information between the SNS server 50 and the reliability evaluation device 10. In this embodiment, the communication unit 31 receives a Web application or posted information from the SNS server 50, and transmits posted text, images, user operation contents, etc. input by the user to the SNS server 50. In addition, the communication unit 31 receives the reliability of the posted information displayed by the terminal device 30 from the reliability evaluation device 10.

[0046] The display control unit 32 interprets screen information of various screens and displays it on the display 506. The operation receiving unit 33 receives various operations for applications displayed on the display 506 by the user.

[0047] <<Functional configuration of the reliability evaluation device>> The reliability evaluation device 10 has a communication unit 11, a posted information acquisition unit 12, a similar information identification unit 13, a group generation unit 14, an information source determination unit 15, and a reliability evaluation unit 16. Each of these units is a function or a means for performing a function that is realized when any of the components shown in FIG.

[0048] The communication unit 11 transmits and receives various information between the SNS server 50 and the terminal device 30. In this embodiment, the communication unit 31 receives a plurality of pieces of posted information from the SNS server 50, and transmits to the terminal device 30 the reliability of the posted information displayed by the user.

[0049] The posted information acquisition unit 12 communicates with the API 52 of the SNS server 50 via the communication unit 11, and acquires posted information from the API 52. There are various types of posted information to be acquired and the timing of acquisition depending on the processing load, the contents of the contract with the SNS server 50, etc. Examples of the posted information to be acquired and the timing of acquisition are as follows. The posted information acquisition unit 12 acquires all posted information at regular time intervals. When a user displays posted information, the posted information acquisition unit 12 receives a notification of the same from the terminal device 30 or the SNS server 50 and acquires other posted information for calculating the reliability of the posted information.

[0050] The similar information identifying unit 13 identifies posted information similar to the posted information displayed by the user from a plurality of posted information acquired from the SNS server 50. Identification of posted information is replaced by a process of searching for similar documents. That is, the similar information identifying unit 13 calculates similarity between documents and identifies documents with similarity equal to or greater than a threshold. When calculating the similarity of documents, it is possible to use a method of vectorizing text using a method such as TF-IDF (Term Frequency - Inverse Document Frequency), LSI (Latent Semantic Indexing), Word2Vec, Doc2Vec, Sent2Vec, and the like, and calculating the cosine similarity thereof, or a method such as WMD (Word Mover's Distance) or LC-RWMD (a method that significantly reduces the amount of calculation in exchange for accuracy compared to WMD). In this embodiment, a method suitable for calculating the similarity of posted information is used as appropriate.

[0051] The similar information identifying unit 13 can determine whether images are similar. Image comparison can be performed by comparing images that are almost the same but have small differences, or by comparing whether similar subjects or scenes are captured. In this embodiment, the latter comparison is performed because it is considered that when an incident such as a fire occurs, the same or different contributors will capture subjects or scenes related to the incident from various angles. This type of image processing is called scene recognition (what is the image as a whole?). In this embodiment, the similar information identifying unit 13 has a learning model that is constructed by deep learning or the like and that determines the scene from the image.

[0052] The series generation unit 14 generates a series of cited posted information by tracing the cited posted information to the information source based on the citation information (if the cited posted information overlaps, it is not necessary to trace it to the information source) starting from the posted information identified by the similar information identification unit 13. Details will be described later.

[0053] The information source determination unit 15 determines whether the information sources of each series were posted by the same poster by comparing the accounts of the posters who posted the information sources of each series. Even if the information sources are different, if multiple information sources are posted by the same poster, the reliability of the posted information displayed by the user decreases. Furthermore, if the information source is a website, the information source determination unit 15 determines whether the information sources are the same website by comparing the URLs of the websites of each series. Therefore, it is possible to determine whether the information sources are the same even if they are websites.

[0054] The reliability evaluation unit 16 evaluates the reliability of posted information displayed by a user based on, for example, the number of series with no overlapping cited posted information. The reliability evaluation unit 16 may take the number of series with no overlapping cited posted information itself as the reliability. Alternatively, the reliability evaluation unit 16 may compare the number of series with a threshold to determine the reliability in N (>2) stages, and calculate this stage as the "reliability." Simply put, if the "two information source rule" is valid, the reliability evaluation unit 16 evaluates that a number of independent series is two or more as reliable.

[0055] <Data Structure> Next, the data structure of the posted information will be described with reference to Fig. 6. Fig. 6 shows a schematic diagram of the posted information posted to the SNS server 50. Each of the posted information 60a to 60c has fields of a posted message ID, a posted message, a posting time, a quote source, a quote URL, and a poster ID. The posted information 60a quotes the posted information 60b, and the posted information 60b quotes the posted information 60c. Therefore, the posted information 60b is the quoted posted information of the posted information 60a, and the posted information 60c is the cited posted information of the posted information 60b. The post ID is identification information that uniquely identifies the posted information. An ID is a combination of a name, code, character string, number, etc. that is used to uniquely distinguish a specific object from multiple objects. The posted text is the posted content of the posted information. In the case of a simple repost that is not a quoted repost, there may be no posted text. The posted text may also be an image. Note that although the image is displayed on the terminal device 30, the image may be quoted via a URL. The posting time is the date and time when the posted information was posted. The source of a quote indicates that the posted information has been reposted, and is information for identifying the cited posted information. The source of a quote is an example of quoted information. - Citation URL is the URL of a website etc. when the poster posts by quoting it. Citation URL is an example of citation information. In the case of a repost, it is assumed that either the citation source or the citation URL is blank (if one of them is not blank, the quoted posted information is branched and traced). If both the citation source and citation URL are blank, this posted information is the information source. If the citation source is blank but there is a citation URL, the website indicated by the citation URL is the information source.

[0056] <Reliability evaluation example> Next, a method for evaluating the reliability of posted information will be described in detail with reference to Fig. 7 to Fig. 10. First, Fig. 7 is a diagram for explaining an evaluation example in which reliability is evaluated as high.

[0057] (1) First, user 9 posts information P A was displayed.

[0058] (2) The similar information identification unit 13 identifies the posted information P A Post information P with similar content to B is identified using indices such as document similarity.

[0059] (3) Next, the series generation unit 14 A By using the citation information held by A Cited Post Information P a1 Identify.

[0060] (4) The series generation unit 14 generates cited posting information P a1 Using the citation information owned bya1 Cited contribution information (in the figure, source R A ) is identified. A ) does not cite other posted information, the series generation unit 14 ends the search. A Source R A is information from an article on a news site.

[0061] By the above process, a certain posted information P A From that source R A Up to now, a series of posted information "P A →P a1 →R A " can be obtained.

[0062] (5) Next, the series generation unit 14 B The same process is carried out for the series "P B →P b1 →R B " is obtained.

[0063] In the example of Figure 7, information source R A , R B Since two sequences that do not intersect until the end are obtained, the reliability evaluation unit 16 determines that the posted information P A is rated as "obtained from two (independent) sources" and as "highly reliable."

[0064] FIG. 8 is a diagram for explaining an example of an evaluation in which the reliability is evaluated as low. A →P a1 →R A " and the series "P B →P b1 →R A " is obtained. In other words, if there are two series with the same cited contribution information (source R A In this case, the reliability evaluation unit 16 determines whether the posted information P A and P B is the same single source R AThe reliability evaluation unit 16 evaluates the series as coming from the same source and does not evaluate it as "highly reliable" (however, if there is a third series and it is independent, it is evaluated as being highly reliable). In this way, reliability is evaluated based on whether or not there is the same cited posted information in different series. When posted information in different series cites cited posted information of the same citation source or citation URL, the reliability evaluation unit 16 determines that the series are connected.

[0065] 9 is a diagram for explaining detailed processing of generating a sequence and evaluating its reliability. The explanation will be given according to the numbers shown in FIG.

[0066] (1) User 9 posts information P on terminal device 30. A was displayed (viewed).

[0067] (2) Next, the similar information identifying unit 13 identifies the posted information P A Post information P with similar content to B is searched for and extracted from the posted information acquired from the SNS server 50. A There is a method to vectorize the posted information using a method such as TF-IDF, vectorize other posted information, and calculate the document similarity between the posted information using cosine similarity, etc. In this case, if the document similarity is equal to or exceeds a certain threshold, the posted information is determined to be "similar."

[0068] (3) Next, the series generation unit 14 A , P B , P C For each of these, search for cited posts (or websites in the case of information sources) that they cite. For example, A Cited Post Information P a1 If the user has reposted the cited post information P a1 Posted information P A It is determined that the submitted information is cited by the A Cited contribution information P in series S1 starting with a1 Add.

[0069] (4)(5) Posting Information P B , P C Similarly, the series generation unit 14 searches for cited posted information for each of the cited posted information P b1 , P c1 In this manner, the group generating unit 14 recursively adds cited posted information (or websites) to the groups S1 and S2.

[0070] (6) Cited post information P a1 cites the news site NS, the group generating unit 14 adds the news site NS to the group S0.

[0071] (7) Next, in series S1, the cited contribution information P b1 If the news site NS, which is the cited posting information obtained by searching from , matches one that has already been added to another series S0, the series generation unit 14 determines that the series S1 has been connected to the series S0 at that point, and ends the search. Therefore, the series S0 and S1 have the same information source.

[0072] (8) On the other hand, in series S2, the cited contribution information P c1 However, if the cited post P has not been reposted or the website has not been cited, the series generation unit 14 ends the search. c1 is a source of information.

[0073] The series generating unit 14 repeats the processes (3) to (8) until the search is completed for all series (= until no cited posted information or citing websites are obtained).

[0074] (9) The reliability evaluation unit 16 finally evaluates the number of non-intersecting (independent) series as the reliability of the posted information P A The reliability of the data is evaluated as “2”.

[0075] FIG. 10 is a diagram showing an example of a search for a sequence that is evaluated as being unreliable. In FIG. 10, A Similar postsP B , PC Starting from this, three series are explored. However, the cited contribution information P a1 , P b1 , P c1 is one source of information R A is the cited posted information. In this case, all the series are connected to one cited posted information, so the number of series is 1. This shows how one particular posted information is spread through various routes. In such a case, since it shows information that does not satisfy the "two information source rule" described above, the reliability evaluation unit 16 judges the posted information P A It is preferable to display an alarm to the effect that the reliability of the

[0076] <Processing or Action> FIG. 11 is a sequence diagram illustrating the flow of a process in which the reliability evaluation device 10 evaluates the reliability of posted information.

[0077] S1: The user 9 inputs an operation to the terminal device 30 to display posted information.

[0078] S2: The operation reception unit 33 of the terminal device 30 receives the operation, and the communication unit 31 requests the posted information from the SNS server 50. The posted information displayed by the user 9 is, for example, a timeline (posted information of posters followed by the user 9). Note that the posted information displayed by the terminal device 30 is arbitrary, and the user 9 can view desired posted information by searching or the like.

[0079] S3: The posted information processing unit 51 of the SNS server 50 transmits, for example, recent posted information of posters that the user 9 is following to the terminal device 30.

[0080] S4: The communication unit 31 of the terminal device 30 receives the posted information, and the display control unit 32 displays the posted information.

[0081] S5: When the posted information is displayed, the communication unit 31 specifies the post ID of the displayed posted information and transmits a reliability evaluation request to the reliability evaluation device 10. In this manner, the communication unit 31 does not automatically transmit the reliability evaluation request to the reliability evaluation device 10, but may transmit the reliability evaluation request to the reliability evaluation device 10 in response to an operation by the user 9.

[0082] S6: The communication unit 11 of the reliability evaluation device 10 receives the posting ID and a reliability evaluation request. The similar information identification unit 13 acquires posted information similar to the posted information identified by the posting ID from the SNS server 50. The similar information identification unit 13 acquires posted information from a certain period of the past, for example, from the API 52. The similar information identification unit 13 identifies similar posted information from the posted information from the certain period of the past. Alternatively, the similar information identification unit 13 may specify the posting ID to the API 52 and acquire the posting ID of posted information similar to the posted information from the SNS server 50.

[0083] The processing flow in FIG. 11 is just an example, and instead of the terminal device 30 requesting a trust evaluation from the trust evaluation device 10, the SNS server 50 may notify the trust evaluation device 10 that a user 9 who has a contract with the trust evaluation device 10 has displayed posted information and the posting ID of the posted information.

[0084] S7: Next, the series generation unit 14 generates a series that starts from the posted information displayed by the terminal device 30 and two or more pieces of posted information similar to the posted information. First, the series generation unit 14 acquires posted information identified by a posting ID (posted information displayed by the terminal device 30 and similar posted information) from the API 52. Next, the series generation unit 14 uses these posted information as the starting points of the series and acquires, for each series, cited posted information of each posted information identified by the citation source or citation URL from the API 52. Details of the processing in step S7 will be described with reference to FIG. 12.

[0085] S8: When the search for all series is completed, the information source determination unit 15 determines whether the information sources of series with no overlapping cited posted information are from the same contributor and excludes them from independent series. The reliability evaluation unit 16 evaluates the reliability based on the number of series with no overlapping cited posted information.

[0086] S9: The communication unit 11 of the reliability evaluation device 10 transmits the reliability to be displayed in association with the posted information displayed by the terminal device 30 to the terminal device 30. This process can be carried out by the communication unit 11 designating the posting ID.

[0087] S10: The communication unit 31 of the terminal device 30 receives the reliability, and the display control unit 32 displays the reliability. It is preferable that the display control unit 32 displays the reliability in association with the posted information displayed in step S3.

[0088] After transmitting the reliability to the terminal device 30, the reliability evaluation device 10 preferably repeats steps S6 to S10 again at regular intervals or at least after a certain time has passed. This is because the number of contributors who post independent information sources may increase over time. For example, if a fire breaks out and contributor A posts information about the fire first, there will be only one series. However, if second and third contributors B and C who notice the fire post information about the fire separately, there will be three series. In this case, if the user 9 views the posted information at the time when only contributor A posts information about the fire, there is a risk that the user 9 will mistakenly think that the reliability is low. For this reason, the reliability evaluation device 10 can provide the user 9 with a more accurate reliability by repeating steps S6 to S10 at regular intervals.

[0089] Fig. 12 is a flow chart for explaining in detail the process of step S7 in Fig. 11. It is assumed that the post IDs of the posted information displayed by the terminal device 30 and other posted information similar thereto have been obtained.

[0090] First, the group generation unit 14 determines one of the posted information displayed by the terminal device 30 and posted information similar to it as one group to be generated (S21). That is, the group generation unit 14 selects one of the posted information displayed by the terminal device 30 and posted information similar to it.

[0091] Next, the group generation unit 14 searches for quoted posted information based on citation information contained in the posted information (or quoted posted information) (S22). The search refers to identifying the posted information (corresponding to the quoted posted information) that is the citation source or the website or the like identified by the citation URL when citation information (the citation source or the citation URL) is present.

[0092] The series generation unit 14 determines whether there is any cited posted information (S23). If there is no cited posted information (Yes in S23), the information source has been reached for one series, and the process proceeds to step S25.

[0093] If there is cited posted information (No in S23), the group generation unit 14 adds the cited posted information found to the group determined in step S21. The group generation unit 14 also determines whether the cited posted information in step S22 overlaps with the cited posted information included in other groups that have already been generated (S24). If there is an overlap of cited posted information (Yes in S24), the groups have been merged, so the process proceeds to step S25.

[0094] If there is no overlapping cited post information (No in S24), the process returns to step S22 to further search for the series.

[0095] Next, in step S25, group generation unit 14 determines whether or not the search for all groups has been completed (S25). The completion of the search for all groups means that there is no posted information that can be used as the starting point of a group. If groups remain, the process returns to step S21.

[0096] When the search for all series is completed, the information source determination unit 15 determines whether the poster is the same based on the poster ID of the information source of the independent series, and reduces the number of series by treating multiple series with the same poster as one series (S26). Similarly, when the information sources of the independent series are websites, the information source determination unit 15 determines whether the websites are the same, and reduces the number of series by treating multiple series from the website with the same information source as one series. In this way, even if the same poster makes multiple posts about the same event, the reliability evaluation device 10 can be prevented from erroneously evaluating them as being highly reliable.

[0097] <Example of screen displayed on terminal device> Next, examples of screens displayed by the terminal device 30 will be described with reference to Figs. 13 and 14. Fig. 13 shows an example of a message displayed when the reliability of posted information is evaluated as high. Fig. 13(a) shows a posted information display screen 110 displayed by the user 9 on the terminal device 30. The posted information display screen 110 shows that a posted message 111 posted by "Sato Ichiro" has been reposted 112 by "Yamada Taro". The posted message 111 posted by "Sato Ichiro" quotes website news 113 via a quoted URL.

[0098] 13(b) and (c) show other posted information similar to posted message 111 displayed by user 9 on terminal device 30. The posted information in FIG. 13(b) has posted message 115 posted by "Suzuki Jiro," and the posted information in FIG. 13(c) has posted message 116 posted by "Tanaka Saburo." The posted information in FIG. 13(c) quotes website news 117 via a quoted URL.

[0099] The reliability evaluation device 10 has determined that the series of posted information in Fig. 13(a) to (c) are independent. For this reason, reliability information 114 of "Reliability: 3 (at least three independent information sources seem to indicate similar information)" is displayed on the posted information display screen 110. The user 9 can evaluate the posted text 111 reposted by "Yamada Taro" as being reliable. Note that the reliability information 114 is not limited to numerical values, and may be icons. The icons indicate the presence or absence of reliability, large or small, or high, medium, or low (five or more levels may be used) by different shapes and colors.

[0100] Fig. 14 shows an example of a message that is displayed when the reliability of posted information is evaluated as low. Note that the explanation of Fig. 14 will mainly focus on the differences from Fig. 13. Fig. 14(a) shows a posted information display screen 120 displayed by the user 9 on the terminal device 30. The posted information display screen 120 shows that a posted text 121 posted by "Sato Ichiro" has been reposted 122 by "Yamada Taro."

[0101] 14(b) shows the information source 125 searched for by the reliability evaluation device 10. This information source 125 was one, regardless of the number of other posted information similar to the posted information displayed by the user 9 on the terminal device 30. In other words, the series are integrated into one, and the information source 125 is an information source common to each series.

[0102] For this reason, the posted information display screen 120 displays reliability information 124 stating, "Similar posts all seem to use the following post as an information source. The reliability of the information source needs to be confirmed." User 9 can determine that the reliability of the posted information reposted by "Yamada Taro" requires caution.

[0103] 14, by displaying a single integrated series of information sources 125, user 9 can easily judge the reliability of the posted information by comparing the posted text and image of the information source with the posted text and image on the posted information display screen 120. For example, if the information source is a personal blog, user 9 can be estimated to have low reliability, and if the information source is a paper written by a scholar affiliated with a university or public institution, user 9 can be estimated to have high reliability even if there is only one information source.

[0104] <Variation 1> 11, the processing load of the series generation unit 14 searching for series increases when there is a large number of pieces of posted information similar to the posted information displayed by the terminal device 30. Therefore, a process for reducing the processing load by narrowing down the number of pieces of posted information similar to the posted information displayed by the terminal device 30 will be described.

[0105] Fig. 15 is a flowchart illustrating a process flow in which the reliability evaluation device 10 evaluates the reliability of the posted information by narrowing down the number of pieces of posted information similar to the posted information displayed by the terminal device 30. Note that the explanation of Fig. 15 may include differences from Fig. 12.

[0106] The similar information identifying unit 13 excludes (S31) posted information that is directly cited by the posted information displayed by the terminal device 30, among posted information similar to the posted information displayed by the terminal device 30. For example, if posted information A displayed by the terminal device 30 directly quotes posted information B and posted information B is included in the similar posted information, posted information B is excluded from the starting point of the series.

[0107] Furthermore, the similar information identifying unit 13 excludes posted information whose difference in posting time from the posted information displayed by the terminal device 30 is not within the threshold (S32). In other words, the similar information identifying unit 13 determines, as the starting point of the series, only registered information that was posted at the same time as the posted information displayed by the terminal device 30. This is because posted information that was not posted at the same time as the posted information displayed by the terminal device 30 is highly likely to be included in any series and is highly likely to be integrated as the starting point of the series.

[0108] By the processing of steps S31 and S32, it is possible to narrow down posted information similar to the posted information displayed by the terminal device 30, and the processing load of the reliability evaluation device 10 can be reduced.

[0109] The processes after FIG. 15 may be similar to those in FIG.

[0110] <Variation 2> When multiple posters who are not the same but have a close relationship post information sources, the reliability may be low even if the information sources are from an independent series. This is because multiple posters who are close to each other (users who belong to the same community, which will be described later) are generally more likely to see the same post or to contact each other through a communication means other than the SNS. It can be said that users in the same community are more likely to pay attention to the same post. In this case, if a user in the same community reposts the content of a post without quoting it, the actual information sources may be the same even if they are from different series. In other words, when multiple posters who are distantly related post information sources from an independent series, the reliability can be estimated to be high. The relationship between posters is the distance between the posters in the SNS space. The distance will be described in FIG. 16. In this modified example, the reliability evaluation unit 16 evaluates the reliability based on the distance between posters who posted information sources from an independent series.

[0111] FIG. 16 is a diagram for explaining distance in an SNS space. FIG. 16 shows the follow relationships between users. Each node represents a user, and the edges (lines) connecting the nodes show the follow relationships. That is, in FIG. 16, edges are shown between the follower and the followed. It does not matter who is the follower and who is the followed.

[0112] The reliability evaluation unit 16 calculates the shortest distance between posters by tracing edges and nodes from a poster who posted an independent information source to a poster who posted another information source. The distance is the number of edges when tracing edges and nodes. Existing methods such as Dijkstra's algorithm and A-star algorithm are used to calculate the shortest distance. For example, in FIG. 16, the distance between G and D is 3. Similarly, the distance between A and C is 2, and the distance between A and F is 1. In this way, it can be estimated that the longer the distance, the longer the relationship between the follower and the followed person, so the distance between posters in the SNS space represents the closeness of the relationship between the posters. In this way, the distance in the SNS space is the distance of the path related to the relationship in which the poster who posted one information source watches (i.e. follows) the posted information of the poster who posted the other information source.

[0113] Next, with reference to FIG. 17, the basis for determining the reliability based on the shortest distance will be described. FIG. 17 is a diagram for explaining the basis for determining the reliability based on the number of edges from a user to another user. When using a network structure of follow relationships between users in an SNS, it is considered that users can be divided into a number of communities 62-64. A community is a group of friends who have similar topics within the SNS. It is not necessary that there are explicit communities 62-64 within the SNS and that users belong to them, but a case in which users who have similar preferred topics are virtually classified may be assumed.

[0114] Since users who talk about similar topics are likely to follow each other, it is generally believed that the shortest path length between users who belong to the same community is short, and the shortest path length between users in different communities is long. Therefore, according to the "two-source rule," information posted by users in different communities is considered to be more reliable than information posted by users in the same community for different information sources.

[0115] In this way, the longer the shortest path length between users who have posted information sources from different groups, the higher the reliability of the posted information displayed by the user can be determined.

[0116] In addition, when users who have posted information sources of different series are not connected by an edge, the reliability evaluation unit 16 sets the distance between these users to a predetermined value. The predetermined value is a distance at which the users may be considered to have no relationship with each other even if they are connected by an edge. Similarly, when the distance between the users exceeds a predetermined value, the reliability evaluation unit 16 may consider the distance between the users to be a predetermined value. In this way, the reliability determination device 10 can reduce the processing load.

[0117] 18 is a flowchart illustrating the process flow of the reliability evaluation device 10 evaluating the reliability of posted information based on the distance between posters who posted information sources of independent groups. Note that the explanation of FIG. 18 may mainly focus on the differences from FIG. 12. The processes of steps S21 to S26 in FIG. 18 may be similar to steps S21 to S26 in FIG. 12.

[0118] In step S47, the reliability evaluation unit 16 determines the distance between the contributors of the independent information sources as the reliability (S47). For example, if the number of independent information sources is N, the calculated distance is N(N-1) / 2 (a combination of two out of N). The reliability evaluation unit 16 normalizes the total distance (for example, converts it to a value between 0 and 100). For normalization, for example, a maximum distance that a user or an operator of the reliability evaluation device 10 may determine as completely independent may be set in advance, and the ratio of the maximum distance to the total distance may be used (if the ratio exceeds 1, it is considered to be 1).

[0119] The reliability evaluation unit 16 may determine the sum of the normalized distances as the reliability as it is, or may compare it with a number of thresholds to evaluate the reliability at several levels.

[0120] In this way, according to this modified example, it is possible to evaluate the reliability based on the distance between posters, which is calculated from the follow relationships within the SNS space.

[0121] In addition, the distance between posters can be expressed not only by a "following relationship," but also by a relationship in which a user has given another user a "Like!" or a relationship in which a user has left a comment on a posted information.

[0122] <Major Effects> The reliability evaluation device 10 of the present embodiment can present reliability to the user when the user displays posted information. The user can refer to the reliability to determine how reliable the posted information is and use it as a reference when taking action. For example, it becomes possible to avoid actions such as reposting unreliable posted information.

[0123] <Other application examples> The above describes the best mode for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0124] For example, in the present embodiment, a method for evaluating reliability has been described using posted information as an example, but the information for which reliability is evaluated is not limited to posted information. Information for which reliability is evaluated can be obtained if there is an information sequence in which information repeats a relationship in which information cites cited information one or more times. For example, when a Web page links to other Web pages, the reliability evaluation device 10 can evaluate the reliability of the Web page being viewed.

[0125] In addition, in this embodiment, the reliability is evaluated at the timing when the user displays the posted information, but the reliability evaluation device 10 may automatically select the posted information to be evaluated for reliability and evaluate it at the selected timing. For example, when there is posted information that has been reposted a certain number of times or posted information that has received a certain number of likes, the reliability evaluation device 10 can determine the reliability.

[0126] In addition, the configuration example in Fig. 5 and the like is divided according to main functions in order to facilitate understanding of the processing by the reliability evaluation device 10, the SNS server 50, and the terminal device 30. The method of dividing the processing units and the names thereof do not limit the present invention. The processing of the reliability evaluation device 10, the SNS server 50, and the terminal device 30 can be divided into more processing units according to the processing contents. Also, the processing units can be divided so that they include more processes.

[0127] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to execute each function described above. [Explanation of symbols]

[0128] 10 Reliability evaluation device 30 Terminal Equipment 50 Social Networking Server 100 Reliability Evaluation System

Claims

1. A reliability evaluation device capable of communicating with a terminal device that displays information via a network, a posting information acquisition unit that acquires first information displayed by the terminal device from a first information sequence in which information repeats a relationship in which the information cites the cited information one or more times; a similar information identifying unit that identifies second information similar to the first information from a second information sequence in which information repeats a relationship in which the information cites the cited information one or more times; a reliability evaluation unit that evaluates the reliability of the first information based on whether a first cited piece of information cited in the first information sequence starting from the first information overlaps with a second cited piece of information cited in the second information sequence starting from the second information; A reliability evaluation device having the following.

2. the second information sequence and the second information are multiple, The reliability evaluation device according to claim 1 , wherein the reliability evaluation unit evaluates the reliability of the first information based on the number of information sequences in which cited information does not overlap in the first information sequence and the plurality of second information sequences.

3. The information includes identification information of a person who posted the information, When there is an information series having the same identification information of the contributor associated with an information source that does not cite cited information in the first information series and the plurality of second information series, the reliability evaluation unit subtracts the number of information series having no overlapping cited information by the number of information sources having the same identification information of the contributor; The reliability evaluation device described in claim 2, wherein when an information source that does not cite cited information is a website and there is an information series that has the same website as an information source, the number of information series that do not have overlapping cited information is reduced by the number of information sources in the information series where the information source is the same website.

4. The information includes a posting time when the information was posted, The reliability evaluation device according to any one of claims 1 to 3, wherein the similar information identification unit excludes from the starting point of the information series, even if the second information is similar to the first information acquired by the posted information acquisition unit, the second information being not posted within a threshold value of the posting time of the first information.

5. The reliability evaluation device of claim 1 , wherein the similar information identification unit excludes the second information that is directly cited by the first information acquired by the posted information acquisition unit, even if the second information is similar to the first information acquired by the posted information acquisition unit.

6. 2. The reliability evaluation device according to claim 1, further comprising: transmitting information regarding the reliability of the first information evaluated by the reliability evaluation unit to the terminal device together with a command to display the information displayed by the terminal device in association with the information to be displayed by the terminal device.

7. transmitting information on the reliability of the first information to the terminal device together with a command to display the information in association with the information displayed by the terminal device, and after a certain period of time has elapsed, The similar information identifying unit identifies second information similar to the first information from the second information sequence again, The reliability evaluation unit evaluates the reliability of the first information based on whether or not a first cited piece of information cited in the first information sequence starting from the first information overlaps with a second cited piece of information cited in the second information sequence starting from the second information; 7. The reliability evaluation device according to claim 6, further comprising: transmitting information regarding the reliability of the first information to the terminal device together with a command to display the information again in association with the information displayed by the terminal device.

8. When the reliability evaluation unit determines that the number of information sequences that do not overlap with cited information is only one in the first information sequence and the plurality of second information sequences, 3. The reliability evaluation device according to claim 2, further comprising: a source of information common to said first information sequence and a plurality of said second information sequences, said source of information being transmitted to said terminal device.

9. It is possible for one poster to monitor the posts of another poster. the second information sequence and the second information are multiple, The reliability evaluation device of claim 1, wherein the reliability evaluation unit identifies information sources in information series in which cited information does not overlap in the first information series and the multiple second information series, and evaluates the reliability of the first information based on the distance of a path related to a relationship in which a contributor who posted one information source watches over the posted information of a contributor who posted the other information source.

10. A reliability evaluation device capable of communicating with a terminal device that displays information via a network, a posting information acquisition unit that acquires first information displayed by the terminal device from a first information sequence in which information repeats a relationship in which the information cites the cited information one or more times; a similar information identifying unit that identifies second information similar to the first information from a second information sequence in which information repeats a relationship in which the information cites the cited information one or more times; a reliability evaluation unit that evaluates reliability of the first information based on whether or not a first cited piece of information cited in the first information sequence starting from the first information overlaps with a second cited piece of information cited in the second information sequence starting from the second information; A program to function as a

Citation Information

Patent Citations

  • Rumor information extraction device and rumor information extraction method

    JP2008165599A

  • Disaster situation determination device, disaster situation determination method, and program

    JP2021140644A