Information processing method, program, and information processing system

Abstract

To support section from a wide variety of information.SOLUTION: An information processing method which is executed by an information processing device includes: an instruction acquisition step of obtaining an input sentence for processing performed by the information processing device; a processing step of performing processing by referring to a predetermined information source based on the input sentence, and calculating a reliability of the predetermined information source; and an output step of outputting the reliability together with a processing result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing method, a program, and an information processing system. [Background technology]

[0002] Cited Document 1 discloses an information processing device that supports sorting of search results. This information processing device sets disease information of a patient, extracts descriptions corresponding to the disease information from search results on an Internet search site, and assigns a reliability to the search results based on the extracted descriptions and evidence information for the disease information. This allows the user to collect reliable information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-036176 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the case of the above prior art, since the evidence information is set based on research papers, it may be difficult to assign a credibility to information that does not exist or has very few research papers. Therefore, there is room for improvement in terms of selecting and discarding a wide range of information, including information for which no research papers exist.

[0005] In consideration of the above, an object of the present invention is to provide support for selecting a wide range of information. [Means for solving the problem]

[0006] According to one embodiment, an information processing method is executed by an information processing device, and includes an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device, a processing step of performing processing based on the input sentence with reference to a specified information source and calculating the reliability of the specified information source, and an output step of outputting the reliability together with the processing result.

[0007] According to a program of one embodiment, an information processing device is caused to execute an information processing method including an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device, a processing step of performing processing based on the input sentence with reference to a specified information source and calculating the reliability of the specified information source, and an output step of outputting the reliability together with the processing result.

[0008] According to one embodiment, the information processing system is executed by an information processing device, and includes an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device, a processing step of performing processing based on the input sentence with reference to a specified information source and calculating the reliability of the specified information source, and an output step of outputting the reliability together with the processing result. Effect of the Invention

[0009] According to one embodiment, it is possible to assist in selecting a wide range of information. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a server according to an embodiment. [Figure 4] 5 is a flowchart illustrating an example of processing executed by the information processing system according to the embodiment. [Diagram 5]11 is an example of a screen display when executed by the information processing system according to the embodiment. [Figure 6] 1 is a graph showing an example of a relationship between reliability and time for each information source in the information processing system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of an information processing system according to the present invention will be described below with reference to Figures 1 to 6. In each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0012] (System Overview) First, an overview of an information processing system 10 according to the present embodiment will be described. The information processing system 10 according to the present embodiment is a system for generating a processing result in response to a prompt input as an input sentence for processing.

[0013] When an input sentence is input by a user U, the information processing system 10 performs processing based on the input sentence with reference to a predetermined information source, and generates the processing result as text (see FIG. 5). Furthermore, the information processing system 10 displays information on the predetermined information source as evidence of the processing result, and also displays the reliability of the predetermined information source. Details will be described later.

[0014] (System Configuration) Fig. 1 is a diagram showing an example of a configuration of an information processing system 10 according to the present embodiment. As shown in Fig. 1, the information processing system 10 according to the present embodiment includes a user terminal 12 and a server 14, which are communicably connected to each other via a network N. The network N is, for example, a wired LAN (Local Area Network), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination of these.

[0015] The user terminal 12 is an example of an information processing device that performs operations for inputting and displaying various information. The user terminal 12 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination of these.

[0016] The server 14 is an example of an information processing device that acquires information on an input sentence input from the user terminal 12, searches for alternative sentences from the web, various documents, databases, etc. based on the input sentence information, performs processing based on the alternative sentences, and outputs the results. The server 14 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. The specific configuration and operation of the server 14 will be described later.

[0017] (Hardware configuration - Server) 2 is a block diagram showing a hardware configuration of the server 14. The server 14 includes a processor 20, a memory 22, a storage 24, and a communication I / F 26, which are communicatively connected to each other via a bus B.

[0018] The processor 20 controls each component of the server 14 and realizes the functions of the server 14 by expanding various programs stored in the storage 24 into the memory 22 and executing them. The programs executed by the processor 20 include, but are not limited to, an OS (Operating System) and various programs described below. The execution of these programs by the processor 20 realizes a part of the state visualization method according to this embodiment. The processor 20 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), or a combination of these.

[0019] The memory 22 is, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), or a combination thereof. The ROM is, for example, a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), or a combination thereof. The RAM is, for example, a Dynamic RAM (DRAM), a Static RAM (SRAM), a Magnetoresistive RAM (MRAM), or a combination thereof.

[0020] The storage 24 stores the OS, various programs described below, and various data. The storage 24 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), or a combination of these.

[0021] The communication I / F 26 is an interface for connecting the server 14 to external devices including the user terminal 12 and the image capturing device 16 via the network N and for controlling communication. The communication I / F 26 is, for example, an adapter compliant with Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (e.g., Fibre Channel), but is not limited thereto.

[0022] The hardware configuration of the user terminal 12 is substantially the same as the hardware configuration of the server 14 described above, and therefore a detailed description thereof will be omitted.

[0023] (Functional configuration - Server) Next, the functional configuration of the server 14 will be described. Fig. 3 is a diagram showing an example of the functional configuration of the server 14. When executing various programs, the server 14 realizes various functions using the above-mentioned hardware resources. The server 14 has a storage unit 30, a processing unit 40, and a communication unit 60 as the functional configuration realized by the server 14. Each functional configuration is realized by the processor 20 reading and executing a program 32 stored in the memory 22 or the storage 24.

[0024] The communication unit 60 is realized by the communication I / F 26. The communication unit 60 transmits and receives information to and from the user terminal 12 via the network N. The communication unit 60 receives information input from the user terminal 12. In addition, the communication unit 60 transmits information to the user terminal 12, and receives a request by a user U from the user terminal 12.

[0025] The storage unit 30 is realized by the memory 18 and the storage 24. The storage unit 30 stores a program 32, a trained model 34, and a result DB 36.

[0026] The trained model 34 is composed of a plurality of trained machine learning models. Specifically, the model includes a large-scale language model and an evaluation model. The large-scale language model is a model trained with a large amount of text data from articles on the Internet, publications such as books, websites, and paper information, and when an input sentence that is a prompt is input, text data corresponding to the prompt is generated and output. The large-scale language model in the trained model 34 of this embodiment generates text data that is a processing result corresponding to the prompt based on the acquired prompt and with reference to a predetermined information source. Note that in this embodiment, the large-scale language model is configured to generate a processing result with reference to a predetermined information source, but this is not limited to this, and the large-scale language model may be configured to search for a predetermined information source using other trained models or algorithms, and the search result may be used by the large-scale language model, or may have other configurations.

[0027] Moreover, as an example, the evaluation model in the trained model 34 is a model trained to output the reliability of text data. Specifically, the reliability of input text data is calculated by training using labeling data given to text data whose reliability is known, feature analysis results of the text data, user feedback data, and the like (the calculation of the reliability will be described in detail later). The evaluation model may be configured to calculate the reliability using a large language model, etc. In addition, in this embodiment, each model constituting the trained model 34 described above is configured to be stored in the storage unit 30, but this is not limiting, and at least one model may be stored externally and connected to the information processing system 10 so as to be communicable.

[0028] The result DB 36 is a database that stores information related to information acquired so far by the information processing system 10 and results of processing executed. The result DB 36 stores, for example, input sentences acquired so far, processed processing results, information related to information sources used in processing, and calculated reliability information, all of which are linked to one another.

[0029] The processing unit 40 is realized by the processor 20 reading and executing the program 32 from the memory 22 (see FIG. 2) and cooperating with other hardware configurations. The processing unit 40 includes an information acquisition unit 42, an information processing unit 44, an information search unit 46, and an output unit 48.

[0030] The information acquiring unit 42 acquires information on an input sentence entered by a user. This input sentence corresponds to a prompt entered by the user on the user terminal 12, and is text data expressing at least a part of a process that the user wishes to execute using the information processing system 10. As an example, when a user requires information on the effect of a specific therapeutic drug for a certain disease, the user may enter "Give me information on the therapeutic effect of steroids for lupus nephritis" as the prompt in the text data PT (see FIG. 5), and the information acquiring unit 42 acquires the text data PT as information on the input sentence, and stores the result in the DB 36.

[0031] The information processing unit 44 performs processing based on the input sentence, referring to a predetermined information source. When referring to a predetermined information source, a search is performed based on the acquired input sentence. The predetermined information source is a paper, the Web including SNS, publications, etc. Note that the paper and publications are stored in a database (not shown) as electronic text data, for example. In the search, the search is executed in the information search unit 46, which will be described later (details will be described later).

[0032] The information processing unit 44 performs analysis of information from a predetermined information source searched by the information search unit 46 as necessary. This analysis is performed on at least one of a plurality of indicators, namely, the field, date and time information, provider, and information amount and quality information of the information source. Each indicator will be described below. Note that the information processing unit 44 can adjust the presence or absence of analysis and the indicators to be analyzed as appropriate depending on the situation, such as when information on each indicator cannot be obtained for the searched predetermined information source, or when analysis of only one indicator rather than all indicators is sufficient in terms of the relationship with other indicators. Also, this adjustment of the indicators may be performed according to an instruction from the user (see "Processing Options" in FIG. 5).

[0033] (Source index: field) When the information processing unit 44 analyzes the field of an information source, it analyzes which field the information source belongs to. As an example, fields such as chemical technology, social science, humanities, business and economics, art and entertainment, sports, law, education, health and medicine, and news and politics are set in advance, and it is analyzed to see which of these fields the information source relates to. The fields may be divided according to the characteristics of the information source. As an example, the fields may be divided according to characteristics such as fields with paper information, fields with little paper information, fields without paper information, etc., or according to other characteristics.

[0034] (Source indicator: date and time information) When the information processing unit 44 analyzes the date and time information of an information source, it analyzes at least one of the issue date, release date, revision date, and expiration date of the information source, and the date and time when an event or phenomenon occurred in the information source. For example, the issue date of a publication, the release date of a news article, blog post, academic paper, report, etc., the expiration date of a law, regulation, guideline, campaign, etc., the occurrence date of an incident reported in a news article, the acquisition date of a scientific observation result in a paper, etc. correspond to date and time information. In the analysis in this embodiment, the specific date of these date and time information is specified.

[0035] (Source indicator: provider) When the information processing unit 44 analyzes the provider of the information source, it analyzes information related to the individual or organization from which the information source was transmitted. As an example, the provider may be an academic institution that publishes research papers or reports, a news media that provides articles and reports provided by newspapers, magazines, television stations, news websites, etc., a company that provides an official website, press releases, or annual reports, a government agency that provides policy documents, statistical data, official announcements, and documents related to intellectual property rights, an expert who transmits information about his or her own field of expertise, an individual user who transmits his or her own opinions and experiences on SNS or blogs, or a non-profit organization that provides research reports, policy proposals, and educational materials. In the analysis in this embodiment, the name, type, etc. of these providers are identified.

[0036] (Source index: amount of information) When the information processing unit 44 analyzes the amount of information of an information source, it analyzes not only the amount of information of the information source itself, but also the amount of information of documents or sentences included in the information source. For example, the amount of information corresponds to the amount of information itself, the length of sentences or documents, and the size and length of images, videos, sounds, charts, etc. In the analysis in this embodiment, the amount of information is also specified, including the amount of bytes, which is the electronic amount of the amount of information.

[0037] (Source indicator: quality information) When the information processing unit 44 analyzes the quality of an information source, the accuracy, reliability, objectivity, timeliness, comprehensiveness, and referability of the information source are analyzed as indicators. That is, for example, for accuracy, an analysis is made as to whether the information contained in the information source is based on facts and does not include errors or misunderstandings. For reliability, an analysis is made as to whether the information source consistently provides accurate and reliable information. For objectivity, an analysis is made as to whether the information of the information source is presented without bias or preconceptions. For timeliness, an analysis is made as to whether the information of the information source is up-to-date and its contents are still appropriate at present. For comprehensiveness, an analysis is made as to whether the information of the information source comprehensively covers topics required in processing. For referability, an analysis is made as to whether the information of the information source is supported, mentioned, linked, etc. by other reliable information sources. In the analysis in this embodiment, the quality is identified by the analysis results of these indicators.

[0038] (Reliability calculation) The information processing unit 44 calculates the reliability of the information source by the evaluation model in the trained model 34 based on the information obtained by the above analysis. That is, as shown in FIG. 6, the reliability of each piece of information increases in the order of information on SNS posted by an individual user, website information, publication information, and paper information. In addition, the time until each piece of information is posted, in other words, the time from the occurrence of a certain event until the event is reflected as information (hereinafter simply referred to as "posting time"), generally takes the longest in the order of information on SNS posted by an individual user, website information, publication information, and paper information. However, the reliability of each information source does not necessarily relate to the length of posting time, and also varies depending on the field, date and time information, provider, and amount and quality of information. As an example, when a process related to the medical field is executed, if a paper and website information exist as information sources, the paper is calculated as a more reliable information source, whereas when a process related to the latest trends in programming or IT technology is executed, if a paper and website information exist as information sources, the website information is calculated as a more reliable information source. In addition, when there are information sources with new date and time information and old date and time information, when processing related to technology or events, the information source with new date and time information is calculated as a highly reliable information source, while when processing related to historical facts, the information source with old date and time information is calculated as a highly reliable information source. Furthermore, when the provider is a public institution information source and an individual user information source, when processing related to medical information, the public institution information source is calculated as a highly reliable information source, while when processing related to consumer requests, the individual user information source is calculated as a highly reliable information source. Furthermore, when there are information sources with a large amount of information and information sources with a small amount of information, when processing related to specified statistics is performed, the information source with a large amount of information is calculated as a highly reliable information source. On the other hand, when the amount of information is small, the reliability of the information source is generally not high, but when the provider is a public institution or an expert, it may be calculated as a highly reliable information source after considering other indicators.In addition, when there are information sources with high and low quality information, when performing processing related to the medical field, the information source with high quality information is calculated as the information source with high reliability, while when performing processing related to the real voices of users of a certain product, the information source with low quality information is calculated as the information source with high reliability. The above is an example of calculation of reliability, but the reliability may be calculated by other judgments without being limited to these. In addition, these judgments may be made using not only each index alone but also a combination of multiple indexes. Furthermore, these reliability is calculated as a score out of 100 points as an example, but it is not limited to this and may be expressed in other ways, such as a 5-point rating. In addition, in FIG. 6, the reliability of the paper information can be further calculated in the paper by the evidence pyramid, but it is also possible to set a similar pyramid in advance for other information sources, not limited to paper information, and calculate the reliability based on this.

[0039] When the information processing unit 44 processes the input sentence, the information search unit 46 extracts keywords from the input sentence and generates a search query, and searches for a predetermined information source based on the extracted keywords. The information search unit 46 transmits information about the search results to the information processing unit 44.

[0040] The output unit 48 transmits the processing result and information on the reliability of the input sentence processed by the information processing unit 44 to the user terminal 12. In this embodiment, it is possible to switch between displaying and not displaying the information on the information source used in the processing, the portion in the processing result where the information on the information source was used, and the reliability of the information source by operating a predetermined operation button (see the "Display information source" button in FIG. 5).

[0041] (Processing performed by information processing system) Next, the operation of the information processing system 10 will be described. Fig. 4 is a flow chart showing an example of the flow of processing by the information processing system 10. The processor 20 reads out the program 32 stored in the storage 24, expands it in the memory 22, and executes it, thereby performing processing. Although not shown, when the processor 20 receives operation information for terminating the operation of the information processing system 10, or information on the end of an operation from the user terminal 12 during the determination processing being executed (these will be simply referred to as "termination operation"), the processor 20 ends the processing based on the program 32 being processed.

[0042] The processor 20 outputs an input area IA (see FIG. 5) for inputting an input sentence so as to be displayed on the user terminal 12 (step S100). Then, the processor 20 determines whether or not an input sentence has been input in the input area IA (step S102). If an input sentence has not been input (step S102: NO), the processor 20 transitions the process to step S100. On the other hand, if an input sentence has been input (step S102: YES), the processor 20 analyzes the input sentence (step S104).

[0043] The processor 20 searches for information sources based on the analysis of the input sentence (step S106), and determines whether or not to perform a field analysis of the information sources in the search results (step S108). If the field analysis is not to be performed (step S108: NO), the processor 20 shifts the process to step S112, which will be described later. On the other hand, if the field analysis is to be performed (step S108: YES), the processor 20 analyzes the field of the information source (step S110).

[0044] The processor 20 determines whether or not to analyze the date and time information of the information source of the search result (step S112). If the date and time information is not to be analyzed (step S112: NO), the processor 20 proceeds to step S116 described below. On the other hand, if the date and time information is to be analyzed (step S112: YES), the processor 20 analyzes the date and time information of the information source (step S114).

[0045] The processor 20 determines whether or not to analyze the provider information of the information source in the search result (step S116). If the provider information is not to be analyzed (step S116: NO), the processor 20 shifts the process to step S120 described below. On the other hand, if the provider information is to be analyzed (step S116: YES), the processor 20 analyzes the provider information of the information source (step S118).

[0046] The processor 20 determines whether or not to analyze the amount of information of the information sources in the search results (step S120). If the amount of information is not to be analyzed (step S120: NO), the processor 20 proceeds to step S124 described below. On the other hand, if the amount of information is to be analyzed (step S120: YES), the processor 20 analyzes the amount of information of the information sources (step S122).

[0047] The processor 20 determines whether or not to analyze quality information of the information sources in the search results (step S124). If the quality information is not to be analyzed (step S124: NO), the processor 20 proceeds to step S128, which will be described later. On the other hand, if the quality information is to be analyzed (step S124: YES), the processor 20 analyzes the quality information of the information sources (step S126).

[0048] The processor 20 performs processing based on the analysis result of the input sentence and, if there is a searched information source, refers to the information source (step S126). If the processor 20 refers to an information source in step S126, it calculates the reliability of the information source based on the analysis result of each index.

[0049] The processor 20 outputs the processing result and, if necessary, information on the information source and the reliability of the information source in an output area OA (see FIG. 5) for displaying the information on the information source on the user terminal 12 (step S130).

[0050] (Actions and Effects of the Embodiments) Next, the operation and effects of this embodiment will be described.

[0051] The information processing system 10 according to the present embodiment calculates the reliability of a specific information source when processing an input sentence based on the input sentence and references the specific information source, and outputs the processing result and the reliability, making it easy for the user to select the processing result based on the reliability. This can support the selection of a wide range of information.

[0052] Furthermore, the information processing system 10 determines the field to which a given information source belongs and calculates the reliability according to the field, so that it can respond to a situation in which the reliability of the information source varies depending on the field even for the same type of information source. In other words, the reliability calculated for the information source can be made more accurate. This makes it possible to more accurately support the selection of a wide range of information. From another perspective, although there may be little or no highly reliable paper information in some fields, it is possible to calculate the reliability of an appropriate information source even in such fields. This makes it possible to support the selection of a wide range of information, not limited to paper information.

[0053] Furthermore, the information processing system 10 obtains date and time information of a given information source and calculates the reliability according to the date and time information, and therefore can respond to situations in which the reliability of the information source changes depending on the date and time due to technological advances and amendments to laws and rules. In other words, the reliability calculated for the information source can be made more accurate. This makes it possible to provide more accurate support for selecting a wide range of information.

[0054] Furthermore, the information processing system 10 obtains provider information of a given information source and calculates the reliability according to the provider information, so that it can handle situations where the reliability of the information source varies depending on the provider. In other words, it is possible to make the reliability calculated for the information source more accurate. This makes it possible to provide more accurate support for selecting a wide range of information.

[0055] In addition, the information processing system 10 obtains information on the amount of information of a given information source and calculates the reliability according to the information on the amount of information, so that it can handle a situation in which the reliability of the information source changes depending on the amount of information. In other words, it is possible to make the reliability calculated for the information source more accurate. This makes it possible to provide more accurate support for selecting a wide range of information.

[0056] Furthermore, the information processing system 10 obtains quality information of a given information source and calculates the reliability according to the quality information, so that it can handle a situation in which the reliability of the information source changes depending on the quality information. In other words, it is possible to make the reliability of the information source calculated more accurate. This makes it possible to provide more accurate support for selecting a wide range of information.

[0057] Furthermore, in this embodiment, the user can specify the index of the information source to be used in the process (see "Processing Options" in FIG. 5), so that the output processing result is the result desired by the user, and the user can adjust it by himself / herself. This allows for more accurate processing in response to the user's request.

[0058] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope that can achieve the object of the present invention are included in the present invention.

[0059] Also, for example, the above-mentioned series of processes can be executed by hardware or software. In other words, the functional configuration is merely an example and is not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and there is no particular limit to what kind of functional block is used to realize this function. In addition, the location of the functional block is not particularly limited and may be arbitrary. For example, the functional block of the server may be transferred to a user terminal or the like. Conversely, the functional block of the user terminal may be transferred to a server or the like. In addition, one functional block may be configured by hardware alone, software alone, or a combination thereof.

[0060] For example, when a series of processes is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0061] Furthermore, for example, a recording medium containing such a program may not only be constituted by a removable medium (not shown) that is distributed separately from the device main body in order to provide the program to the user, but may also be constituted by a recording medium that is provided to the user in a state in which it is pre-installed in the device main body.

[0062] In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed chronologically according to the order, but also processes that are not necessarily performed chronologically but are executed in parallel or individually. In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices, a plurality of means, etc.

[0063] <Additional Notes> The present embodiment includes the following disclosure.

[0064] (Appendix 1) An information processing method executed by an information processing device, An instruction acquisition step of acquiring an input sentence for processing by the information processing device; A processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; An information processing method comprising:

[0065] (Appendix 2) The processing step includes determining a field to which the predetermined information source belongs and calculating the reliability according to the field. 2. An information processing method as described in Appendix 1.

[0066] (Appendix 3) The processing step includes acquiring date and time information of the predetermined information source, and calculating the reliability according to the date and time information. 2. An information processing method as described in Appendix 1.

[0067] (Appendix 4) The processing step includes acquiring provider information of the predetermined information source, and calculating the reliability according to the provider information. 2. An information processing method as described in Appendix 1.

[0068] (Appendix 5) The processing step includes acquiring information amount information of the predetermined information source, and calculating the reliability according to the information amount information. 2. An information processing method as described in Appendix 1.

[0069] (Appendix 6) The processing step includes acquiring quality information of the predetermined information source, and calculating the reliability according to the quality information. 2. An information processing method as described in Appendix 1.

[0070] (Appendix 7) In the information processing device, An instruction acquisition step of acquiring an input sentence for processing by the information processing device; A processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; A program for executing an information processing method including the steps of:

[0071] (Appendix 8) An information processing system executed by an information processing device, An instruction acquisition step of acquiring an input sentence for processing by the information processing device; A processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; An information processing system that performs the above. [Explanation of symbols]

[0072] 10 Information Processing Systems 14 Servers (information processing devices)

Claims

1. An information processing method executed by an information processing device, An instruction acquisition step of acquiring an input sentence for processing by the information processing device; A processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; An information processing method comprising:

2. The processing step includes determining a field to which the predetermined information source belongs and calculating the reliability according to the field. The information processing method according to claim 1 .

3. The processing step includes acquiring date and time information of the predetermined information source, and calculating the reliability according to the date and time information. The information processing method according to claim 1 .

4. The processing step includes acquiring provider information of the predetermined information source, and calculating the reliability according to the provider information. The information processing method according to claim 1 .

5. The processing step includes acquiring information amount information of the predetermined information source, and calculating the reliability according to the information amount information. The information processing method according to claim 1 .

6. The processing step includes acquiring quality information of the predetermined information source, and calculating the reliability according to the quality information. The information processing method according to claim 1 .

7. In the information processing device, An instruction acquisition step of acquiring an input sentence for processing by the information processing device; A processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; A program for executing an information processing method including the steps of:

8. An information processing system executed by an information processing device, An instruction acquisition step of acquiring an input sentence for processing by the information processing device; A processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; An information processing system that performs the above.

Images