Information processing method, program, and information processing system

Abstract

To support section from a wide variety of information.SOLUTION: An information processing method which is executed by an information processing device includes: an instruction acquisition step of obtaining an input sentence for processing performed by the information processing device; a processing step of performing processing by referring to a predetermined information source based on the input sentence, and calculating a reliability of the predetermined information source; and an output step of outputting the reliability together with a processing result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing method, a program, and an information processing system. [Background technology]

[0002] Cited Document 1 discloses an information processing device that supports sorting of search results. This information processing device sets disease information of a patient, extracts descriptions corresponding to the disease information from search results on an internet search site, and assigns a reliability to the search results based on the extracted descriptions and evidence information for the disease information. This allows the user to collect highly reliable information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-036176 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the case of the above prior art, since the evidence information is set based on research papers, it may be difficult to assign a reliability to information that does not exist or has very few research papers. Therefore, there is room for improvement in selecting a wide range of information, including information for which no research papers exist.

[0005] In consideration of the above, the present invention aims to support the selection of a wide range of information. [Means for solving the problem]

[0006] According to one embodiment, the information processing method is executed by an information processing device, and includes an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device, a processing step of performing processing based on the input sentence with reference to a specified information source and calculating the reliability of the specified information source, and an output step of outputting the reliability together with the processing result.

[0007] According to one embodiment of the program, an information processing device is caused to execute an information processing method including an instruction acquisition step of acquiring an input sentence for processing to be performed by the information processing device, a processing step of performing processing based on the input sentence with reference to a specified information source and calculating the reliability of the specified information source, and an output step of outputting the reliability together with the processing result.

[0008] According to one embodiment, the information processing system is executed by an information processing device, and includes an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device, a processing step of performing processing based on the input sentence with reference to a specified information source and calculating the reliability of the specified information source, and an output step of outputting the reliability together with the processing result. [Effects of the Invention]

[0009] According to one embodiment, it is possible to support selection of a wide range of information. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to an embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of a server according to an embodiment. [Figure 4] 10 is a flowchart illustrating an example of processing executed by the information processing system according to the embodiment. [Figure 5]10 is a diagram illustrating an example of a screen display when executed by the information processing system according to the embodiment. [Figure 6] 10 is a graph schematically illustrating an example of the relationship between reliability and time for each information source in the information processing system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of an information processing system according to the present invention will be described below with reference to Figures 1 to 6. In each drawing, the same or equivalent components and parts are denoted by the same reference numerals. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0012] (System Overview) First, an overview of the information processing system 10 according to this embodiment will be described. The information processing system 10 according to this embodiment is a system for generating a processing result in response to a prompt input as an input statement for processing.

[0013] When a user U inputs an input sentence, the information processing system 10 processes the input sentence with reference to a predetermined information source, and generates the processing result as text (see FIG. 5). Furthermore, the information processing system 10 displays information about the predetermined information source as evidence of the processing result, and also displays the reliability of the predetermined information source. Details will be described later.

[0014] (System Configuration) Fig. 1 is a diagram showing an example of the configuration of an information processing system 10 according to this embodiment. As shown in Fig. 1, the information processing system 10 according to this embodiment includes a user terminal 12 and a server 14, which are connected to each other so as to be able to communicate with each other via a network N. The network N is, for example, a wired local area network (LAN), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination thereof.

[0015] The user terminal 12 is an example of an information processing device that performs operations for inputting and displaying various information. The user terminal 12 may be a personal computer (PC), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination thereof.

[0016] The server 14 is an example of an information processing device that acquires information about an input sentence entered from the user terminal 12, searches for alternative sentences from the web, various documents, databases, etc. based on the input sentence information, performs processing based on the alternative sentences, and outputs the results. The server 14 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. The specific configuration and operation of the server 14 will be described later.

[0017] (Hardware configuration - Server) 2 is a block diagram showing the hardware configuration of the server 14. The server 14 includes a processor 20, a memory 22, a storage 24, and a communication I / F 26, which are communicatively connected to each other via a bus B.

[0018] The processor 20 controls each component of the server 14 and realizes the functions of the server 14 by loading various programs stored in the storage 24 into the memory 22 and executing them. The programs executed by the processor 20 include, but are not limited to, an OS (Operating System) and various programs described below. Execution of these programs by the processor 20 realizes part of the state visualization method according to this embodiment. The processor 20 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), or a combination thereof.

[0019] The memory 22 is, for example, a read-only memory (ROM), a random access memory (RAM), or a combination thereof. The ROM is, for example, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a combination thereof. The RAM is, for example, a dynamic random access memory (DRAM), a static random access memory (SRAM), a magnetoresistive random access memory (MRAM), or a combination thereof.

[0020] The storage 24 stores the OS, various programs (described later), and various data. The storage 24 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), or a combination of these.

[0021] The communication I / F 26 is an interface for connecting the server 14 to external devices, including the user terminal 12 and the image capturing device 16, via the network N and for controlling communication. The communication I / F 26 is, for example, an adapter compliant with Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (e.g., Fibre Channel), but is not limited to these.

[0022] The hardware configuration of the user terminal 12 is substantially the same as the hardware configuration of the server 14 described above, and therefore a detailed description thereof will be omitted.

[0023] (Functional configuration - Server) Next, the functional configuration of the server 14 will be described. Fig. 3 is a diagram showing an example of the functional configuration of the server 14. When executing various programs, the server 14 realizes various functions using the above-mentioned hardware resources. The server 14 has a storage unit 30, a processing unit 40, and a communication unit 60 as the functional configuration realized by the server 14. Each functional configuration is realized when the processor 20 reads and executes a program 32 stored in the memory 22 or the storage 24.

[0024] The communication unit 60 is realized by the communication I / F 26. The communication unit 60 transmits and receives information to and from the user terminal 12 via the network N. The communication unit 60 receives information input from the user terminal 12. The communication unit 60 also transmits information to the user terminal 12 and receives requests from the user U from the user terminal 12.

[0025] The storage unit 30 is realized by the memory 18 and the storage 24. The storage unit 30 stores a program 32, a trained model 34, and a result DB 36.

[0026] The trained model 34 is composed of multiple trained machine learning models. Specifically, it includes a large-scale language model and an evaluation model. The large-scale language model is a model trained with a large amount of text data from online articles, publications such as books, websites, and paper information. When an input sentence that is a prompt is input, the large-scale language model generates and outputs text data corresponding to the prompt. The large-scale language model in the trained model 34 of this embodiment generates text data that is a processing result corresponding to the prompt based on the acquired prompt and with reference to a predetermined information source. Note that in this embodiment, the large-scale language model is configured to generate a processing result with reference to a predetermined information source. However, this is not limited to this. The predetermined information source may be searched for using other trained models or algorithms, and the search results may be used by the large-scale language model, or another configuration may be used.

[0027] Moreover, as an example, the evaluation model in the trained model 34 is a model trained to output the reliability of text data. Specifically, the reliability of input text data is calculated by training using labeling data assigned to text data with known reliability, feature analysis results of the text data, user feedback data, etc. (The calculation of reliability will be described in detail later). Note that the evaluation model may also be configured to calculate the reliability using a global language model, etc. Furthermore, in this embodiment, each model constituting the trained model 34 described above is configured to be stored in the storage unit 30, but this is not limiting, and at least one model may be stored externally and communicably connected to the information processing system 10.

[0028] The result DB 36 is a database that stores information related to information acquired so far by the information processing system 10 and results of processing executed. The result DB 36 stores, for example, input sentences acquired so far, processed processing results, information related to the information sources used in the processing, and calculated reliability information, all of which are linked to one another.

[0029] The processing unit 40 is realized by the processor 20 reading and executing the program 32 from the memory 22 (see FIG. 2) and working in cooperation with other hardware components. The processing unit 40 includes an information acquisition unit 42, an information processing unit 44, an information search unit 46, and an output unit 48.

[0030] The information acquisition unit 42 acquires information on an input sentence entered by a user. This input sentence corresponds to a prompt entered by the user on the user terminal 12, and is text data that represents at least a part of the process that the user wants to execute using the information processing system 10. As an example, if the user needs information on the effectiveness of a specific therapeutic drug for a certain disease, the user can enter the prompt "Give me information on the therapeutic effectiveness of steroids for lupus nephritis" in text data PT (see FIG. 5), and the information acquisition unit 42 will acquire the text data PT as information on the input sentence and store it in the result DB 36.

[0031] The information processing unit 44 performs processing based on the input sentence, referring to a predetermined information source. When referring to a predetermined information source, a search is performed based on the acquired input sentence. The predetermined information source may be a paper, the web including SNS, publications, etc. Note that the paper and publications are stored as digitized text data in a database (not shown), for example. The search is performed in the information search unit 46 (described later in detail) for the search.

[0032] The information processing unit 44 analyzes, as necessary, information from a predetermined information source searched by the information search unit 46. This analysis is performed on multiple indicators, namely, the field, date and time information, provider, and / or information on the information source, amount of information, and / or quality information. Each indicator will be described below. Note that the information processing unit 44 can adjust whether to perform analysis or which indicators to analyze as appropriate depending on the situation, such as when information on each indicator cannot be obtained for the searched predetermined information source, or when analysis of only one indicator is sufficient rather than all indicators due to their relationship with other indicators. Furthermore, this adjustment of indicators may be performed in response to a user instruction (see "Processing Options" in FIG. 5).

[0033] (Source index: field) When the information processing unit 44 analyzes the field of an information source, it analyzes to which field the information source belongs. As an example, fields such as chemical technology, social science, humanities, business and economics, arts and entertainment, sports, law, education, health and medicine, and news and politics are pre-set, and it analyzes to which of these fields the information source relates. The fields may also be divided according to the characteristics of the information source. For example, the fields may be divided according to characteristics such as fields with paper information, fields with little paper information, fields with no paper information, or other characteristics.

[0034] (Source indicator: Date and time information) When the information processing unit 44 analyzes the date and time information of an information source, it analyzes at least one of the issue date, release date, revision date, expiration date, and the date and time of an event or phenomenon occurring in the information source. Examples of date and time information include the issue date of a publication, the release date of a news article, blog post, academic paper, or report, the expiration date of a law, regulation, guideline, or campaign, the occurrence date of an incident reported in a news article, or the date of acquisition of scientific observation results in a paper. In the analysis of this embodiment, specific dates in the date and time information are identified.

[0035] (Source indicator: provider) When the information processing unit 44 analyzes information providers, it analyzes information related to the individuals or organizations that are the source of the information sources. Examples of providers include academic institutions that publish research papers and reports, news media that provide articles and reports such as newspapers, magazines, television stations, and news websites, companies that provide official websites, press releases, and annual reports, government agencies that provide policy documents, statistical data, official announcements, and documents related to intellectual property rights, experts who disseminate information about their fields of expertise, individual users who disseminate their opinions and experiences on social media sites and blogs, and non-profit organizations that provide research reports, policy recommendations, and educational materials. In this embodiment, the analysis identifies the names and types of these providers.

[0036] (Source index: amount of information) When the information processing unit 44 analyzes the amount of information in an information source, it analyzes not only the amount of information in the information source itself, but also the amount of information in each document or sentence included in the information source. For example, the amount of information corresponds to the amount of information itself, the length of the sentence or document, and the size and length of images, videos, audio, charts, etc. In the analysis in this embodiment, the amount of information is also identified, including the amount of bytes, which is the electronic amount of this information.

[0037] (Source indicator: quality information) When the information processing unit 44 analyzes the quality of an information source, it analyzes the accuracy, reliability, objectivity, timeliness, comprehensiveness, and referentiality of the information source as indicators. Specifically, for accuracy, for example, it analyzes whether the information contained in the information source is based on facts and free of errors or misunderstandings. For reliability, for example, it analyzes whether the information source consistently provides accurate and reliable information. For objectivity, for example, it analyzes whether the information source is presented without bias or preconceptions. For timeliness, for example, it analyzes whether the information source is up-to-date and its content is still relevant. For comprehensiveness, for example, it analyzes whether the information source comprehensively covers topics required for processing. For referentiality, for example, it analyzes whether the information source is supported, mentioned, linked, etc. by other reliable information sources. In the analysis in this embodiment, quality is determined based on the analysis results of these indicators.

[0038] (Reliability calculation) Based on the information obtained by the above analysis, the information processing unit 44 calculates the reliability of the information sources using the evaluation model in the trained model 34. That is, as shown in FIG. 6, the reliability of each piece of information generally increases in the following order: information posted on SNS by an individual user, website information, publication information, and paper information. Furthermore, the time it takes for each piece of information to be posted, in other words, the time from the occurrence of a specific event until the event is reflected as information (hereinafter simply referred to as "posting time"), generally takes the longest in the following order: information posted on SNS by an individual user, website information, publication information, and paper information. However, the reliability of each information source does not necessarily correlate with the length of posting time, but also varies depending on the field, date and time information, provider, and information quantity and quality. For example, when performing a process related to the medical field, if both paper and website information are available as information sources, the paper is calculated as the more reliable information source. On the other hand, when performing a process related to programming or the latest trends in IT technology, if both paper and website information are available as information sources, the website information is calculated as the more reliable information source. Furthermore, when there are information sources with newer date and time information and older date and time information, the information source with the newer date and time information is calculated as a more reliable source when processing technology or events, while the information source with the older date and time information is calculated as a more reliable source when processing historical facts. Furthermore, when there are information sources from public institutions and information sources from individual users, when processing medical information, the information source from the public institution is calculated as a more reliable source, while when processing consumer requests, the information source from the individual user is calculated as a more reliable source. Furthermore, when there are information sources with a large amount of information and information sources with a small amount of information, the information source with a large amount of information is calculated as a more reliable source when processing specific statistics. On the other hand, when the amount of information is small, the reliability of the information source is generally not high, but when the provider is a public institution or expert, it may be calculated as a more reliable source after taking other indicators into consideration.Furthermore, when there are sources of information with high and low quality, the source with high quality information is calculated as the most reliable source when performing processing related to the medical field, while the source with low quality information is calculated as the most reliable source when performing processing related to the real voices of users of a certain product. While the above is an example of reliability calculation, reliability may be calculated using other methods. These determinations may be made using not only a single indicator but also a combination of multiple indicators. Furthermore, while these reliability levels are calculated as a score out of 100, for example, they may be expressed in other ways, such as a 5-point scale. Note that in Figure 6, the reliability of paper information can be further calculated within the paper using an evidence pyramid. However, similar pyramids may be pre-set for other information sources, not just paper information, and reliability may be calculated based on these pyramids.

[0039] When the information processing unit 44 performs processing based on the input sentence, the information search unit 46 extracts keywords from the input sentence and generates a search query, and based on this, searches for a predetermined information source. The information search unit 46 transmits information about the search results to the information processing unit 44.

[0040] The output unit 48 transmits the processing results and information on the reliability of the input sentence processed by the information processing unit 44 to the user terminal 12. In this embodiment, it is possible to switch between displaying and not displaying the information on the information sources used in the processing, the parts of the processing results where the information on the information sources is used, and the reliability of the information sources by operating a predetermined operation button (see the "Display Information Source" button in FIG. 5).

[0041] (Processing performed by the information processing system) Next, the operation of the information processing system 10 will be described. Fig. 4 is a flowchart showing an example of the flow of processing by the information processing system 10. The processor 20 reads out the program 32 stored in the storage 24, expands it in the memory 22, and executes it, thereby performing processing. Although not shown, when the processor 20 receives operation information to terminate the operation of the information processing system 10 or information on the termination of an operation from the user terminal 12 during the determination processing being executed (these will be simply referred to as "termination operations"), the processor 20 terminates the processing based on the program 32 being processed.

[0042] The processor 20 outputs an input area IA (see FIG. 5) for inputting an input sentence so as to be displayed on the user terminal 12 (step S100). Then, the processor 20 determines whether or not an input sentence has been input in the input area IA (step S102). If an input sentence has not been input (step S102: NO), the processor 20 shifts the process to step S100. On the other hand, if an input sentence has been input (step S102: YES), the processor 20 analyzes the input sentence (step S104).

[0043] Processor 20 searches for information sources based on the analysis of the input sentence (step S106), and determines whether to perform a field analysis on the information sources in the search results (step S108). If a field analysis is not to be performed (step S108: NO), processor 20 proceeds to step S112, which will be described later. On the other hand, if a field analysis is to be performed (step S108: YES), processor 20 analyzes the field of the information source (step S110).

[0044] The processor 20 determines whether or not to analyze the date and time information of the information sources in the search results (step S112). If the date and time information is not to be analyzed (step S112: NO), the processor 20 proceeds to step S116, which will be described later. On the other hand, if the date and time information is to be analyzed (step S112: YES), the processor 20 analyzes the date and time information of the information sources (step S114).

[0045] The processor 20 determines whether or not to analyze the provider information of the information source in the search result (step S116). If the provider information is not to be analyzed (step S116: NO), the processor 20 proceeds to step S120, which will be described later. On the other hand, if the provider information is to be analyzed (step S116: YES), the processor 20 analyzes the provider information of the information source (step S118).

[0046] The processor 20 determines whether or not to analyze the amount of information of the information sources in the search results (step S120). If the amount of information is not to be analyzed (step S120: NO), the processor 20 proceeds to step S124, which will be described later. On the other hand, if the amount of information is to be analyzed (step S120: YES), the processor 20 analyzes the amount of information of the information sources (step S122).

[0047] The processor 20 determines whether or not to analyze the quality information of the information sources in the search results (step S124). If the quality information is not to be analyzed (step S124: NO), the processor 20 proceeds to step S128, which will be described later. On the other hand, if the quality information is to be analyzed (step S124: YES), the processor 20 analyzes the quality information of the information sources (step S126).

[0048] The processor 20 performs processing based on the analysis result of the input sentence and, if there is a searched information source, by referring to the searched information source (step S126). If the processor 20 refers to an information source in step S126, it calculates the reliability of the information source based on the analysis result of each index.

[0049] The processor 20 outputs the processing result and, if necessary, information about the information source and the reliability of the information source in an output area OA (see FIG. 5) for display on the user terminal 12 (step S130).

[0050] (Actions and Effects of the Embodiments) Next, the operation and effects of this embodiment will be described.

[0051] The information processing system 10 according to this embodiment calculates the reliability of a predetermined information source when processing an input sentence based on the input sentence and outputs the processing result and the reliability, making it easy for the user to select and discard the processing result based on the reliability. This can support the selection of a wide range of information.

[0052] Furthermore, the information processing system 10 determines the field to which a given information source belongs and calculates its reliability according to that field, thereby enabling it to handle situations in which the reliability of an information source varies depending on the field, even for the same type of information source. In other words, the reliability calculated for an information source can be made more accurate. This allows for more accurate support in selecting a wide range of information. From another perspective, although there may be little or no highly reliable paper information in some fields, the reliability of appropriate information sources can be calculated even in such fields. This allows for support in selecting a wide range of information, not just paper information.

[0053] Furthermore, the information processing system 10 acquires date and time information for a given information source and calculates reliability based on the date and time information, which allows it to respond to situations in which the reliability of an information source changes depending on the date and time due to technological advances, amendments to laws and rules, etc. In other words, the reliability calculated for an information source can be made even more accurate. This allows for even more accurate support for selecting from a wide range of information.

[0054] Furthermore, the information processing system 10 acquires provider information of a given information source and calculates the reliability according to the provider information, thereby enabling it to deal with situations in which the reliability of an information source varies depending on the provider. In other words, the reliability calculated for an information source can be made more accurate. This allows for more accurate support in selecting from a wide range of information.

[0055] Furthermore, the information processing system 10 acquires information volume information of a predetermined information source and calculates reliability according to the information volume information, thereby enabling it to deal with situations in which the reliability of an information source varies depending on the amount of information. In other words, it is possible to further increase the accuracy of the reliability calculated for an information source. This allows for more accurate support for selecting from a wide range of information.

[0056] Furthermore, the information processing system 10 acquires quality information of a predetermined information source and calculates reliability according to the quality information, so it can handle situations where the reliability of the information source changes depending on the quality information. In other words, the reliability calculated for the information source can be made more accurate. This allows for more accurate support for selecting a wide range of information.

[0057] Furthermore, in this embodiment, the user can specify the index of the information source to be used in the processing (see "Processing Options" in FIG. 5), so that the output processing result is the result desired by the user, and the user can make adjustments himself / herself. This allows for more accurate processing to be performed in accordance with the user's requests.

[0058] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope of achieving the object of the present invention are included in the present invention.

[0059] Furthermore, for example, the above-described series of processes can be executed by hardware or software. In other words, the functional configuration is merely an example and is not particularly limited. That is, it is sufficient that the information processing system has the function of being able to execute the above-described series of processes as a whole, and there is no particular limit to the type of functional block used to realize this function. Furthermore, the location of the functional block is also not particularly limited and may be arbitrary. For example, a functional block of a server may be transferred to a user terminal, etc. Conversely, a functional block of a user terminal may be transferred to a server, etc. Furthermore, one functional block may be configured as a single piece of hardware, a single piece of software, or a combination thereof.

[0060] Furthermore, for example, when a series of processes is executed by software, the programs constituting the software are installed onto a computer or the like from a network or a recording medium. The computer may be a computer incorporated into dedicated hardware. Furthermore, the computer may be a computer capable of executing various functions by installing various programs thereon, such as a server, a general-purpose smartphone, or a personal computer.

[0061] Furthermore, for example, the recording medium containing such a program may be configured not only as a removable medium (not shown) that is distributed separately from the device main body in order to provide the program to the user, but also as a recording medium that is provided to the user in a state that is pre-installed in the device main body.

[0062] In this specification, the steps describing the program to be recorded on the recording medium include not only processes that are performed in chronological order, but also processes that are not necessarily performed in chronological order but are performed in parallel or individually. In addition, in this specification, the term "system" refers to an overall device that is made up of a plurality of devices, a plurality of means, etc.

[0063] <Additional Notes> The present embodiment includes the following disclosure.

[0064] (Appendix 1) An information processing method executed by an information processing device, an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device; a processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; An information processing method including:

[0065] (Appendix 2) The processing step includes determining a field to which the predetermined information source belongs and calculating the reliability according to the field. 1. The information processing method described in Appendix 1.

[0066] (Appendix 3) The processing step includes acquiring date and time information of the predetermined information source and calculating the reliability according to the date and time information. 1. The information processing method described in Appendix 1.

[0067] (Appendix 4) The processing step includes acquiring provider information of the predetermined information source, and calculating the reliability based on the provider information. 1. The information processing method described in Appendix 1.

[0068] (Appendix 5) The processing step includes acquiring information amount information of the predetermined information source, and calculating the reliability according to the information amount information. 1. The information processing method described in Appendix 1.

[0069] (Appendix 6) The processing step includes acquiring quality information of the predetermined information source and calculating the reliability according to the quality information. 1. The information processing method described in Appendix 1.

[0070] (Appendix 7) In the information processing device, an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device; a processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; A program for executing an information processing method including the steps of:

[0071] (Appendix 8) An information processing system executed by an information processing device, an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device; a processing step of performing processing based on the input sentence with reference to a predetermined information source and calculating the reliability of the predetermined information source; an output step of outputting the reliability together with the processing result; An information processing system that performs the following: [Explanation of symbols]

[0072] 10 Information Processing Systems 14 servers (information processing devices)

Claims

1. An information processing method executed by an information processing device, an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device; a processing step of generating text data based on the input sentence using a large-scale language model with reference to a predetermined information source, and calculating the reliability of the predetermined information source; an output step of outputting the confidence together with the text data resulting from the processing; Including, The processing step includes determining a field to which the predetermined information source belongs and calculating the reliability according to the field. Information processing methods.

2. the processing step includes a process of acquiring date and time information of the predetermined information source and calculating the reliability in accordance with the date and time information. The information processing method according to claim 1 .

3. the processing step includes a process of acquiring provider information of the predetermined information source and calculating the reliability in accordance with the provider information. The information processing method according to claim 1 .

4. the processing step includes a process of acquiring information amount information of the predetermined information source and calculating the reliability in accordance with the information amount information. The information processing method according to claim 1 .

5. the processing step includes a process of acquiring quality information of the predetermined information source and calculating the reliability in accordance with the quality information. The information processing method according to claim 1 .

6. In the information processing device, an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device; a processing step of generating text data based on the input sentence using a large-scale language model with reference to a predetermined information source, and calculating the reliability of the predetermined information source; an output step of outputting the confidence together with the text data resulting from the processing; Including, The processing step includes determining a field to which the predetermined information source belongs and calculating the reliability according to the field. A program for executing an information processing method.

7. An information processing system executed by an information processing device, an instruction acquisition step of acquiring an input sentence for processing performed by the information processing device; a processing step of generating text data based on the input sentence using a large-scale language model with reference to a predetermined information source, and calculating the reliability of the predetermined information source; an output step of outputting the confidence together with the text data resulting from the processing; Including, The processing step includes determining a field to which the predetermined information source belongs and calculating the reliability according to the field. Information processing system.

Images