Programs, methods, information processing devices, systems

Abstract

This involves comparing information disseminated in unspecified fields with information extracted from other sources. [Solution] A program for operating a computer, the program causing the computer's processor to execute: a step of receiving a text specification; a step of generating a prompt including an instruction to extract at least one source text containing a description similar to the specified text from information sources available to the artificial intelligence system and output information about the source text; an instruction to compare the extracted source text with the specified text; and a step of inputting the prompt into a large-scale language model provided by the artificial intelligence system and presenting the user with information about the source text and the result of the comparison between the specified text and the source text based on the response obtained from the large-scale language model.

Description

Technical Field

[0001] The present disclosure relates to programs, methods, information processing apparatuses, and systems.

Background Art

[0002] In recent years, information dissemination has been actively carried out mainly via the Internet. However, the disseminated information may include information of unknown authenticity. If false information spreads in society, incorrect judgments and actions based on that information may spread, leading to social chaos.

[0003] In Patent Document 1, a posting data acquisition unit 111 requests SNS posting data related to a railway line to be evaluated from an SNS server 103 and receives the SNS posting data. A highly reliable information acquisition unit 112 requests highly reliable information regarding the delay of the railway line to be evaluated from a highly reliable information distribution server 104 and receives the highly reliable information. An accuracy evaluation unit 123 determines the correctness of the presence or absence of the delay mentioned in the posting through comparison with the aggregation result of the highly reliable information, and a posting evaluation apparatus that determines an accuracy index is described.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The technology described in Patent Document 1 can only compare a predetermined information dissemination with information extracted from other information sources in a specific field, and cannot compare information dissemination in an unspecified field with information extracted from other information sources.

[0006] This disclosure involves comparing information disseminated in an unspecified field with information extracted from other sources. [Means for solving the problem]

[0007] A program for operating a computer, the program causing the computer's processor to perform the following steps: receiving a text specification; generating a prompt including an instruction for extracting at least one source text containing a description similar to the specified text from information sources available to the artificial intelligence system and outputting information about the source text; and an instruction for comparing the extracted source text with the specified text; and inputting the prompt into a large-scale language model provided by the artificial intelligence system and presenting the user with information about the source text and the result of the comparison between the specified text and the source text based on the response obtained from the large-scale language model. [Effects of the Invention]

[0008] According to this disclosure, it is possible to compare information disseminated in an unspecified field with information extracted from other sources. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the functional configuration of System 1. [Figure 2] This is a block diagram showing the functional configuration of the terminal device 10. [Figure 3] This is a block diagram showing the functional configuration of Server 20. [Figure 4] This diagram shows the data structure of User Table 2021. [Figure 5] This diagram shows the data structure of the information source table 2022. [Figure 6] This diagram shows the data structure of the prompt information table 2023. [Figure 7] This diagram shows the data structure of the response information table 2024. [Figure 8] This is a flowchart of the text comparison process according to the present embodiment. [Figure 9] This is a schematic diagram showing an example of the reception screen D1 for the specified text. [Figure 10] This is a schematic diagram showing an example of the first result screen D2. [Figure 11] This is a schematic diagram showing an example of the second result screen D3. [Figure 12] This is a flowchart of a modified example of the text comparison process. [Figure 13] This is a schematic diagram showing an example of the extraction result screen D4. [Figure 14] This is a schematic diagram showing an example of the comparison result screen D5. [Figure 15] This is a block diagram showing the basic hardware configuration of the computer 90.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, an embodiment of the present invention will be described in detail based on the drawings. In the drawings for explaining the embodiment, the same reference numerals are generally given to the same components, and the repeated description thereof will be omitted.

[0011] <1 Configuration Diagram of the Entire System> FIG. 1 is a block diagram showing an example of the overall configuration of the system 1. The system 1 shown in FIG. 1 includes, for example, a terminal device 10, a server 20, and an artificial intelligence system 40. The terminal device 10, the server 20, and the artificial intelligence system 40 are communicatively connected via, for example, a network 80.

[0012] In FIG. 1, the number of terminal devices 10 included in the system 1 is not limited to one. The number of terminal devices 10 included in the system 1 may be two or more.

[0013] In FIG. 1, an example in which the system 1 includes one server 20 is shown, but the number of servers 20 included in the system 1 is not limited to one. The server 20 may be composed of a plurality of servers according to the functions it has. Also, the server 20 may be, for example, an aggregate of a plurality of devices regarded as one server. The way of distributing the plurality of functions required to implement the server 20 according to the present embodiment for one or more hardware can be appropriately determined in view of the processing capabilities of each hardware and / or the specifications required for the server 20, etc.

[0014] The terminal device 10 shown in FIG. 1 is realized, for example, by a mobile terminal such as a smartphone or a tablet corresponding to a mobile communication system. In addition to this, the terminal device 10 may be realized, for example, by a stationary PC (Personal Computer) or a laptop PC. Also, the terminal device 10 may be realized, for example, by a wearable terminal such as an HMD (Head Mount Display).

[0015] The terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19. The input device 13 is a device (for example, a touch panel, a touch pad, etc.) for receiving an input operation from the user. The output device 14 is a device (display, speaker, etc.) for presenting information to the user.

[0016] The server 20 is realized, for example, by an information processing device connected to the network 80. As shown in FIG. 1, the server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29. The input / output IF 23 functions as an interface for an input device for receiving an input operation from the user and an output device for presenting information to the user.

[0017] The artificial intelligence system 40 is implemented, for example, by one or more information processing devices connected to the network 80. The artificial intelligence system 40 may also be configured as part of the server 20.

[0018] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the terminal device 10, server 20, and artificial intelligence system 40, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0019] <Terminal device configuration> Figure 2 is a block diagram showing an example configuration of the terminal device 10 shown in Figure 1. As shown in Figure 2, the terminal device 10 comprises a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a storage unit 180, and a control unit 190. Each block included in the terminal device 10 is electrically connected, for example, by a bus.

[0020] The communication unit 120 performs processing such as modulation and demodulation processing for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to an external source (for example, the server 20). The communication unit 120 performs reception processing on the signal received from an external source and outputs it to the control unit 190.

[0021] The input device 13 is a device for a user operating the terminal device 10 to input instructions or information. The input device 13 can be implemented, for example, by a touch-sensitive device 131 on which instructions are input by touching the operating surface. If the terminal device 10 is a PC, the input device 13 may be implemented by a reader, keyboard, mouse, etc. The input device 13 converts the instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may also include, for example, a receiving port that accepts electrical signals input from an external input device.

[0022] The output device 14 is a device for presenting information to the user operating the terminal device 10. The output device 14 is implemented, for example, by a display 141. The display 141 displays data according to the control of the control unit 190. The display 141 is implemented, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0023] The audio processing unit 17 performs, for example, digital-to-analog conversion processing of the audio signal. The audio processing unit 17 converts the signal received from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is implemented, for example, by an audio processing processor. The microphone 171 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 17. The speaker 172 converts the audio signal received from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0024] The storage unit 180 is implemented by, for example, memory 15 and storage 16, and stores data and programs used by the terminal device 10. The storage unit 180 stores, for example, user information 181.

[0025] User information 181 stores information about the user performing the operation. User information includes, for example, user ID, name, age, address, date of birth, and date of registration for the service.

[0026] The control unit 190 is realized when the processor 19 reads a program stored in the memory unit 180 and executes instructions contained in the program. The control unit 190 controls the operation of the terminal device 10. By operating according to the program, the control unit 190 performs the functions of an operation reception unit 191, a transmission / reception unit 192, and a presentation control unit 193.

[0027] The operation reception unit 191 processes instructions or information input from the input device 13. For example, the operation reception unit 191 receives instructions or information input from a touch-sensitive device 131 or the like.

[0028] Furthermore, the operation reception unit 191 receives voice information input from the microphone 171. Specifically, for example, the operation reception unit 191 receives voice data input from the microphone 171 and converted into digital data by the voice processing unit 17.

[0029] The transmitting / receiving unit 192 performs processing to enable the terminal device 10 to send and receive data with an external device such as the server 20 in accordance with a communication protocol. Specifically, for example, the transmitting / receiving unit 192 sends instructions input by the user to the server 20. The transmitting / receiving unit 192 receives information provided by the server 20.

[0030] The presentation control unit 193 controls the output device 14 and other devices in order to present information such as information provided by the server 20 to the user.

[0031] <Server Functional Configuration> Figure 3 shows an example of the functional configuration of server 20. As shown in Figure 3, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0032] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0033] The storage unit 202 includes, for example, a user table 2021, an information source table 2022, a prompt information table 2023, a response information table 2024, a prompt template 2028, and the like.

[0034] User Table 2021 is a table that stores information about users who register for the services described herein.

[0035] Information Source Table 2022 is a table that stores information about information sources that can be obtained via Network 80 (for example, including all information sources that transmit information via Network 80, such as academic paper / journal article databases, electronic journals, news media, and public institutions).

[0036] The prompt information table 2023 is a table that stores prompt information, which is information related to prompts. Specifically, for example, prompt information is input data to be embedded in each field of the prompt template 2028, which will be described later.

[0037] The response information table 2024 is a table that stores response information, which is information about the responses that the large-scale language model of the artificial intelligence system 40 outputs in response to prompt inputs.

[0038] Prompt template 2028 is prompt template data having fields for embedding specified input data. That is, by inputting predetermined input data into prompt template 2028, a prompt to be input to a large-scale language model is generated. Prompt template 2028 is, for example, text data. Prompt template 2028 is, for example, a combination of at least one of the instructions 2028a, 2028b, 2028c, 2028d, and 2028e, which are sentences indicating the respective instructions. In this disclosure, when sentences included in instructions 2028a to 2028e are used to create a prompt, it is not necessary for all of the sentences included in instructions 2028a to 2028e to be combined; some or more parts that make up each of the sentences in instructions 2028a to 2028e may be combined. In the following example, the fields for embedding input data are shown in the parts enclosed in curly braces.

[0039] Instruction 2028a includes, as a first example, instructions for an artificial intelligence system to extract at least one source document containing a description similar to the specified document from available sources, and to output information about the source document.

[0040] (Example 1 of Instruction 2028a) Extract at least one source document from your available sources that contains a description similar to the specified #Specified Text, and output the information according to the #Source Document Information section. #specified text {specified text} #Information about the source text • Title of the source • Access URL to the source document • Name and confidence level of the information source from which the data was extracted • Date the source document was published • The main text of the source document

[0041] Instruction 2028a, as a second example, includes instructions to extract at least one source document containing a description similar to the specified document from a pre-specified source, and to output information about the source document.

[0042] (Second example of Instruction 2028a) Extract at least one source document containing a description similar to the specified #specified document from any of the specified #information sources, and output the information according to the items in #Source Document Information. #specified text {specified text} #Source (Source 1) • Source name: {Source name} • Source URL: {Access destination} • Confidence level: {confidence level} (Source 2) • Source name: .... #Information about the source text • Title of the source • Access URL to the source document • Name and confidence level of the information source from which the data was extracted • Date the source document was published • The main text of the source document

[0043] Instruction 2028a includes, as a third example, an instruction to obtain the text written on a specified webpage as the specified text from information used to identify the specified webpage (webpage identification information such as a URL). In this case, the webpage is a webpage that contains text data or image data containing characters, such as a webpage assigned to an individual post on a social networking service (SNS). The text written on the webpage is, for example, the text displayed on the terminal when the webpage is accessed. Thus, for example, the artificial intelligence system 40 obtains the text on the webpage as the specified text by accessing information used to identify the webpage.

[0044] (Third example of Instruction 2028a) Using the specified #SNS posts as the specified text, extract at least one source text from any of the specified #information sources that contains similar content to the specified text, and output the information according to the #Source Text Information section. #SNSpost {SNS post URL} #Source (Source 1) • Source name: {Source name} • Source URL: {Access destination} • Confidence level: {confidence level} (Source 2) • Source name: .... #Information about the source text • Title of the source • Access URL to the source document • Name and confidence level of the information source from which the data was extracted • Date the source document was published • The main text of the source document

[0045] Instruction 2028b includes, for example, instructions to compare the extracted source text with a specified text. Instruction 2028b also includes, for example, the following instructions: • An instruction to output a degree of match between the source text and the specified text. • An instruction to output the basis for determining the degree of agreement. • An instruction to output at least one of the following: information regarding the matching portions and information regarding the differences between the specified text and the source text.

[0046] (Example from instruction 2028b) Compare the extracted source text with the specified text, and output the information according to the items in #Information about comparison results. #Information on comparison results • Matching parts between the specified text and the source text • Differences between the specified text and the source text • The degree of matching between the specified text and the source text. • Basis for determining the degree of agreement

[0047] In addition, Instruction 2028b may include instructions to compare the source document and the designated document with emphasis on at least one specific viewpoint. This specific viewpoint may be, for example, at least one of the viewpoints relating to each item of the “Viewpoint” column in Source Table 2022, or a viewpoint different from the viewpoints relating to each item of the “Viewpoint” column in Source Table 2022. The viewpoint items in Instruction 2028b may be predetermined, for example, by the service provider of this disclosure.

[0048] (Additional example for instruction 2028b) When comparing the source text with the specified text, please pay attention to the following points. #perspective • Perspective 1: Clarity of Sources • Perspective 2: Transparency of Information

[0049] Instruction 2028c includes an instruction to output a credibility score indicating the degree of credibility of a specified document, based on the comparison result and the credibility of the source information of the source document. For example, instruction 2028c includes an instruction to output a credibility score by performing an arbitrary calculation based on the degree of agreement indicated in instruction 2028b and the credibility score presented in instruction 2028a. The credibility score may also be calculated by the server 20 using any method, such as the calculation formula below.

[0050] (Example from instruction 2028c) Please determine the credibility using the formula. If multiple source documents were compared with the specified document, the [Source Confidence] should be the average of the source confidence scores of each source document compared with the specified document. #calculation formula [Credibility] = [Agreement] * [Source Credibility] / 100

[0051] Instruction 2028d includes instructions to select at least one source document from multiple source documents based on the reliability of the information source from which the source documents were extracted, and to compare it with the specified document, if multiple source documents are extracted. For example, if multiple source documents are extracted, instruction 2028d includes instructions to select a source document under the conditions exemplified below and to compare it with the specified document. Select a predetermined number of source documents, ordered by the reliability of the information source. Select all source documents whose reliability is above a specified value.

[0052] (Example from instruction 2028d) If multiple source documents can be extracted, select the three source documents with the highest reliability and compare each of them with the specified document.

[0053] Instruction 2028e includes, as a first example, instructions for determining the reliability of a source of a source document based on at least one specific perspective. This specific perspective may be, for example, at least one of the perspectives relating to each item in the Source Table 2022, or a different perspective from those relating to each item in the Source Table 2022. If prompt template 2028 includes instruction 2028e, the reliability of the source is output from the large-scale language model. Therefore, in this case, prompt template 2028 does not need to include information regarding the reliability of each source, as shown in the second example of instruction 2028a. The perspective items in instruction 2028e are predetermined, for example, by the service provider of this disclosure.

[0054] (Example 1 of Instruction 2028e) Once you have extracted the source text, determine the reliability of the information source for that text by scoring each of the following criteria on a scale from 0 to 100, and then averaging the scores for each criterion to determine its reliability. #perspective • Perspective 1: Clarity of Sources • Perspective 2: Transparency of Information • Perspective 3: Political bias • Perspective 4: Appropriateness of the interview format ...

[0055] Furthermore, as a second example, instruction 2028e may include instructions for determining the reliability of a source document based on at least one specific viewpoint. For example, instruction 2028e may include instructions for determining the reliability of a source document by analyzing the content of the source document itself from a specific viewpoint.

[0056] (Example 2 of Instruction 2028e) Once you have extracted the source text, determine its reliability by analyzing its content from the following perspectives. Score the source text for each perspective on a scale of 0 to 100, and determine the reliability of the source text by averaging the scores for each perspective. #perspective • Perspective 1: Clarity of Sources • Perspective 2: Transparency of Information • Perspective 3: Political bias • Perspective 4: Appropriateness of the interview format ...

[0057] Furthermore, it is preferable that the response data from the large-scale language model be in a structured format for each item, such as a JSON string. This makes it easy to extract the text data for each item that needs to be extracted from the response data output from the large-scale language model in response to the input prompt. For this reason, it is preferable that the prompt template 2028 includes instructions to structure the response for each item. This makes it possible to obtain the response data from the large-scale language model in a structured format for each item.

[0058] The control unit 203 is realized when the processor 29 reads a program stored in the memory unit 202 and executes instructions contained in the program. By operating according to the program, the control unit 203 performs functions as a receive control module 2031, a transmit control module 2032, a prompt generation module 2033, a data analysis module 2034, and a presentation module 2035.

[0059] The receiving control module 2031 controls the process by which the server 20 receives signals from external devices according to a communication protocol. Specifically, for example, the receiving control module 2031 receives signals transmitted from the terminal device 10 and signals transmitted from the artificial intelligence system 40.

[0060] The transmission control module 2032 controls the process by which the server 20 transmits signals to external devices according to a communication protocol. Specifically, for example, the transmission control module 2032 transmits information related to various UIs, which will be displayed on the display 141 of the terminal device 10, to the terminal device 10. Also, for example, the transmission control module 2032 transmits a request including a prompt to the artificial intelligence system 40.

[0061] The prompt generation module 2033 generates prompts to be input into the large-scale language model of the artificial intelligence system 40. Specifically, it generates prompts by inputting information received from the user into designated fields in the prompt template 2028. For example, the prompt generation module 2033 generates prompts by inputting information obtained from the information source table 2022 and the prompt information table 2023 into the fields of each item in the prompt template 2028.

[0062] For example, the prompt generation module 2033 generates a prompt by entering the following information into each field of the prompt template 2028. • {Specified Text}: Information contained in the "Specified Text" item of the Prompt Information Table 2023. • {Information Source Name}: Information contained in the "Information Source Name" item in Information Source Table 2022. • {Access Destination}: Information contained in the "Access Destination" field of Information Source Table 2022. • {Confidence Level}: Information included in the "Confidence Level" item of Information Source Table 2022. • {SNS post URL}: Information received from the user to identify the specified webpage (webpage identification information such as URL)

[0063] The data analysis module 2034 controls the process of analyzing response data output from a large-scale language model and extracting necessary information. Specifically, the data analysis module 2034 extracts information about the source text and information about the comparison results between the specified text and the source text from the response data.

[0064] For example, as mentioned above, accepting response data in a structured format such as JSON makes it easier to analyze the response data. The data analysis module 2034 divides the structured response data into individual items and retrieves information corresponding to each item from the response data to generate the result UI described later. Even if the response data is not structured, the data analysis module 2034 can extract text data for any item from the response data using any natural language processing technique.

[0065] Furthermore, if multiple source documents are output from a large-scale language model (i.e., the response data contains information about multiple source documents), the data analysis module 2034 may, based on predetermined criteria, select a predetermined number of source documents from the multiple source documents and extract information about the selected source documents and information about the comparison results between those source documents and the specified documents from the response data.

[0066] The specified criteria may include, for example, a criterion for the reliability value of the information sources in the source text, and a criterion for the degree of similarity between the source text and the specified text. When multiple criteria are combined, one of the criteria may be applied preferentially.

[0067] For example, based on criteria regarding the reliability of the source information of the source text, the data analysis module 2034 selects the source text as follows: Select a predetermined number of source documents, ordered by the reliability of the information source. Select all source documents whose reliability is above a specified value.

[0068] For example, based on criteria regarding the degree of similarity between the source text and the specified text, the data analysis module 2034 selects the source text as follows: Select a predetermined number of source documents, ordered by their degree of match with the specified text. Select all source texts whose degree of matching with the specified text is equal to or greater than a predetermined value.

[0069] Furthermore, if multiple source texts are output from a large-scale language model, the data analysis module 2034 may sort the response data for each source text in an order based on predetermined criteria, and display the information about the source text and the comparison results between the specified text and the source text in the results UI described later, in the sorted order. The predetermined criteria may include at least one of the following: a criterion for the confidence value of the information source of the source text, and a criterion for the degree of agreement between the source text and the specified text. When combining multiple criteria, one of the criteria may be applied preferentially.

[0070] For example, based on criteria regarding the reliability of the source information in the source text, the data analysis module 2034 sorts the response data as follows: Sort the response data by source text in order of highest reliability of the information source.

[0071] For example, based on criteria for the degree of similarity between the source text and the specified text, the data analysis module 2034 sorts the response data as follows: Sort the response data for each source text in descending order of its degree of match with the specified text.

[0072] Presentation module 2035 presents various UIs to the user. For example, presentation module 2035 presents various UIs, etc., as described later, to the user.

[0073] <Functional Configuration of Artificial Intelligence System 40> The artificial intelligence system 40 is a system that includes at least one information processing device and outputs a response to a request that has been received as input.

[0074] The artificial intelligence system 40 is, for example, a large-scale language model (LLM) system that performs natural language processing. For example, when a server 20 sends a prompt to the large-scale language model of the artificial intelligence system 40, the artificial intelligence system 40 sends data including text data to the server 20 as a response to the prompt. The artificial intelligence system 40 may also be a generative AI system that generates image (including still images and videos) data and audio data. For example, when a server 20 sends a prompt to the model of the artificial intelligence system 40, the artificial intelligence system 40 sends data including image data and audio data to the server 20 as a response to the prompt. The artificial intelligence system 40 may also be a system capable of processing multiple types of data, such as text, images, audio, and videos, in a single operation. The artificial intelligence system 40 includes, for example, ChatGPT, Perplexity, Copilot, Gemini, Midjourney, Stable Diffusion, Sora, etc. The artificial intelligence system 40 may also be an artificial intelligence system provided by the operator of the server 20.

[0075] In this embodiment, the artificial intelligence system 40 is described using a system that handles a large-scale language model as an example. In this embodiment, a request to the artificial intelligence system 40 includes text data of a prompt to be input to the large-scale language model, and the response from the artificial intelligence system 40 includes response data output from the large-scale language model.

[0076] <2 Data Structure> Figures 4 to 7 show examples of the data structure of the information stored by server 20. Note that Figures 4 to 7 are just examples and do not exclude any data not shown.

[0077] Figure 4 shows the data structure of User Table 2021. User Table 2021 is a table with User ID as the primary key and columns such as User Name.

[0078] The "User ID" field stores the user ID that identifies the user.

[0079] The "Username" field stores the user's name. The username can be any string, such as the user's full name or nickname.

[0080] Figure 5 shows the data structure of the Information Source Table 2022. The Information Source Table 2022 is a table with Information Source ID as the primary key and columns such as Information Source Name, Access Destination, Perspective, and Confidence Level.

[0081] The "Source ID" field stores the source ID that identifies the source.

[0082] The "Information Source Name" field stores the name of the information source. This name can include any string of characters, such as the name of an information provider (e.g., a public institution or news media), a website, a journal or magazine, or a publication.

[0083] The "Destination" field stores the link to access the information source. For example, the "Destination" field stores a string representing the URL to access the information source's website.

[0084] The "Perspective" field stores information about the evaluation of an information source from a given perspective. As shown in Figure 5, the information source table 2022 stores information about the evaluation of an information source from at least one perspective, such as "Perspective 1," "Perspective 2," etc., as fields related to the "Perspective" field. Specifically, the "Perspective" field stores a numerical value indicating the degree of evaluation of the information source from a given perspective. For example, the higher the evaluation for each perspective, the larger the numerical value stored in the "Perspective" field.

[0085] Furthermore, the predetermined viewpoints for which evaluations are stored in the "Viewpoint" item specifically include viewpoints for judging the reliability of the information source. For example, the predetermined viewpoints include the following viewpoints. That is, "Viewpoint 1," "Viewpoint 2," etc. in Figure 5 can be reinterpreted as, for example, "Clarity of Source," "Transparency of Information," etc. • Perspectives on clarity: clarity of sources, transparency of information, etc. • Accuracy-related aspects: accuracy of information, consistency of information, date and time of information publication and update frequency, etc. Objectivity-related aspects: Objectivity of information, fairness (political bias, etc.), etc. • Perspectives on expertise: the specialized knowledge possessed by the information source, the author's research and writing skills, the history, authority, and track record of the information source, etc.

[0086] The "Reliability" field stores information about the reliability of the information source. Specifically, the reliability of the information source is stored as a numerical value in the "Reliability" field. For example, the higher the reliability, the larger the numerical value stored in the "Reliability" field. Reliability may also be ranked as "High," "Medium," "Low," etc., according to the numerical value of the reliability.

[0087] The reliability score may be determined based on an evaluation of each perspective of the information source. For example, the value stored in the "Reliability" item may be determined based on the numerical values ​​stored in each item of the "Perspective" item. For example, the value of the "Reliability" item may be determined by performing any calculation (sum, average, etc.) on the numerical values ​​of each item of the "Perspective" item.

[0088] Furthermore, the reliability may be predetermined for each information source by the service provider of this disclosure, etc., rather than being determined by the numerical value stored in the item "Perspective". In this case, the item "Perspective" does not need to be included in Information Source Table 2022.

[0089] Figure 6 shows the data structure of the prompt information table 2023. The prompt information table 2023 is a table with prompt information ID as the primary key and columns such as user ID and specified text.

[0090] The "Prompt Information ID" field stores the prompt information ID that identifies the prompt information. Prompt information is the information combined with the prompt template to generate a prompt.

[0091] The "User ID" field stores the User ID of the user who sent the prompt information.

[0092] The item "Specified Text" stores information about the specified text, which is a text specified by the user. For example, the item "Specified Text" stores text data of a text entered by the user via the terminal device 10, or text data of a text acquired by the server 20 based on information entered by the user to identify the specified text (for example, identification information of an SNS post related to the specified text, or identification information of a web page containing the specified text).

[0093] The "Source ID" field stores the Source ID of the source specified when extracting source text in the prompt. The "Source ID" field may store one or more Source IDs.

[0094] Figure 7 shows the data structure of the response information table 2024. The response information table 2024 is a table with a response information ID as the primary key and columns such as prompt information ID and response information.

[0095] The item "Response Information ID" is an item that stores a response information ID that identifies the response information sent from the artificial intelligence system 40 as a response to a prompt.

[0096] The item "Prompt Information ID" stores the prompt information ID of the prompt information used to generate the corresponding prompt for the response information. Specifically, the item "Prompt Information ID" stores the prompt information ID of the prompt information used to generate the prompt that the server 20 sent to the artificial intelligence system 40 in order to receive the response information identified by the response information ID. For example, if the server 20 sends a prompt generated using prompt information with prompt information ID: P001 to the artificial intelligence system 40, and the artificial intelligence system 40 outputs response information with response information ID: RP001, then the record with response information ID RP001 will store P001 as the prompt information ID.

[0097] The item "Response Information" is an item that stores the response information output from the artificial intelligence system 40. Specifically, the item "Response Information" stores data related to the response output by the artificial intelligence system 40 in response to a received request. For example, the item "Response Information" stores data in a format appropriate to the type of artificial intelligence system 40 (text data, image data, audio data, etc.).

[0098] <3 operations> The text comparison process of this embodiment will now be described. The text comparison process involves sending a request to the artificial intelligence system 40 that includes prompts, which include an instruction to extract source texts similar to the specified text, generated based on the specified text specified by the user, and an instruction to compare the specified text with the source text. The comparison result between the specified text and the source text is then presented to the user based on the response information obtained from the artificial intelligence system 40. Figure 8 is a flowchart of the text comparison process of this embodiment.

[0099] First, the terminal device 10 executes an application to extract source text and compare it with a specified text. Note that the extraction of source text and the comparison may be performed via a web browser. For example, the user operates the terminal device 10 to select an application and has the terminal device 10 execute it. Once the terminal device 10 executes the application, the control unit 190 displays a login screen for user authentication on the display 141.

[0100] On the login screen, the user enters, for example, a user ID and password. However, the information entered by the user on the login screen is not limited to a user ID and password; the user may also log in by entering biometric information, etc.

[0101] In step S11, the server 20 accepts access from the terminal device 10 and authenticates the user by accepting the user ID and password.

[0102] The server 20 presents the UI for receiving the specified document to the user via the terminal device 10. Specifically, for example, the server 20 generates the UI for receiving the specified document based on the information about the information source stored in the information source table 2022, and displays the screen for receiving the specified document on the display 141 of the terminal device 10.

[0103] Figure 9 is a schematic diagram showing an example of the specified text reception screen D1. The specified text reception screen D1 includes operation object D11, operation object D12, operation object D13, display object D14, and operation object D15.

[0104] Operation object D11 accepts operations for inputting text specifications. Operation object D11 accepts input of specified text from the user, for example, by accepting text data input from the user.

[0105] Operation object D13 accepts an operation to input the specification of information sources to be searched in the source document. For example, when operation object D13 accepts an operation from the user, it displays a list of information source names stored in the information source table 2022 on the specified document acceptance screen D1, making them selectable. Terminal device 10 accepts the specification of information sources to be searched in the source document by accepting the selection of each object representing the information source name from the user. Each object representing the information source name may be divided into groups according to the numerical value of the reliability of the information source, such as "high reliability," "medium reliability," and "low reliability," based on a predetermined criterion regarding the numerical value of the reliability of the information source, and each group may be selectable all at once.

[0106] Note that operation object D13 is not required; in that case, at least one information source predetermined by the service provider of this disclosure is designated as the source of the source document. For example, an information source with a confidence level of "high" in information source table 2022 may be designated as the source.

[0107] Display object D14 displays the specified information source. For example, display object D14 displays a list of information sources that the terminal device 10 has received a request for when operation object D13 receives an operation from the user.

[0108] Operation object D15 receives input from the user instructing it to extract source text for a specified text and compare the specified text with the source text. When operation object D15 receives input from the user, terminal device 10 sends an instruction to server 20 to generate a prompt and send the prompt to artificial intelligence system 40, along with the information obtained from the input operations to operation objects D11 and D13.

[0109] In step S12, the server 20 receives instructions to specify a text and an information source, as well as instructions to generate a prompt and send a request to the artificial intelligence system 40. Specifically, the server 20 receives information about the specified text and information about the specified information source from the terminal device 10, based on the operation received by the user in the specified text reception UI presented in step S11. The information about the specified text and information about the specified information source are examples of prompt information in this embodiment.

[0110] Server 20 stores the prompt information received from terminal device 10 in prompt information table 2023. Specifically, for example, the following information is stored in each item of a new record in prompt information table 2023. • Prompt Information ID: Newly assigned prompt information ID • User ID: The user ID received in step S11 • Specified text: Text data of the specified text received. • Source ID: The source ID of the source that was specified.

[0111] Server 20 searches the information source table 2022 based on the information source ID stored in the prompt information table 2023 and retrieves the information for the corresponding record's fields: "Information Source Name," "Access Destination," and "Trust Level."

[0112] In step S13, the server 20 generates a prompt that includes an instruction for the artificial intelligence system to extract at least one source document containing a description similar to the document that received the designation from the available information sources, and an instruction to compare the extracted source document with the designated document. Specifically, the server 20 generates a prompt to send to the artificial intelligence system 40 by calling a prompt template 2028 stored in the memory unit 202 and inputting the prompt information stored in the prompt information table 2023 into the corresponding fields in the prompt template 2028.

[0113] For example, the prompt generation module 2033 generates a prompt by inputting the following information into each field of the prompt template 2028. • {Specified text}: Information stored in the "Specified text" item of the prompt information table 2023 in step S12. • {Information Source Name}: Information obtained in step S12 from the "Information Source Name" item in Information Source Table 2022. • {Access Destination}: Information obtained in step S12 from the "Access Destination" item in Information Source Table 2022. • {Confidence Level}: Information obtained from the "Confidence Level" item in Information Source Table 2022 in step S12.

[0114] In step S14, the server 20 inputs the generated prompt into the large-scale language model provided by the artificial intelligence system 40. Specifically, for example, the server 20 sends a request containing the text data of the prompt generated in step S13 and the necessary parameters (e.g., specifying the number of tokens) to the API endpoint of the artificial intelligence system 40.

[0115] When the artificial intelligence system 40 receives a request from the server 20, it sends response information, which is a response to the request, to the server 20 via the API endpoint.

[0116] In step S15, the server 20 receives response information sent from the artificial intelligence system 40. The response information includes text data (answer data) from the large-scale language model in response to the prompt included in the request sent by the server 20. The server 20 stores the response information in the storage unit 202. Specifically, for example, in a new record in the response information table 2024, the prompt information ID assigned in step S12 is stored in the item "Prompt Information ID", and the response information received from the artificial intelligence system 40 is stored in the item "Response Information".

[0117] In step S16, the server 20 extracts the necessary data from the acquired response information. Specifically, for example, the server 20 extracts from the response data information about the extracted source text and information about the comparison result between the specified text and the source text. The information about the source text is, for example, the following: • The title of the source text from which the source text was extracted (the title of the webpage or article in which the source text is published) • Source of the extracted text • Reliability of the information source • Source of the extracted text (e.g., web page accessed) • Publication time of the source document (date, time, etc.) • The main text of the source document Information regarding the comparison results between the specified text and the source text includes, for example, the following: • Matching parts between the specified text and the source text • Differences between the specified text and the source text • The degree of matching between the specified text and the source text. • Basis for determining the degree of agreement Comments on the comparison results

[0118] If multiple source documents are extracted from the response data, the server 20 may use the data analysis module 2034 to select a predetermined number of source documents from the multiple source documents based on predetermined criteria, and extract information about the selected source documents and information about the comparison results between those source documents and the specified documents from the response data. Alternatively, the response data for each source document may be rearranged in an order based on predetermined criteria.

[0119] Server 20 generates a results UI to present the user with information about the source text and information about the comparison result between the specified text and the source text, based on the information of each item extracted from the response data. Specifically, for example, Server 20 generates a results UI that corresponds to the response data by inputting the information of each item extracted from the response data into fields provided in the results UI corresponding to each item.

[0120] In step S17, the server 20 presents the user with information about the source document and the results of comparing the specified document with the source document. Specifically, the server 20 presents the result UI generated in step S16 to the user via the terminal device 10. For example, the server 20 displays the result screen on the display 141 of the terminal device 10.

[0121] Figure 10 is a schematic diagram showing an example of the first results screen D2. As shown in Figure 10, the first results screen D2 displays display object D21 and display object D22 for each extracted source document.

[0122] Display object D21 displays information related to the source document. Specifically, display object D21 displays the following information, for example: • The title of the source text from which the source text was extracted (the title of the webpage or article in which the source text is published) • Source of the extracted text • Reliability of the information source • Source of the extracted text (e.g., web page accessed) • Publication time of the source document (date, time, etc.) • The main text of the source document

[0123] Display object D22 displays information regarding the comparison result between the specified text and the source text. Specifically, display object D22 displays information such as the following: • Matching parts between the specified text and the source text • Differences between the specified text and the source text • The degree of matching between the specified text and the source text. • Basis for determining the degree of agreement Comments on the comparison results

[0124] In step S16, if the data analysis module 2034 has extracted information from the response data relating to a predetermined number of source documents from among multiple source documents, the first results screen D2 will display only the information relating to the source documents selected by the data analysis module 2034 using display objects D21 and D22. On the other hand, information relating to source documents not selected by the data analysis module 2034 will not be displayed.

[0125] Furthermore, in step S16, if the data analysis module 2034 has rearranged the source text information included in the response data into a predetermined order, the first results screen D2 will display the display objects D21 and D22 related to each source text in that order.

[0126] Figure 11 is a schematic diagram showing an example of the second results screen D3. As shown in Figure 11, the second results screen D3 includes the display object D31 and the operation objects D331 to D333.

[0127] Display object D31 displays information regarding the credibility of the specified document. Specifically, for example, display object D31 displays the following information: • Credibility score • Comments regarding the credibility of the specified document. For example, comments on the basis for determining the credibility, or the results of comparing the specified document with the designated source document.

[0128] Operation objects D331 to D333 accept input for specifying questions regarding information presented to the user. For example, when operation objects D331 to D333 accept input from the user, terminal device 10 sends a request to server 20 to input questions about the string displayed on the object that accepted the operation into the large-scale language model. Server 20 generates a prompt containing the questions received from terminal device 10 and inputs it into the large-scale language model of artificial intelligence system 40. When server 20 receives a response from artificial intelligence system 40, it extracts the answer data contained in the response and presents it to the user via terminal device 10.

[0129] In the example in Figure 11, the operation object D331 displays a question about the content of a specified document, for example, a specific description contained in the specified document. For example, an input operation to operation object D331 causes the server 20 to generate a prompt containing instructions that explain the details of the description based on the description in the source document, and input this into the large language model to obtain an answer to the user's question from the large language model.

[0130] In the example in Figure 11, the operation object D332 displays a question regarding supplementary information for the specified document. For example, an input operation to operation object D332 causes the server 20 to generate a prompt that includes instructions to present supplementary information for the specified document, such as information not included in the specified document but included in the source document, and input this into the large language model to obtain an answer to the user's question from the large language model.

[0131] In the example shown in Figure 11, the operation object D333 displays a question about the content of the source document. For example, an input operation to operation object D333 causes the server 20 to generate a prompt containing instructions for explaining, summarizing, or describing the source document, and input this into the large-scale language model to obtain an answer to the user's question from the large-scale language model.

[0132] In addition, the server 20 may display objects on the first result screen D2 and the second result screen D3 for inputting evaluations (feedback) regarding information about the source text presented to the user and information about the comparison results between the specified text and the source text. For example, an object for receiving user evaluations of the information displayed on each object may be displayed near at least one of the display object D21, display object D22, and display object D31. The object may accept user evaluations such as "good" or "bad" in a selectable format. The terminal device 10 receives information about the user's evaluation of each piece of information by accepting operations on the object and transmits it to the server 20. The server 20 receives the user's evaluation information and stores it in the storage unit 202. The server 20 transmits the response data related to the information presented to the user and the user's evaluation information to the artificial intelligence system 40. This allows the user's evaluation of the response data to be used for training a large-scale language model or for the operator of the artificial intelligence system 40 to improve the artificial intelligence system 40.

[0133] <Variation> Each step of the above text comparison process can be executed on either the terminal device 10 or the server 20. While the above description shows an example of executing each step in a specific order, the execution order of each step is not limited to the example described, unless there are dependencies.

[0134] In the above embodiment, the server 20 received information such as the specified text from the user by presenting the specified text reception screen D1 to the user, and presented information about the source text and information about the comparison result between the specified text and the source text by presenting the first result screen D2 and the second result screen D3 to the user. However, the server 20 may also execute each step of the text comparison process while the web page containing the specified text is displayed on the display 141 of the terminal device 10.

[0135] For example, terminal device 10 is pre-running an application for text comparison processing. Upon user input, terminal device 10 launches a web browser or similar program, accesses a webpage containing the specified text, and displays the specified text on display 141.

[0136] For example, if a user operates the terminal device 10 and selects a range of text displayed on the display 141 using a web browser application, the terminal device 10 may send the text data of the selected range to the server 20, treating it as the specified text. The server 20 may then perform the text comparison process from step S12 onward based on the text data of the specified text sent from the terminal device 10. In this case, the information about the source text and the information about the comparison result between the specified text and the source text may be presented to the user by pop-up display or overlay display on the screen of the web browser application or the like that was displaying the specified text. This allows the user to easily compare the specified text with the source text.

[0137] Furthermore, in the above embodiment, the data analysis module 2034 of the server 20 performed various processes. However, the processes performed by the data analysis module 2034 may be executed by the artificial intelligence system 40. Specifically, the prompt generation module 2033 generates instructions regarding the processes performed by the data analysis module 2034 as prompts, and the server 20 inputs these instructions into the large-scale language model of the artificial intelligence system 40, thereby receiving results from the large-scale language model that are similar to the processing results performed by the data analysis module 2034. This reduces the processing performed by the server 20.

[0138] The following is an example of the processing performed by the data analysis module 2034, which can be executed by the artificial intelligence system 40. This process involves selecting a predetermined number of source documents from multiple source documents and extracting information about the selected source documents and the results of comparing those source documents with a specified document from the response data. By outputting only the information about the predetermined number of source documents selected from multiple source documents from the large-scale language model, the number of tokens can be reduced. • A process to sort the response data for each source text in an order based on predetermined criteria.

[0139] Furthermore, in the above embodiment, the prompt does not specify an information source, as in the first example of instruction 2028a, in which case the source text is extracted from an available information source by the artificial intelligence system 40. Alternatively, the information source from which the source text is extracted is specified, as in the second example of instruction 2028a. However, the information sources from which the artificial intelligence system 40 extracts the source text may be pre-trained. For example, the server 20 may, at predetermined timings (e.g., at regular intervals), input the information stored in the information source table 2022 into the large-scale language model of the artificial intelligence system 40, thereby training it on information about the information sources stored in the information source table 2022. The prompt generation module 2033 may then generate a prompt that includes an instruction to extract source text from a pre-trained information source.

[0140] Furthermore, in the above embodiment, source text extraction and comparison were performed for one specified text. However, it is also possible to accept multiple specified texts from the user and perform source text extraction and comparison for multiple specified texts.

[0141] Furthermore, in the above embodiment, the input of a specified text from the user was accepted by accepting text data from the user to the operation object D11. However, the server 20 may accept web page identification information from the user instead of text data of the specified text. The server 20 may then obtain the specified text from the web page based on that identification information. For example, the server 20 may obtain the text published on the web page identified by the accepted identification information from an external server as the specified text using any method. Any method could be, for example, obtaining the web page text via an API published by the web page provider, or obtaining the web page text using any web scraping technique.

[0142] Furthermore, in the above embodiment, the system accepted the user's specification of a text and determined the credibility of the specified text. However, it is also possible to accept the user's specification of the poster of a text in a predetermined text posting service (SNS, etc.) and determine the credibility of multiple texts posted by the specified poster (specified poster).

[0143] For example, the user enters identification information to identify the poster on the reception screen displayed on the terminal device 10. This information to identify the poster is information to identify the poster's account in the text posting service (for example, the poster's user ID, the URL of the poster's personal page, etc.). The terminal device 10 sends the received identification information to the server 20.

[0144] Server 20 retrieves at least one text posted by the poster on the text posting service based on the received identification information. For example, Server 20 retrieves one or more texts posted by the poster on the text posting service via the API by sending the poster's ID to the API endpoint provided by the text posting service.

[0145] Server 20 obtains the credibility of the obtained text by executing the process from step S13 onwards of the text comparison process, using at least one of the obtained texts as the specified text. At this time, for example, the prompt generation module 2033 includes instructions such as "All of the #specified texts were posted by the same poster A. After outputting the credibility of the #specified text, please determine the credibility of the poster by comprehensively considering the credibility and the #specified text, etc." in the prompt and inputs it into the large-scale language model, thereby obtaining the credibility of the poster from the large-scale language model.

[0146] Server 20 presents the acquired credibility score of the poster to the user via terminal device 10.

[0147] Furthermore, in the above embodiment, the reliability of the information source was determined, for example, by the information in the "Reliability" item of Information Source Table 2022, and this value remained constant unless the numerical value of the "Perspective" item was changed. However, the reliability value may be increased or decreased as appropriate. Specifically, the values ​​of the "Perspective" item may be weighted based on the perspectives that the service provider or user of the disclosed service considers important, and the reliability value may be determined based on the weighted values ​​of the "Perspective" item.

[0148] Furthermore, in the above embodiment, the viewpoints were predetermined in instructions 2028b and 2028e. However, the viewpoints may be determined as appropriate according to user input. For example, the designated document reception screen D1 may have an operation object for receiving input operations for viewpoints that the user considers important. The operation object may, for example, display the item names of the "Viewpoint" items in the information source table 2022. When the user selects a viewpoint they want to emphasize from each of the viewpoints displayed on the designated document reception screen D1, the terminal device 10 sends information about the accepted viewpoint items to the server 20. The server 20 updates instructions 2028b and 2028e of the prompt template 2028 with the accepted viewpoint item names to generate a prompt for the viewpoints that the user considers important.

[0149] Furthermore, in the above embodiment, in step S13, the server 20 generated a prompt including an instruction (first instruction) to extract at least one source document containing a description similar to the document that was specified from information sources available to the artificial intelligence system, and an instruction (second instruction) to compare the extracted source document with the specified document. The server 20 input the prompt including the first instruction and the second instruction into the large-scale language model and obtained a response. Based on the response from the large-scale language model, the server 20 also presented the user with information about the extracted source document and the result of the comparison between the specified document and the source document.

[0150] However, the server 20 may generate a first prompt containing a first instruction and input it into the large-scale language model. Based on the response data from the large-scale language model, it may then present the user with information about the source text. The server may then accept from the user the specification of a source text to be compared with a specified text. The server may then generate a second prompt containing a second instruction to compare the source text specified by the user with the specified text and input it into the large-scale language model. Based on the response data from the large-scale language model, the server may then present the user with information about the comparison result between the source text specified by the user and the specified text.

[0151] A specific example of the processing in this modified example is described below. Figure 12 is a flowchart showing an example of the processing in this modified example.

[0152] Steps S21 and S22 are the same as steps S11 and S12 of the text comparison process.

[0153] In step S23, the server 20 generates a prompt containing a first instruction. Specifically, for example, the server 20 generates the first prompt by embedding the prompt information stored in the prompt information table 2023 in step S22 into a prompt template containing instruction 2028a.

[0154] In step S24, the server 20 inputs the generated first prompt into the large-scale language model provided by the artificial intelligence system 40. Specifically, for example, the server 20 sends a request containing the text data of the generated first prompt and necessary parameters (e.g., specifying the number of tokens) to the API endpoint of the artificial intelligence system 40.

[0155] When the artificial intelligence system 40 receives a request from the server 20, it sends response information, including response data from a large-scale language model, to the server 20 via the API endpoint.

[0156] In step S25, the server 20 receives the response information sent from the artificial intelligence system 40.

[0157] In step S26, the server 20 extracts information about the source text from the response data of the large-scale language model included in the response information. Specifically, for example, it extracts the following information: • The title of the source text from which the source text was extracted (the title of the webpage or article in which the source text is published) • Source of the extracted text • Reliability of the information source • Source of the extracted text (e.g., web page accessed) • Publication time of the source document (date, time, etc.) • The main text of the source document

[0158] Server 20 generates an extraction result UI for presenting information about the source document to the user based on the information extracted from the response information. Specifically, for example, Server 20 generates the extraction result UI by inputting the information of each extracted item into the fields provided in the extraction result UI corresponding to each item.

[0159] In step S27, the server 20 presents the user with information about the extracted source text. Specifically, the server 20 presents the generated extraction result UI to the user via the terminal device 10. For example, the server 20 displays the extraction result screen on the display 141 of the terminal device 10.

[0160] Figure 13 is a schematic diagram showing an example of the extraction results screen D4. As shown in Figure 12, the extraction results screen D4 includes operation object D411, operation object D412, operation object D42, and display object D43.

[0161] Operation objects D411 and D412 accept operations to input requests to narrow down the information about the source document displayed by display object D43 based on specific conditions.

[0162] For example, when the operation object D411 receives input from the user, an object showing a list of items with predetermined confidence values ​​(90, 80, 70, ..., etc.) or a list of items indicating the degree of confidence (high, medium, low, ..., etc.) is displayed on the extraction results screen D4. For example, when this object receives a specification for an item with a predetermined value, the information displayed by the display object D43 is narrowed down to information extracted from information sources with a confidence level equal to or greater than the specified confidence value or degree. As an example, if an item with a confidence level of "90" is specified, the information about the source document displayed by the display object D43 will be narrowed down to information extracted from information sources with a confidence level of "90" or higher.

[0163] Furthermore, for example, when the operation object D412 receives input from the user, an object for specifying a date range is displayed on the extraction results screen D4. For example, when this object receives input for a predetermined date range, the information displayed by the display object D43 is narrowed down to information published within that predetermined date range.

[0164] The operation object D42 accepts input for a request to sort the source document information displayed by the display object D43 based on specific conditions. For example, when the operation object D42 accepts input from the user, a pull-down object for specifying specific sorting conditions, such as by confidence level or by date, is displayed on the extraction results screen D4. For example, when this object accepts the specified conditions, the display object D43 is sorted according to the accepted conditions. As an example, if the condition "by confidence level" is specified, the display object D43 is sorted in descending order of confidence level of the source information.

[0165] Display object D43 is an object that displays information about the source text. Alternatively, display object D43 can also be described as an object that displays information about each source text extracted as a candidate for comparison with the specified text. Specifically, for example, display object D43 displays the following information about the extracted source texts. Note that the extraction result screen D4 has an independent display object D43 for each extracted source text. • The title of the source text from which the source text was extracted (the title of the webpage or article in which the source text is published) • Source of the extracted text • Reliability of the information source • Publication time of the source document (date, time, etc.) • The main text of the source document

[0166] Furthermore, each display object D43 includes operation objects D431 and D432. Operation objects D431 and D432 accept operations related to the source document associated with the display object D43.

[0167] Operation object D431 accepts input for a request to view the source document. For example, operation object D431 has embedded identification information of the source from which the source document was extracted (e.g., a URL link). When operation object D431 accepts input from the user, terminal device 10 accesses the source via the embedded link and presents the user with information including the body of the source document.

[0168] Operation object D432 accepts an input request to compare a source document with a specified document. For example, when operation object D432 receives an input from a user, terminal device 10 sends information about the source document related to operation object D432, along with a request to compare the source document with the specified document, to server 20.

[0169] Furthermore, on the extraction results screen D4, the terminal device 10 may accept requests to compare multiple source documents with a specified document. For example, it may be possible to select multiple operation objects D432 related to each source document simultaneously.

[0170] In step S28, the server 20 receives information about the request and source document sent by the terminal device 10 by accepting an operation on the operation object D432.

[0171] In step S29, the server 20 generates a second prompt that includes a second instruction to compare the source document from which the information was received with the specified document. Specifically, for example, the server 20 generates a prompt that includes an instruction to compare the specified document received in step S22 with the source document received in step S28.

[0172] (Example of a second prompt) Referencing the information in #SourceText, compare the #SourceText with the #SpecifiedText and output the information according to the items in #InformationAboutComparisonResults. #Information about the source text {Information about the source text} #Source text {source text} #specified text {specified text} #Information on comparison results • Matching parts between the specified text and the source text • Differences between the specified text and the source text • The degree of matching between the specified text and the source text. • Basis for determining the degree of agreement

[0173] In the example of the second prompt above, {information about the source document} is replaced with the information about the source document received in step S28. Also, {source document} is replaced with the body of the source document received in step S28. Also, {specified document} is replaced with the specified document received in step S22. Furthermore, the second prompt may appropriately include documents included in instructions 2028b to 2028e. For example, the second prompt may include an instruction to output the credibility of the specified document received in step S22 based on the comparison result and the credibility of the source information received from the terminal device 10.

[0174] In step S210, the server 20 inputs the generated second prompt into the large-scale language model provided by the artificial intelligence system 40.

[0175] When the artificial intelligence system 40 receives a request from the server 20, it sends response information, including response data from a large-scale language model, to the server 20 via the API endpoint.

[0176] In step S211, the server 20 receives response information sent from the artificial intelligence system 40.

[0177] In step S212, the server 20 extracts information regarding the comparison result between the source text and the specified text from the response data of the large-scale language model included in the response information.

[0178] Server 20 generates a comparison result UI to present the user with information regarding the comparison results between the specified text and the source text. Specifically, for example, Server 20 generates a comparison result UI that corresponds to the response data by inputting the information regarding the comparison results extracted from the response data into fields provided in the comparison result UI that correspond to each item of the comparison result.

[0179] In step S213, the server 20 presents the user with information regarding the comparison result between the specified text received in step S22 and the source text whose information was received in step S28. Specifically, the server 20 presents the generated comparison result UI to the user via the terminal device 10. For example, the server 20 displays the comparison result screen on the display 141 of the terminal device 10.

[0180] Figure 14 is a schematic diagram showing an example of the comparison results screen D5. As shown in Figure 14, the comparison results screen D5 includes display object D51, display object D52, and display object D53.

[0181] Display object D51 displays information about the source document that the server 20 received in step S28. For example, display object D51 displays information similar to the information displayed by display object D21 on the first result screen D2 regarding the source document that the server 20 received in step S28.

[0182] Display object D52 displays information regarding the comparison result between the specified text received by the server 20 in step S22 and the source text for which information was received in step S28. Specifically, for example, display object D52 displays information similar to that displayed by display object D22 on the first result screen D2 regarding the comparison result between the specified text received by the server 20 in step S22 and the source text for which information was received in step S28.

[0183] Display object D53 displays information regarding the credibility of the specified document received by server 20 in step S22. Specifically, for example, display object D53 displays information similar to the information displayed by display object D31 on the second result screen D3 regarding the specified document received by server 20 in step S22.

[0184] The processing of this modified version is completed as described above. According to this modified version, in step S27, the server 20 presents the user with a list of source documents extracted as candidates for comparison with the specified document. Then, in step S28, the server 20 accepts the specification of a source document to be compared with the specified document. Then, in step S213, the server 20 presents the user with the comparison result between the specified source document and the specified document. This allows the user to select the source document to be compared with the specified document themselves, and thus the comparison between the specified document and the source documents can be performed according to the user's intentions. Therefore, the user can more appropriately examine the credibility of the specified document.

[0185] <Summary> As explained above, Server 20 accepts a text specification. Server 20 generates a prompt that includes an instruction to extract at least one source text containing a description similar to the specified text from the information sources available to the artificial intelligence system 40, and to output information about the source text, and an instruction to compare the extracted source text with the specified text. Server 20 inputs the prompt into a large-scale language model provided by the artificial intelligence system, and based on the response obtained from the large-scale language model, presents the user with information about the source text and the comparison result between the specified text and the source text. This makes it possible to compare information dissemination in an unspecified field with information extracted from other information sources.

[0186] Furthermore, the prompt includes instructions to extract source text from at least one predetermined information source. This allows the artificial intelligence system 40 to extract source text from a specific information source. For example, it can extract source text from an information source specified by the user.

[0187] Furthermore, each information source is assigned a confidence level indicating its reliability. The prompt includes an instruction to output a confidence level indicating the degree of credibility of the specified document, based on the comparison result and the confidence level of the source information in the source document. The server 20 then presents the confidence level to the user. This allows the user to understand the credibility of the specified document.

[0188] Furthermore, the instructions include instructions for determining the reliability of the source information in the source text based on at least one specific perspective. This allows a large-scale language model to determine the reliability of the source information. Consequently, the burden on users and operators of the services disclosed can be reduced in determining reliability. In addition, reliability can be determined objectively while excluding the subjectivity of users and operators of the services disclosed.

[0189] The prompt also includes instructions to output a degree of match, indicating the degree of similarity between the source text and the specified text. Server 20 presents the degree of match to the user, allowing the user to better understand the comparison results between the source text and the specified text.

[0190] The prompt also includes instructions on outputting the basis for determining the degree of match. Server 20 presents the basis to the user, allowing the user to better understand the comparison results between the source text and the specified text.

[0191] The prompt also includes instructions to output at least one of the information regarding the matching portions and the differences between the specified text and the source text. Server 20 presents the user with at least one of the information regarding the matching portions and the differences between the specified text and the source text. This allows the user to better understand the comparison results between the source text and the specified text.

[0192] Furthermore, when multiple source documents are output from the large-scale language model, the server 20 presents a predetermined number of source documents to the user based on predetermined criteria. This makes the information about the source documents displayed to the user more concise and improves the readability of the information about the source documents.

[0193] Furthermore, when multiple source documents are output from a large-scale language model, the presentation step presents the user with information about the source documents and comparison results for each of the multiple source documents in an order based on predetermined criteria. This organizes the information about the source documents and comparison results displayed to the user, improving the visibility of the information about the source documents and comparison results.

[0194] Furthermore, each information source is assigned a reliability rating indicating its degree of trustworthiness. The prescribed criteria are standards related to the reliability of the information source for each source document. This allows for filtering and sorting of information regarding the source document and comparison results according to the reliability rating of the information source.

[0195] The prompt also includes instructions for outputting the degree of similarity between the source text and the specified text. The predetermined criteria are the criteria for the degree of similarity between each source text and the specified text. This allows for filtering and sorting of information regarding the source text and comparison results according to the degree of similarity.

[0196] Furthermore, each information source is assigned a confidence level indicating its reliability. When multiple source documents are extracted, the prompt includes instructions to select at least one source document from the multiple source documents based on the confidence level of the information source from which the source documents were extracted, and to compare it with the specified document. This allows for comparison with only source documents that meet the confidence level of a given information source. For example, a user can narrow down the comparison to source documents extracted from highly credible information sources, making it easier to determine the credibility of the specified document. Additionally, since the number of source documents to be compared is reduced, the number of tokens consumed when using a large-scale language model can be decreased.

[0197] Furthermore, the specified text is text contained in posts on social networking services. This allows users to compare posts on social networking services with information extracted from other sources.

[0198] <4. Basic Hardware Configuration of a Computer> Figure 15 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 includes at least a processor 94, main memory 95, auxiliary storage 96, and a communication interface 99. These are electrically connected to each other by a bus.

[0199] The processor 94 is hardware for executing the instruction set written in a program. The processor 94 consists of an arithmetic unit, registers, peripheral circuits, etc.

[0200] Main memory 95 is used to temporarily store programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0201] Auxiliary storage device 96 refers to a storage device for saving data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, and semiconductor memory.

[0202] A communication IF99 is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards.

[0203] A network consists of various mobile communication systems built using the internet, LANs, wireless base stations, etc. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0204] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

[0205] <Basic Functional Configuration of Computer 90> The functional configuration of the computer realized by the basic hardware configuration of computer 90 shown in Figure 14 will be explained. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

[0206] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

[0207] The control unit is realized when the processor 94 reads various programs stored in the auxiliary storage device 96, loads them into the main memory device 95, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0208] The memory unit is implemented by a main memory 95 and an auxiliary memory 96. The memory unit stores data, various programs, and various databases. The processor 94 can also reserve memory areas corresponding to the memory unit in the main memory 95 or the auxiliary memory 96 according to the program. The control unit can also cause the processor 94 to perform operations such as adding, updating, and deleting data stored in the memory unit according to the various programs.

[0209] A database, specifically a relational database, is used to manage and link data sets called tables, which are structurally defined by rows and columns. In a database, tables are called tables, the columns of a table are called columns, and the rows of a table are called records. In a relational database, relationships can be established and linked between tables.

[0210] Typically, each table has a key column to uniquely identify records, but setting a key on a column is not mandatory. The control unit can instruct the processor 94 to add, delete, or update records in specific tables stored in the memory unit according to various programs.

[0211] The communication unit is implemented by the communication IF99. The communication unit provides the functionality to communicate with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 94 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.

[0212] The functions realized by the components described herein may be implemented in a circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor includes transistors and other circuits and is considered a circuitry or processing circuitry. A processor may be a programmed processor that executes a program stored in memory. In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein. If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0213] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0214] <Note> The details described in each of the above embodiments are noted below. (Note 1) A program for operating a computer, the program causing the computer's processor to perform the following steps: receiving a text specification; generating a prompt including an instruction for extracting at least one source text containing a description similar to the specified text from information sources available to the artificial intelligence system and outputting information about the source text; and an instruction for comparing the extracted source text with the specified text; and inputting the prompt into a large-scale language model provided by the artificial intelligence system and presenting the user with information about the source text and the result of the comparison between the specified text and the source text based on the response obtained from the large-scale language model. (Note 2) The prompt is the program described in Appendix 1, which includes instructions to extract source text from at least one predetermined source. (Note 3) The program, as described in Appendix 1 or Appendix 2, includes instructions for a prompt that output a credibility score indicating the degree of credibility of a specified document based on the comparison result and the credibility score of the source document, and presents the credibility score to the user in the presenting step. (Note 4) The instructions include the program described in Appendix 3, which includes instructions for determining the reliability of the sources of a source document based on at least one specific perspective. (Note 5) The program described in any of Appendix 1 to Appendix 4 includes instructions for outputting a degree of match indicating the degree of similarity between the source text and the specified text, and presents the degree of match to the user in the steps provided. (Note 6) The program described in Appendix 5 includes instructions for outputting the basis for determining the degree of agreement, and presents the basis to the user in the step of presenting. (Note 7) The program described in any of Appendix 1 to Appendix 6 includes instructions for outputting at least one of the matching portions and differences between a specified document and a source document, and in the steps of presenting, presents to the user at least one of the matching portions and differences between the specified document and the source document. (Note 8) A program described in any of Appendix 1 to Appendix 7, which, when multiple source documents are output from a large-scale language model, presents a predetermined number of source documents to the user based on predetermined criteria in the presentation step. (Note 9) A program described in any of Appendix 1 to Appendix 8, which, when multiple source documents are output from a large-scale language model, presents to the user, in the presentation step, information about the multiple source documents and comparison results for each source document in an order based on predetermined criteria. (Note 10) Each information source has a defined reliability rating indicating its degree of trustworthiness. The specified criteria are the program described in Appendix 8 or Appendix 9, which are the criteria for the reliability of the information sources in each source document. (Note 11) The program described in any of the appendices 8 to 10 includes instructions for outputting the degree of match between a source document and a specified document, where the predetermined criteria are criteria for the degree of match between each source document and the specified document. (Note 12) The information source has a defined confidence level indicating the degree of reliability of the information source, and the prompt includes instructions for selecting at least one source document from the multiple source documents based on the confidence level of the information source from which the source documents were extracted, and comparing it with the specified document, as described in any of the appendices 1 to 11. (Note 13) The specified text is a text included in a post on a social networking service, and is one of the programs described in Appendix 1 to Appendix 12. (Note 14) A method to be performed on a computer having a processor and memory, the method comprising: the step of the processor receiving a specification of a document; the step of generating a prompt including an instruction for an artificial intelligence system to extract at least one source document containing a description similar to the specified document from an available source of information and output information about the source document; and the step of presenting the user with information about the source document and the result of the comparison between the specified document and the source document, based on a response obtained from a large-scale language model provided by the artificial intelligence system by inputting the prompt into the large-scale language model. (Note 15) An information processing device comprising a control unit and a storage unit, wherein the control unit performs the following steps: receiving a text specification; generating a prompt including an instruction for extracting at least one source text containing a description similar to the specified text from an information source available to the artificial intelligence system and outputting information about the source text; and comparing the extracted source text with the specified text; and inputting the prompt into a large-scale language model provided by the artificial intelligence system and presenting the user with information about the source text and the result of comparing the specified text with the source text based on the response obtained from the large-scale language model. (Note 16) A system including at least one information processing device having a control unit and a storage unit, the system performing steps of: receiving a specification of a sentence; extracting at least one source sentence including a description similar to a specified sentence which is the specified sentence, from an information source available to an artificial intelligence system, and outputting an instruction regarding information on the source sentence; generating a prompt including an instruction to compare the extracted source sentence and the specified specified sentence; inputting the prompt into a large language model provided by the artificial intelligence system, and presenting, to a user, information regarding the source sentence and a comparison result between the specified sentence and the source sentence based on a response obtained from the large language model.

Explanation of Signs

[0215] 1…System 10…Terminal device 12…Communication IF 120…Communication unit 13…Input device 131…Button 14…Output device 141…Display 15…Memory 150…Position information sensor 16…Storage 160…Camera 17…Audio processing unit 171…Microphone 172…Speaker 180…Storage unit 19…Processor 190…Control unit 20…Server

Claims

1. A program for operating a computer, wherein the program is configured on the computer's processor. Steps to accept text specifications, A step of generating a prompt that includes an instruction for the artificial intelligence system to extract at least one source document containing a description similar to the specified document from an available information source and output information about the source document, and an instruction to compare the extracted source document with the specified document, The steps include: inputting the prompt into a large-scale language model provided by the artificial intelligence system, and then presenting the user with information about the source text and the results of comparing the specified text with the source text, based on the response obtained from the large-scale language model; A program that executes the command.

2. The program according to claim 1, wherein the prompt includes an instruction to extract the source text from at least one predetermined source.

3. The aforementioned information source is assigned a confidence level indicating the degree of reliability of the information source. The prompt includes an instruction to output a credibility score indicating the degree of credibility of the specified document, based on the comparison result and the credibility score of the information source of the source document. In the steps described above, the credibility level is presented to the user. The program according to claim 1.

4. The instructions include instructions for determining the reliability of the information source in the source document based on at least one specific viewpoint, The program according to claim 3.

5. The prompt includes instructions to output a degree of match indicating the degree of similarity between the source text and the specified text. In the steps described above, the degree of agreement is presented to the user. The program according to claim 1.

6. The prompt includes instructions for outputting the basis for determining the degree of agreement, In the steps described above, the basis is presented to the user. The program according to claim 5.

7. The prompt includes instructions to output at least one of the information relating to the matching portions and the differences between the specified text and the source text. In the steps described above, the user is presented with at least one of the information relating to the matching portion and the information relating to the differences between the specified document and the source document. The program according to claim 1.

8. The program according to claim 1, wherein, when multiple source documents are output from the large-scale language model, in the presenting step, a predetermined number of the multiple source documents are presented to the user based on predetermined criteria.

9. The program according to claim 1, wherein, when multiple source documents are output from the large-scale language model, in the presenting step, information about the multiple source documents and the comparison results are presented to the user in an order based on predetermined criteria.

10. The aforementioned information source is assigned a confidence level indicating the degree of reliability of the information source. The program according to claim 8 or 9, wherein the predetermined criteria are criteria relating to the reliability of the information source for each source document.

11. The prompt includes instructions for outputting the degree of match between the source document and the specified document, The program according to claim 8 or 9, wherein the predetermined criterion is a criterion relating to the degree of agreement between each source document and the specified document.

12. The aforementioned information source is assigned a confidence level indicating the degree of reliability of the information source. If multiple source documents are extracted, the prompt includes instructions to select at least one of the multiple source documents based on the confidence level of the information source from which the source documents are extracted, and to compare it with the specified document. The program according to claim 1.

13. The program according to claim 1, wherein the specified text is text included in a post on a social networking service.

14. A method to be performed on a computer comprising a processor and memory, wherein the processor Steps to accept text specifications, A step of generating a prompt that includes an instruction for the artificial intelligence system to extract at least one source document containing a description similar to the specified document from an available information source and output information about the source document, and an instruction to compare the extracted source document with the specified document, The steps include: inputting the prompt into a large-scale language model provided by the artificial intelligence system, and then presenting the user with information about the source text and the results of comparing the specified text with the source text, based on the response obtained from the large-scale language model; How to do it.

15. An information processing apparatus comprising a control unit and a storage unit, wherein the control unit is Steps to accept text specifications, A step of generating a prompt that includes an instruction for the artificial intelligence system to extract at least one source document containing a description similar to the specified document from an available information source and output information about the source document, and an instruction to compare the extracted source document with the specified document, The steps include: inputting the prompt into a large-scale language model provided by the artificial intelligence system, and then presenting the user with information about the source text and the results of comparing the specified text with the source text, based on the response obtained from the large-scale language model; An information processing device that performs the following actions.

16. A system comprising at least one information processing device having a control unit and a storage unit, Steps to accept text specifications, A step of generating a prompt that includes an instruction for the artificial intelligence system to extract at least one source document containing a description similar to the specified document from an available information source and output information about the source document, and an instruction to compare the extracted source document with the specified document, The steps include: inputting the prompt into a large-scale language model provided by the artificial intelligence system, and then presenting the user with information about the source text and the results of comparing the specified text with the source text, based on the response obtained from the large-scale language model; A system that executes this process.

Images