Information processing system and program

Abstract

Make users aware of the degree to which elements of the output data may contain errors caused by AI. [Solution] An information processing system comprising a processor that acquires input data, acquires output data generated by AI (Artificial Intelligence) in accordance with the input data, and outputs the output data and error probability information indicating the degree to which elements of the output data are likely to contain errors caused by the AI.

Description

[Technical Field]

[0001] The present invention relates to an information processing system and a program. [Background technology]

[0002] Patent Document 1 describes a knowledge determination device that detects knowledge errors in a sentence. This knowledge determination device extracts entity words that meet predetermined conditions from a sentence, extracts entity word pairs that are combinations of entity words, infers related words in the entity word pairs from the sentence, generates a dialogue graph connecting the entity word pairs with the inferred related words, searches for related words in the entity word pairs, generates a knowledge graph connecting the entity word pairs with the related words, generates feature values ​​for each entity word from both the dialogue graph and the knowledge graph, and inputs the feature values ​​of the entity word in the dialogue graph and the feature values ​​of the entity word in the knowledge graph for each entity word to classify the knowledge as correct or incorrect.

[0003] Patent Document 2 describes a processing device that processes bilingual data including an input sentence written in a first language and a translation of the input sentence into a second language. This processing device acquires first bilingual data that is a pair of a first sentence written in the first language and a first translated sentence that is a translation of the first sentence in the second language, evaluates whether the first bilingual data is bilingual data that may be mistranslated based on words and phrases included in the first sentence and the first translated sentence, and outputs information based on the evaluation result.

[0004] Patent document 3 describes an information processing device that acquires a target sentence, divides the target sentence into unit strings, calculates the occurrence probability of each unit string of the target sentence divided by a division unit using a sentence model that has learned the order of unit strings in multiple sentences, determines the number of times each unit string of the target sentence has been learned in the sentence model, and determines whether the number of times the determined unit string has been learned exceeds a threshold. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-60429 [Patent Document 2] Japanese Patent Application Publication No. 2019-3552 [Patent Document 3] Japanese Patent Publication No. 2020-52818 Summary of the Invention [Problem to be solved by the invention]

[0006] When output data is generated by AI (Artificial Intelligence) based on input data, elements of the output data may contain errors made by the AI. In such cases, a configuration is adopted that outputs whether or not the elements of the output data contain errors made by the AI. However, this does not allow the user to recognize the degree of possibility that the elements of the output data contain errors made by the AI.

[0007] The purpose of the present invention is to make users aware of the degree to which elements of output data may contain errors caused by AI. [Means for solving the problem]

[0008] The invention described in claim 1 is an information processing system comprising a processor that acquires input data, acquires output data generated by AI (Artificial Intelligence) in accordance with the input data, and outputs the output data and error probability information that indicates the degree of possibility that elements of the output data include errors made by the AI. The invention described in claim 2 is the information processing system described in claim 1, in which the processor outputs the output data and the error possibility information with the error possibility information added to the element. The invention described in claim 3 is an information processing system described in claim 2, in which the output data is text data, and the processor adds the error probability information to the element by setting the attribute of a character corresponding to the element of the text data to an attribute according to the error probability information. A fourth aspect of the present invention is the information processing system according to the first aspect, wherein the elements are all elements of the output data. A fifth aspect of the present invention is the information processing system according to the first aspect, wherein the element is an element of a predetermined type from among a plurality of elements of the output data. The invention described in claim 6 is an information processing system described in claim 5, wherein the output data is text data and the predetermined type of element is at least one of an element of a predetermined part of speech and an element consisting of numbers. A seventh aspect of the present invention is the information processing system according to the first aspect, wherein the element is an element other than an element of a predetermined type among the plurality of elements of the output data. An eighth aspect of the present invention is the information processing system according to the seventh aspect, wherein the output data is text data, and the predetermined type of element is an element of a predetermined part of speech. The invention described in claim 9 is an information processing system described in claim 7, wherein the output data is text data and the predetermined type of element is at least one of an antonym element and a translation element. An invention described in claim 10 is the information processing system described in claim 1, wherein the processor acquires, as the output data, data generated by the AI ​​based on a large-scale language model, and outputs, as the error possibility information, information generated based on identifiability information indicating a degree of possibility that a knowledge neuron corresponding to the element can be identified in the large-scale language model. An invention described in claim 11 is the information processing system described in claim 10, wherein the processor generates the error possibility information indicating that there is a high possibility that the element contains an error made by the AI ​​when the identifiability information indicates that there is a low possibility that a knowledge neuron corresponding to the element can be identified in the large-scale language model. The invention described in claim 12 is an information processing system described in claim 10, in which the processor acquires the identifiability information based on correspondence information that associates each element of the multiple elements of the output data with the probability that neurons in multiple layers of each element will contribute to determining the next element of the multiple elements. The invention described in claim 13 is an information processing system described in claim 12, in which the processor obtains, as the identifiability information, information based on the probability of any layer associated with any element other than the last element among the multiple elements in the correspondence information. The invention described in claim 14 is an information processing system described in claim 12, in which the processor obtains, as the identifiability information, information based on the probability of one of the layers associated with the last element of the multiple elements in the correspondence information. The invention described in claim 15 is an information processing system described in claim 12, in which the processor obtains, as the identifiability information, information based on the maximum probability among the probabilities of any layer associated with any of the multiple elements in the correspondence information. The invention described in claim 16 is an information processing system described in claim 12, wherein the processor obtains, as the identifiability information, information based on the probability of any layer associated with any element other than the last element of the multiple elements in the correspondence information, the probability of any layer associated with the last element of the multiple elements in the correspondence information, and the maximum probability of any layer associated with any of the multiple elements in the correspondence information. The invention described in claim 17 is a program for causing a computer to realize the following functions: acquiring input data; acquiring output data generated by AI (Artificial Intelligence) according to the input data; and outputting the output data and error probability information indicating the degree of possibility that elements of the output data may contain errors made by the AI. [Effects of the Invention]

[0009] According to the invention of claim 1, the user can be made aware of the degree to which elements of the output data are likely to contain errors caused by AI. According to the invention of claim 2, it becomes easy for the user to recognize the correspondence between the elements of the output data and the degree of possibility that the data may contain errors caused by the AI. According to the invention of claim 3, the user can recognize at a glance the correspondence between the elements of the output data and the degree of possibility that the data may contain errors caused by the AI. According to the invention of claim 4, it is possible to make the user aware of the degree of possibility that each element of the output data contains an error caused by AI. According to the invention of claim 5, it is possible to make the user aware of the degree of possibility that each element of a predetermined type of output data contains an error caused by AI. According to the invention of claim 6, the user can be made aware of the degree of possibility that each element of the output data, which is at least one of a predetermined part of speech and an element consisting of numbers, may contain an error made by AI. According to the invention of claim 7, for elements other than the predetermined types of elements in the output data, the user can be made aware of the degree of possibility that each element may contain an error made by AI. According to the invention of claim 8, for elements of the output data other than elements of predetermined parts of speech, the user can be made aware of the degree of possibility that each element may contain an error made by the AI. According to the invention of claim 9, for elements other than at least one of the antonym elements and translation elements of the output data, the user can be made aware of the degree of possibility that each element may contain an error made by AI. According to the invention of claim 10, it is possible to make the user aware of the degree of possibility that elements of the output data may contain errors caused by AI, without requiring an external database. According to the invention of claim 11, when it is unlikely that a knowledge neuron corresponding to an element can be identified in a large-scale language model, the user can be made aware that there is a high possibility that the element of the output data contains an error made by AI. According to the invention of claim 12, it becomes easy to recognize the degree of possibility of being able to identify knowledge neurons corresponding to elements in a large-scale language model. According to the invention of claim 13, by focusing on the probability of the layer associated with an element other than the last element of the output data, it is possible to recognize the degree of possibility of identifying a knowledge neuron corresponding to an element in a large-scale language model. According to the invention of claim 14, by focusing on the probability of the layer associated with the last element of the output data, it is possible to recognize the degree of possibility of identifying a knowledge neuron corresponding to an element in a large-scale language model. According to the invention of claim 15, by focusing on the maximum probability among the probabilities of the layers associated with the elements of the output data, it is possible to recognize the degree of possibility of identifying a knowledge neuron corresponding to the element in a large-scale language model. According to the invention of claim 16, it is possible to recognize the degree of possibility of identifying a knowledge neuron corresponding to an element in a large-scale language model by focusing on the probability of the layer associated with an element other than the last element of the output data, the probability of the layer associated with the last element of the output data, and the maximum probability among the probabilities of the layers associated with the elements of the output data. According to the invention of claim 17, the user can be made aware of the degree of possibility that elements of the output data may contain errors caused by AI. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of a sentence generation system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal according to the present embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to the present embodiment. [Figure 4] FIG. 10 is a diagram illustrating an example of a screen displayed on a user terminal. [Figure 5] FIG. 2 is a schematic diagram illustrating internal processing of a server. [Figure 6]FIG. 10 is a schematic diagram showing word prediction when no hallucination occurs. [Figure 7] FIG. 10 is a schematic diagram showing word prediction when hallucination occurs. [Figure 8] 1 is a block diagram showing an example of a functional configuration of a sentence generation system according to an embodiment of the present invention; [Figure 9] FIG. 10 is a schematic diagram showing a process in which a knowledge neuron analysis unit predicts a word using a large-scale language model in a normal state. [Figure 10] FIG. 10 is a schematic diagram showing a process in which a knowledge neuron analysis unit predicts a word using a large-scale language model in which noise is given to all neurons. [Figure 11A] FIG. 10 is a schematic diagram showing a process in which a knowledge neuron analysis unit predicts a word using a large-scale language model in which noise is given to one neuron. [Figure 11B] FIG. 10 is a schematic diagram showing a process in which a knowledge neuron analysis unit predicts a word using a large-scale language model in which noise is given to one neuron. [Figure 11C] FIG. 10 is a schematic diagram showing a process in which a knowledge neuron analysis unit predicts a word using a large-scale language model in which noise is given to one neuron. [Figure 12] FIG. 10 is a diagram showing a heat map recorded by the knowledge neuron analysis unit when hallucination does not occur. [Figure 13] FIG. 10 is a diagram showing a heat map recorded by the knowledge neuron analysis unit when hallucination occurs. [Figure 14] FIG. 10 is a diagram illustrating an index calculated by an index calculation unit. [Figure 15] 10 is a flowchart showing an example of the operation of a server constituting the sentence generation system in the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings.

[0012] (Outline of this embodiment) This embodiment provides an information processing system that acquires input data, acquires output data generated by AI in accordance with the input data, and outputs the output data and error probability information that indicates the degree of possibility that elements of the output data contain errors made by the AI.

[0013] Here, "data" may be either text data or image data, but the following explanation will be given taking text data as an example. In particular, the explanation will be given taking a sentence as an example of text data. A sentence is a unit consisting of one or more sentences. In other words, sentences are also included in the term "sentence."

[0014] Furthermore, an "element" may be any data that constitutes a part of "data." In the following, a sentence will be used as an example of "data," and an "element" will be explained using a word as an example. Furthermore, "AI-induced errors" refers to AI outputting incorrect information. For example, suppose the input data is "Please tell me about F Industries." Here, let's assume that F Industries was actually founded in 1934. Despite this, the AI ​​may output the following data: "F Industries is a Japanese manufacturer. It was founded in 1939..." This type of case constitutes an "AI-induced error." Hereinafter, "AI-induced errors" will be referred to as "hallucination."

[0015] Furthermore, a "system" may be composed of a single device or multiple devices. In the following, an information processing system composed of a single device will be used as an example. The single device will be explained as a server in a text generation system.

[0016] (Overall structure of the text generation system) FIG. 1 is a diagram showing an example of the overall configuration of a text generation system 1 according to this embodiment. As shown in the figure, the text generation system 1 includes a user terminal 10 and a server 40. The user terminal 10 can be wirelessly connected to a communication line 80 via an access point 70 using wireless communication such as Wi-Fi (registered trademark). The server 40 is also connected to the communication line 80. Although only one user terminal 10 and one server 40 are shown in the figure, there may be multiple users and multiple servers. The communication line 80 may be the Internet, for example.

[0017] The user terminal 10 is a terminal device used by a user to input a sentence and receive a sentence generated by an AI in response to that sentence. An application that displays the sentence input by the user and the sentence generated by the AI ​​is installed in the user terminal 10. The user terminal 10 transmits the sentence input by the user to the server 40 and receives the sentence generated by the AI ​​from the server 40. The user terminal 10 may be realized by, for example, a smartphone.

[0018] When the server 40 receives a sentence from the user terminal 10, it uses AI to generate a sentence that serves as a response to the received sentence. Then, the server 40 transmits the generated sentence to the user terminal 10. The server 40 may be realized by, for example, a personal computer.

[0019] (Hardware configuration of user terminal) 2 is a diagram showing an example of the hardware configuration of a user terminal 10 in this embodiment. As shown in the figure, the user terminal 10 includes a processor 11. The user terminal 10 further includes a RAM 12 and a ROM 13. The user terminal 10 further includes a touch panel 14, a voice input mechanism 15, and a voice output mechanism 16. The user terminal 10 further includes a short-range wireless communication interface (I / F) 17, a wireless circuit 18, and an antenna 19.

[0020] The processor 11 executes various software such as an OS (Operating System) and applications. The RAM 12 and the ROM 13 are storage areas for storing various software programs and data used for executing the software programs. The touch panel 14 displays various information and receives operation inputs from the user. The voice input mechanism 15 is a mechanism for inputting voice, and is, for example, a microphone. The audio output mechanism 16 is a mechanism for outputting audio, and is, for example, a speaker. The short-distance wireless communication I / F 17 is an interface for transmitting and receiving various information to and from other devices via short-distance wireless communication, such as NFC (Near Field Communication). The radio circuit 18 and antenna 19 are used to perform wireless communication via a base station. Here, the radio circuit 18 includes a baseband LSI (not shown). The baseband LSI performs signal processing of digital data transmitted and received wirelessly.

[0021] (Server hardware configuration) 3 is a diagram showing an example of the hardware configuration of server 40 in this embodiment. As shown in the figure, server 40 includes a processor 41. Server 40 further includes a main memory 42 and an HDD (Hard Disk Drive) 43. Server 40 further includes a communication interface (I / F) 44, a display device 45, and an input device 46.

[0022] The processor 41 executes various software such as an OS (Operating System) and applications, and realizes the functions described below. The main memory 42 is a storage area for storing various software programs and data used for executing the software programs. The HDD 43 is a storage area for storing input data for various software programs and output data from various software programs. The communication interface (I / F) 44 is an interface for communicating with the outside. The display device 45 is a device for displaying information, and is, for example, a display. The input device 46 is a device for inputting information, and is, for example, a keyboard or a mouse.

[0023] (Example of display on a user device) 4 is a diagram showing an example of a screen 300 displayed on the user terminal 10. As shown in the figure, the screen 300 includes a user text display area 310 and a bot text display area 320. The user text display area 310 is an area where text entered by the user is displayed. The bot text display area 320 is an area where text presented to the user by the bot is displayed. Such text includes text generated by the server 40 (see FIG. 1).

[0024] First, it is assumed that the user inputs a sentence 311. The sentence 311 is "Please tell me the company history of F Industry." In this case, the sentence 311 input by the user is displayed in the user sentence display area 310, as shown on the screen 300a.

[0025] Then, the server 40 uses the large-scale language model (LLM) 600 to start generating a sentence that will be an answer to the sentence 311. At that time, as shown on the screen 300a, a sentence 321 saying "Answer generating..." is displayed in the bot sentence display area 320.

[0026] Thereafter, suppose that the server 40 generates sentence 322, which is a response to sentence 311. Sentence 322 is, "F Industries is a Japanese manufacturer. It was founded in 1939, and photosensitive materials have been very popular since then..." Meanwhile, while generating sentence 322, the server 40 analyzes the inside of the large-scale language model 600.

[0027] As a result, the server 40 calculates the hallucination probability of each word in the sentence 322. Here, the hallucination probability refers to the probability that the word is hallucination. At this time, as shown on the screen 300b, the sentence 322 generated by the server 40 is displayed in the bot sentence display area 320. Then, in the sentence 322, attributes are set for each word according to the hallucination probability of that word. Here, the attributes include the color, darkness, font, size, and thickness of the characters, the color and darkness of the background of the characters, whether or not the characters are underlined, etc.

[0028] In the figure, the first attribute is set to the word "1939." The first attribute may be, for example, that the background color of the text is red and the text is bold. In the figure, the first attribute is represented by a bold solid frame 323. This indicates that the hallucination probability of this word is at the highest level. In the figure, a second attribute is set for the words "year" and "popular." The second attribute may be, for example, an attribute that the background color of the text is orange. In the figure, the second attribute is represented by a thin solid-line frame 324. This indicates that the hallucination probability of these words is the second lowest level. Furthermore, in the figure, no attribute is set for the other words, which indicates that the hallucination probability of the other words is at the lowest level. In the following, the hallucination probability will be explained as being divided into three levels, but it may also be divided into four or more levels.

[0029] Setting an attribute for each word according to the hallucination probability of that word is an example of setting an attribute of a character corresponding to an element of text data to an attribute according to error probability information.

[0030] In the above, an attribute corresponding to the hallucination probability is set for two or more words that make up the output sentence, and it is acceptable for some words in the output sentence not to have an attribute corresponding to the hallucination probability set.

[0031] For example, the attribute corresponding to the hallucination probability may be set to a predetermined type of word among the multiple words constituting the output sentence. In this case, the predetermined type of word may be at least one of a word of a predetermined part of speech and a word consisting of numbers. Here, the predetermined part of speech may be a noun, a verb, an adjective, an adjectival verb, etc. Alternatively, the attribute according to the hallucination probability may be set to words other than words of a predetermined type among the multiple words constituting the output sentence. In this case, the predetermined type of words may be words of a predetermined part of speech. Here, the predetermined part of speech may be a conjunction, particle, auxiliary verb, etc. Furthermore, the predetermined type of words may be at least one of antonym words and translation words.

[0032] On the other hand, the attribute according to the hallucination probability may be set for all words that make up the output sentence.

[0033] Next, the internal processing of the server 40 for providing such a display on the user terminal 10 will be described. FIG. 5 is a schematic diagram showing such internal processing. First, it is assumed that the user inputs a sentence 331. The sentence 331 is "What is F Industry?"

[0034] Then, the server 40 predicts that the next word in the sentence 331 is "F." Also, it is assumed that the prediction probability of the next word "F" in the sentence 331 is 40%. Next, the server 40 predicts that the word that follows the sentence 331 and the word "F" is "industry." Also, assume that the prediction probability of the word "industry" following the sentence 331 and the word "F" is 45%. Next, the server 40 predicts that the word that comes next to the sentence 331 and the word "F" and "industry" is "wa". Also, the prediction probability of the word "wa" that comes next to the sentence 331 and the word "F" and "industry" is assumed to be 20%.

[0035] Next, the server 40 predicts that the word that follows the sentence 331 and the words "F", "industry", and "wa" is "Japan". Also, the prediction probability of the word "Japan" following the sentence 331 and the words "F", "industry", and "wa" is assumed to be 15%. Next, the server 40 predicts that the word that follows the sentence 331 and the words "F", "industry", "wa", and "Japan" is "no". Also, the prediction probability of the word "no" following the sentence 331 and the words "F", "industry", "wa", and "Japan" is assumed to be 30%. Next, the server 40 predicts that the word that follows the sentence 331 and the words "F," "industry," "is," "Japan," and "of" is "manufacturer." Also, the prediction probability of the word "manufacturer" following the sentence 331 and the words "F," "industry," "is," "Japan," and "of" is 30%.

[0036] The server 40 repeats this process and outputs sentence 332, which is the sentence obtained by removing sentence 331 from the final sentence. Sentence 332 is "F Industries is a Japanese manufacturer."

[0037] Here, we focus on the sixth internal process among the internal processes shown in Figure 5. That is, we focus on the process that predicts that the next word after "F Industry is Japanese" is "manufacturer." To make such a prediction, the large-scale language model 600 must have the knowledge that "F Industry" is a "manufacturer." Knowledge neuron analysis explores which neurons in the large-scale language model 600 are associated with such knowledge. Knowledge neuron analysis is described in the paper "Locating and Editing Factual Associations in GPT (Meng et al., 2022)." In this embodiment, this knowledge neuron analysis is applied to the detection of hallucination.

[0038] 6 is a schematic diagram showing word prediction when hallucination does not occur. In the figure, the server 40 predicts that the word that comes next after "F Industry is Japanese" is "manufacturer." In this case, the knowledge neuron associated with the knowledge that "F Industry" is a "manufacturer" has been identified as neuron 611.

[0039] 7 is a schematic diagram showing word prediction when hallucination occurs. In the figure, the server 40 predicts that the word that comes next after "The year F Industry was founded is" is "1939." In this case, the knowledge neuron associated with the knowledge that "The year F Industry was founded" is "1939" has not been identified.

[0040] Therefore, in this embodiment, analysis is performed for each word, and if the position of the knowledge neuron cannot be identified, it is determined to be hallucination.

[0041] Suppose the prediction probability of the next word "manufacturer" after "F Industries is a Japanese company" is 30%. In contrast, the prediction probability of the next word "1939" after "When was F Industries founded?" may be 85%. Therefore, the prediction probability of the next word is unrelated to whether or not hallucination has occurred. In other words, hallucination cannot be determined solely from the prediction probability of the next word.

[0042] (Functional configuration of text generation system) FIG. 8 is a block diagram showing an example of the functional configuration of the text generation system 1 according to this embodiment. First, a description will be given of an example of the functional configuration of the user terminal 10. As shown in the figure, the user terminal 10 includes an operation accepting unit 21, a transmitting unit 22, a receiving unit 23, and a display control unit 24.

[0043] The operation acceptance unit 21 accepts an operation in which a user inputs a sentence on, for example, the touch panel 14 (see FIG. 2). Hereinafter, the sentence input by the user will be referred to as an “input sentence.” In other words, the operation acceptance unit 21 accepts the input sentence from, for example, the touch panel 14.

[0044] The transmitting unit 22 transmits the input text received by the operation receiving unit 21 to the server 40 using the wireless circuit 18 (see FIG. 2).

[0045] The receiving unit 23 receives the text to be output by the user terminal 10 from the server 40 using the wireless circuit 18. Hereinafter, the text to be output by the user terminal 10 will be referred to as "output text."

[0046] The receiving unit 23 also receives hallucination information for words in the output sentence from the server 40 using the wireless circuit 18. Hallucination information is information indicating the degree of possibility that the word is a hallucination. The receiving unit 23 may receive the hallucination information in a state separate from the output sentence. An example of such hallucination information is annotation information that associates words with the possibility that they are hallucinations. Alternatively, the receiving unit 23 may receive the hallucination information in a state attached to the words in the output sentence. An example of such hallucination information is the attributes set for words shown in FIG. 4.

[0047] The display control unit 24 receives the input sentence received by the operation receiving unit 21. Then, the display control unit 24 performs control so that the input sentence is displayed on the touch panel 14, for example.

[0048] The display control unit 24 also receives the output sentence and hallucination information received by the receiving unit 23 from the server 40. The display control unit 24 then controls the output sentence and hallucination information to be displayed, for example, on the touch panel 14. Here, suppose the receiving unit 23 receives the hallucination information separated from the output sentence. Then, the display control unit 24 controls the hallucination information to be displayed separately from the output sentence. An example of such hallucination information is annotation information that associates words with the possibility of hallucination. Alternatively, suppose the receiving unit 23 receives the hallucination information attached to the words of the output sentence. Then, the display control unit 24 controls the hallucination information to be displayed attached to the words of the output sentence. An example of such hallucination information is the attribute set to the words shown in FIG. 4.

[0049] Next, we will explain an example of the functional configuration of the server 40. As shown in the figure, the server 40 includes a receiving unit 51, a sentence acquisition unit 52, and a knowledge neuron analysis unit 53. The server 40 also includes an index calculation unit 54, a hallucination determination unit 55, and a transmission unit 56.

[0050] The receiving unit 51 receives an input sentence from the user terminal 10. In this embodiment, the input sentence is used as an example of input data. In addition, in this embodiment, the processing of the receiving unit 51 is performed as an example of obtaining input data.

[0051] The sentence acquisition unit 52 acquires an output sentence that is a response to the input sentence received by the receiving unit 51. Specifically, the sentence acquisition unit 52 acquires an output sentence that an AI generates based on the input sentence using a large-scale language model. In this embodiment, the output sentence is used as an example of output data generated by an AI in response to input data or data generated by an AI based on a large-scale language model. Also, in this embodiment, the processing of the sentence acquisition unit 52 is performed as an example of acquiring output data.

[0052] The knowledge neuron analysis unit 53 performs knowledge neuron analysis when the AI ​​generates an output sentence using a large-scale language model. The knowledge neuron analysis is an analysis of the degree to which each neuron in the large-scale language model contributes to predicting the next word in the output sentence. The knowledge neuron analysis unit 53 then records the results of the knowledge neuron analysis as a heat map. The heat map is information in which each word in the output sentence is associated with a prediction probability indicating the degree to which neurons in each layer contribute to predicting the word. In this embodiment, the heat map is used as an example of correspondence information in which each element of multiple elements in the output data is associated with the probability that neurons in multiple layers of each element will contribute to determining the next element of the multiple elements.

[0053] The index calculation unit 54 calculates an identifiability index indicating the degree of possibility that a knowledge neuron for a word can be identified in the large-scale language model. Specifically, the index calculation unit 54 calculates the identifiability index based on the heat map recorded by the knowledge neuron analysis unit 53. In this embodiment, the identifiability index is used as an example of identifiability information indicating the degree of possibility that a knowledge neuron corresponding to an element can be identified in the large-scale language model. Also, in this embodiment, the processing of the index calculation unit 54 is performed as an example of obtaining identifiability information based on correspondence information.

[0054] The hallucination determination unit 55 receives the identifiability index calculated by the index calculation unit 54. Then, the hallucination determination unit 55 determines whether the word is a hallucination based on the identifiability index. Based on this determination result, the hallucination determination unit 55 generates hallucination information for the word in the output sentence. In this embodiment, hallucination information is used as an example of error probability information generated based on identifiability information.

[0055] For example, suppose the identifiability index indicates that it is unlikely that a knowledge neuron for the word can be identified. Then, the hallucination determination unit 55 determines that the word is likely to be hallucination. The hallucination determination unit 55 generates hallucination information indicating that the word is likely to be hallucination. In this embodiment, this processing by the hallucination determination unit 55 is performed as an example of generating error possibility information indicating that the element is likely to contain an error made by AI when the identifiability information indicates that it is unlikely that a knowledge neuron corresponding to the element can be identified in the large-scale language model.

[0056] On the other hand, if the identifiability index indicates that there is a high possibility that a knowledge neuron for the word can be identified, the hallucination determination unit 55 determines that the word is unlikely to be hallucination. The hallucination determination unit 55 generates hallucination information indicating that the word is unlikely to be hallucination.

[0057] The transmitting unit 56 receives the output sentence acquired by the sentence acquiring unit 52 and the hallucination information generated by the hallucination determining unit 55. Then, the transmitting unit 56 transmits the output sentence and the hallucination information to the user terminal 10. In this embodiment, this processing by the transmitting unit 56 is performed as an example of outputting output data and error probability information.

[0058] At this time, the transmission unit 56 may transmit the hallucination information to the user terminal 10 in a state where the hallucination information is separated from the output sentence. Alternatively, the transmission unit 56 may transmit the hallucination information attached to the words of the output sentence to the user terminal 10. In this embodiment, this processing by the transmission unit 56 is performed as an example of outputting the output data and the error probability information with the error probability information attached to the elements.

[0059] Here, the processing contents of the knowledge neuron analysis unit 53 will be explained in detail. 9 to 11C are schematic diagrams showing the process of performing the calculation of the knowledge neuron analysis by the knowledge neuron analysis unit 53. Here, the process of predicting that the word that comes next after "F Industry is Japanese" is "manufacturer" is shown.

[0060] 9 shows the process of predicting a word using the normal large-scale language model 600. In this case, the prediction probability of the next word "manufacturer" after "F Industry is Japanese" is 30%.

[0061] 10 shows the process of predicting a word using a large-scale language model 600 in which noise is given to all neurons. In this case, the prediction probability of the next word "manufacturer" after "F Industry is a Japanese company" is 20%.

[0062] 11A to 11C show a process for predicting a word using a large-scale language model 600 in which noise is added to one neuron. In this process, one of the neurons to which noise is added in FIG. 10 is replaced with a noiseless neuron. In Figure 11A, neuron 601 is replaced with a noiseless neuron. In this case, the prediction probability of the next word "manufacturer" after "F Industry is Japanese" is 22%. In Figure 11B, neuron 602 is replaced with a noise-free neuron. In this case, the prediction probability of the next word "manufacturer" after "F Industry is Japanese" is 21%. In Figure 11C, neuron 611 is replaced with a noise-free neuron. In this case, the prediction probability of the next word "manufacturer" after "F Industry is Japanese" is 28%. Here, the prediction probability of the word "maker" in Figure 11C is extremely high. Therefore, neuron 611 is considered to be a knowledge neuron.

[0063] 12 and 13 are schematic diagrams showing the process of recording the result of the knowledge neuron analysis by the knowledge neuron analysis unit 53. Here, the process of recording the result of the knowledge neuron analysis as a heat map 650 is shown. In this heat map 650, the vertical axis shows words that make up a sentence. These words may be tokens obtained by dividing a sentence using a tokenizer. In this heat map 650, the horizontal axis indicates the layers in the neural network of the large-scale language model 600. Note that although this may vary depending on the type of document generation model used, here we assume that there are layers 0 to 46. The intersection of the word on the vertical axis and the layer on the horizontal axis shows the predicted probability of the next word in the sentence. This predicted probability is the predicted probability of the next word when the neurons in the layer on the horizontal axis of the vertical word are replaced with noise-free neurons. The predicted probability is also represented by the color intensity of the vertical rectangular area corresponding to the layer on the horizontal axis of the vertical word. The relationship between the predicted probability and the color intensity of the vertical rectangular area is shown in legend 655.

[0064] Fig. 12 shows a heat map 650 when hallucination is not occurring. In Fig. 12, the vertical axis shows words 661 to 665 that make up "F Industry is Japanese." In addition, the predicted probability p(manufacturer) of the word "manufacturer" is shown at the intersection of the word on the vertical axis and the layer on the horizontal axis. In this heat map 650, the color density of the vertically elongated rectangular area indicates that knowledge neurons exist around stars 668 and 669.

[0065] FIG. 13 shows a heat map 650 in the case where hallucination is occurring. In FIG. 13, the vertical axis shows words 671 to 676 that make up "What year was F Industry founded?". Also, at the intersection of the word on the vertical axis and the layer on the horizontal axis, the predicted probability p(1939) of the word "1939" is shown. In this heat map 650, the position of the knowledge neuron cannot be identified due to the color density of the vertically elongated rectangular area.

[0066] Next, the processing content of the index calculation unit 54 will be described in detail. FIG. 14 is a diagram showing the indexes calculated by the index calculation unit 54. As shown in FIG. The heat map 650 shown in FIG. 12 when no hallucination is occurring has three characteristics.

[0067] The first feature is that there is only one area with a high prediction probability in the upper left of the heat map 650. This first feature may be, for example, that there is only one area with a high prediction probability in layers 0 to 20 other than the last line. Although this depends on the type of document generation model used, here, layers 0 to 20 are taken as the left-hand layer as an example. In the following, the index representing the first feature is "A". Therefore, in the figure, a rectangle 681 marked with "A" is shown at a position related to the first feature.

[0068] The second feature is that the prediction probability is high on the right side of the last line of the heat map 650. This second feature may be, for example, that the prediction probability is high on the 35th to 46th layers of the last line. Although it depends on the type of document generation model used, here, the 35th to 46th layers are taken as the right-hand layer. In the following, the index representing the second feature is referred to as "B". Therefore, in the figure, a rectangle 682 marked with "B" is shown at a position related to the second feature.

[0069] The third feature is that there is a large difference in prediction probability between a region with a high prediction probability and a region with a low prediction probability. Hereinafter, the index representing the third feature will be referred to as "C." Therefore, in the figure, a rectangle 683 labeled "C" is shown in association with the legend 655.

[0070] As a result, the higher the values ​​of the indices A, B, and C, the lower the probability of hallucination. Conversely, the lower the values ​​of the indices A, B, and C, the higher the probability of hallucination.

[0071] First, the index calculation unit 54 calculates the index A. The index calculation unit 54 determines the highest average of prediction probabilities for five consecutive layers from the 0th to 20th layers excluding the last row as index A1. In the figure, a rectangle 684 labeled "A1" is shown at a position related to this index. The index calculation unit 54 determines the second highest average of prediction probabilities for five consecutive layers from 0 to 20, excluding the last row, as index A2. In the figure, a rectangle 685 labeled "A2" is shown at a position related to this index. The index calculation unit 54 sets the maximum value of the predicted probability in all layers of all rows as MAX. The index calculation unit 54 calculates the index A by "A=(A1-A2) / MAX." The final division by MAX is for standardization, i.e., to make the upper limit of the index A equal to 1.

[0072] Here, the index A is a value calculated from the index A1 and the index A2 by "A=(A1-A2) / MAX." However, the index A is not limited to this value. The index A may be any value based on the predicted probability of any layer corresponding to any row other than the last row of the heat map 650. In this sense, the index A is an example of information based on the probability of any layer associated with any element other than the last element among the multiple elements in the correspondence information.

[0073] Next, the index calculation unit 54 calculates index B. The index calculation unit 54 determines the average of the predicted probabilities of the 35th to 46th layers in the last row as index B'. The index calculation unit 54 determines the average of the predicted probabilities in all layers of all rows as the MEAN. The index calculation unit 54 sets the maximum value of the predicted probability in all layers of all rows as MAX. The index calculation unit 54 calculates the index B by "B=(B'-MEAN) / MAX." The final division by MAX is for standardization, i.e., to make the upper limit of index B equal to 1.

[0074] Here, the index B is a value calculated from the index B' by "B = (B' - MEAN) / MAX". However, the index B is not limited to this value. The index B may be any value based on the predicted probability of any layer corresponding to the last row of the heat map 650. In this sense, the index B is an example of information based on the probability of any layer associated with the last element of the multiple elements in the correspondence information.

[0075] Next, the index calculation unit 54 calculates the index C. The index calculation unit 54 determines the average of the predicted probabilities in all layers of all rows as the MEAN. The index calculation unit 54 sets the maximum value of the predicted probability in all layers of all rows as MAX. The index calculation unit 54 calculates the index C by "C=(MAX-MEAN) / MAX." The final division by MAX is for standardization, i.e., to make the upper limit of the index C equal to 1.

[0076] Here, the index C is a value calculated by "C = (MAX-MEAN) / MAX." However, the index C is not limited to this value. The index C may be any value based on the maximum predicted probability among the predicted probabilities of any layer corresponding to any row of the heat map 650. In this sense, the index C is an example of information based on the maximum probability among the probabilities of any layer associated with any of the multiple elements in the correspondence information.

[0077] Thereafter, the index calculation unit 54 calculates the index TRUTHFUL by "TRUTHFUL=A+B+C."

[0078] Here, the index TRUTHFUL is a value calculated by "TRUTHFUL=A+B+C." However, the index TRUTHFUL is not limited to this value.

[0079] The indicator TRUTHFUL may be any value based on the indicator A. In this case, the indicator TRUTHFUL is an example of information based on the probability of any layer associated with any element other than the last element among the multiple elements in the correspondence information. The indicator TRUTHFUL may be any value based on the indicator B. In this case, the indicator TRUTHFUL is an example of information based on the probability of any one of the layers associated with the last element of the multiple elements in the correspondence information. The index TRUTHFUL may be any value based on the index C. In this case, the index TRUTHFUL is an example of information based on the maximum probability of any layer associated with any of the multiple elements in the correspondence information.

[0080] Furthermore, the index TRUTHFUL does not have to be the sum of the indexes A, B, and C. The index TRUTHFUL may be, for example, a weighted sum in which weighting is assigned to an index that should be emphasized among the indexes A, B, and C. In general, the index TRUTHFUL may be any value based on the indexes A, B, and C. In this case, the index TRUTHFUL is an example of information based on the probability of any layer associated with any element other than the last element of the multiple elements in the correspondence information, the probability of any layer associated with the last element of the multiple elements in the correspondence information, and the maximum probability of any layer associated with any of the multiple elements in the correspondence information.

[0081] As a result, the greater the value of the index TRUTHFUL, the lower the probability of hallucination. Conversely, the smaller the value of the index TRUTHFUL, the higher the probability of hallucination.

[0082] Suppose the index calculation unit 54 calculates indexes A, B, and C from the heat map 650 shown in Fig. 12 using a certain document generation model. In this case, suppose index A is "0.151", index B is "0.215", and index C is "0.233". Then, the total index TRUTHFUL is "0.599".

[0083] On the other hand, suppose that the index calculation unit 54 calculates indexes A, B, and C from the heat map 650 shown in Fig. 13 using a certain document generation model. In this case, suppose that index A is "0.020", index B is "0.022", and index C is "0.064". Then, the total index TRUTHFUL is "0.106".

[0084] Next, the processing contents of the hallucination determination unit 55 will be described in detail. As described above, the index TRUTHFUL when hallucination is not occurring is, for example, "0.599." Also, the index TRUTHFUL when hallucination is occurring is, for example, "0.106."

[0085] Therefore, the hallucination assessment unit 55 determines whether the index TRUTHFUL is less than 0.2. If the index TRUTHFUL is less than 0.2, the hallucination assessment unit 55 generates first hallucination information. This indicates that the hallucination probability is at the highest level.

[0086] The hallucination assessment unit 55 also determines whether the index TRUTHFUL is less than 0.5. If the index TRUTHFUL is less than 0.5, the hallucination assessment unit 55 generates second hallucination information. This indicates that the hallucination probability is at the second highest level.

[0087] Furthermore, the hallucination assessment unit 55 determines whether the index TRUTHFUL is equal to or greater than 0.5. If the index TRUTHFUL is equal to or greater than 0.5, the hallucination assessment unit 55 does not generate hallucination information. This indicates that the hallucination probability is at the lowest level.

[0088] (Server operation example) FIG. 15 is a flowchart showing an example of the operation of the server 40 constituting the text generation system 1 in this embodiment.

[0089] As shown in the figure, in the server 40, the receiving unit 51 first receives an input sentence from the user terminal 10 (step 501). Next, the sentence acquisition unit 52 acquires an output sentence that is a response to the input sentence received in step 501 (step 502). Specifically, the sentence acquisition unit 52 acquires an output sentence that the AI ​​has generated based on the input sentence using the large-scale language model 600.

[0090] Next, the knowledge neuron analysis unit 53 generates a heat map 650 based on the result of the knowledge neuron analysis (step 503). Here, the knowledge neuron analysis is performed when the AI ​​generates an output sentence using the large-scale language model 600. The knowledge neuron analysis is an analysis of how much each neuron of the large-scale language model contributes to predicting the next word in the output sentence.

[0091] Next, the index calculation unit 54 calculates an index A from the heat map 650 generated in step 503 (step 504). For example, the index A may be an index indicating that there is only one region with a high prediction probability in the upper left corner of the heat map 650. Furthermore, the index calculation unit 54 calculates index B from the heat map 650 generated in step 503 (step 505). For example, index B may be an index indicating that the prediction probability on the right side of the last row of the heat map 650 is high. Furthermore, the index calculation unit 54 calculates an index C from the heat map 650 generated in step 503 (step 506). For example, the index C may be an index indicating a large difference in prediction probability between a region with a high prediction probability and a region with a low prediction probability. As a result, the index calculation unit 54 calculates the index TRUTHFUL (step 507). For example, the index calculation unit 54 may set the sum of the indexes A, B, and C calculated in steps 504 to 506 as the index TRUTHFUL.

[0092] Next, the hallucination determination unit 55 determines whether the index TRUTHFUL is less than a threshold value TH1 (step 508). For example, the threshold value TH1 may be set to "0.2." Assume that it is determined in step 508 that the index TRUTHFUL is less than the threshold value TH1. Then, the hallucination determination unit 55 generates first hallucination information (step 509). Here, the first hallucination information is information indicating that the hallucination probability is at the highest level.

[0093] The hallucination determination unit 55 also determines whether the index TRUTHFUL is less than a threshold value TH2 (step 510). Here, the threshold value TH2 is assumed to be greater than the threshold value TH1. For example, the threshold value TH2 may be set to "0.5." Assume that it is determined in step 510 that the index TRUTHFUL is less than the threshold value TH2. Then, the hallucination determination unit 55 generates second hallucination information (step 511). Here, the second hallucination information is information indicating that the hallucination probability is at the second highest level.

[0094] Thereafter, the transmitting unit 56 transmits the output sentence to the user terminal 10 (step 512). Here, it is preferable to use the sentence obtained in step 502 as the output sentence. Hallucination information may be added to the output sentence. That is, suppose that it is determined in step 508 that the index TRUTHFUL is less than the threshold value TH1. In this case, the first hallucination information generated in step 509 may be added to the output sentence. Also, suppose that it is determined in step 510 that the index TRUTHFUL is less than the threshold value TH2. In this case, the second hallucination information generated in step 511 may be added to the output sentence. Furthermore, suppose that it is determined in step 510 that the index TRUTHFUL is equal to or greater than the threshold value TH2. In this case, hallucination information need not be added to the output sentence.

[0095] (Variation) In the above, the server 40 generates the hallucination information for a word in the output sentence by analyzing the entire output sentence. However, the server 40 may also generate the hallucination information for a word in the output sentence by analyzing the part of the output sentence preceding that word. Furthermore, the above description does not mention the timing at which the hallucination information is displayed by the user terminal 10. The user terminal 10 may be configured to display the hallucination information simultaneously or sequentially while the output sentence is being generated by the AI.

[0096] Above, hallucination was described as a phenomenon in which an AI outputs factual errors that contradict the facts, but it is not limited to this. For example, hallucination can be defined as a factual error in which an AI outputs content that does not contain any facts that correspond to the answer. This embodiment is also applicable to cases where hallucination is a fidelity error. Here, a fidelity error is a phenomenon in which content that contradicts instructions given to a large-scale language model is output. A fidelity error is also a phenomenon in which content that contradicts information given to a large-scale language model is output.

[0097] (Processor) In this embodiment, each process is executed by an arbitrary computer. Furthermore, the arbitrary computer may execute these processes by a processor as hardware, a program as software, or a combination thereof. In this case, the processor is configured to execute various processes in this embodiment in cooperation with the program, and may function as each unit or means in this embodiment. Furthermore, the order in which the processes are executed by the processor is not limited to the order described and may be changed as appropriate. The arbitrary computer may be a general-purpose computer, a computer for specific applications, a workstation, or any other system capable of executing each process.

[0098] The processor may be composed of one or more pieces of hardware, and the type of hardware is not limited. For example, the processor may be composed of hardware such as a programmable logic device such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or an FPGA (Field Programmable Gate Array), a dedicated circuit for executing specific processes such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). The type of hardware may also be a combination of different types of hardware. When multiple pieces of hardware are configured to execute one or more processes of a certain processor, the multiple pieces of hardware may exist in devices physically separated from each other or in the same device. In any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware may be composed of an electric circuit or the like, which is a combination of circuit elements such as semiconductor elements.

[0099] Furthermore, the program may be software, such as firmware or microcode. The program may also be, for example, a group of program modules, each function of which may be implemented by a processor configured to perform the respective function. The program may be program code or multiple code segments stored in one or more non-transitory computer-readable media (e.g., storage media or other storages). The program may be stored across multiple non-transitory computer-readable media that reside in physically separate devices. The program code or code segment may represent a procedure, a function, a subprogram, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. The program code or code segment may be connected to another code segment or a hardware circuit by sending or receiving information, data, arguments, parameters, or memory contents.

[0100] (program) The present invention is also applicable to programs and program products. For example, a program and program product to which this embodiment is applied enables a computer to realize the following functions: acquiring input data; acquiring output data generated by AI according to the input data; and outputting the output data and error probability information indicating the degree of possibility that elements of the output data may contain errors made by the AI. The program to which this embodiment is applied can be provided by a communication means. Also, the program to which this embodiment is applied can be provided by being stored on a recording medium such as a CD-ROM.

[0101] (Addendum) (((1))) a processor; The processor: Get the input data, Obtaining output data generated by AI (Artificial Intelligence) in accordance with the input data; outputting the output data and error probability information indicating the degree of possibility that the elements of the output data include an error made by the AI; Information processing system. (((2))) The information processing system according to (((1))), wherein the processor outputs the output data and the error probability information with the error probability information added to the element. (((3))) the output data is text data, The information processing system described in (((2)))), wherein the processor adds the error probability information to the element by setting the attribute of a character corresponding to the element of the text data to an attribute corresponding to the error probability information. (((4))) The information processing system according to any one of ((1))) to ((3))), wherein the elements are all elements of the output data. (((5))) The information processing system according to any one of ((1))) to ((3))), wherein the element is an element of a predetermined type from among a plurality of elements of the output data. (((6))) the output data is text data, The information processing system according to (((5))), wherein the predetermined type of element is at least one of an element of a predetermined part of speech and an element consisting of a number. (((7))) The information processing system according to any one of ((1))) to ((3))), wherein the element is an element other than an element of a predetermined type among the plurality of elements of the output data. (((8))) the output data is text data, The information processing system according to (((7))), wherein the predetermined type of element is an element of a predetermined part of speech. (((9))) the output data is text data, The information processing system according to (((7))), wherein the predetermined type of element is at least one of an antonym element and a translation element. (((10))) The processor: As the output data, data generated by the AI ​​based on a large-scale language model is obtained; The information processing system according to any one of (((1))) to (((9))), wherein the information processing system outputs, as the error probability information, information generated based on identifiability information indicating a degree of possibility of being able to identify a knowledge neuron corresponding to the element in the large-scale language model. (((11))) The information processing system according to (((10))), wherein the processor generates the error probability information indicating that the element is likely to contain an error made by the AI ​​when the identifiability information indicates that it is unlikely that a knowledge neuron corresponding to the element can be identified in the large-scale language model. (((12))) The information processing system described in (((10)))), wherein the processor acquires the identifiability information based on correspondence information that associates each element of the multiple elements of the output data with the probability that neurons in multiple layers of each element will contribute to determining the next element of the multiple elements. (((13))) The information processing system described in (((12)))), wherein the processor obtains, as the identifiability information, information based on the probability of any layer associated with any element other than the last element among the plurality of elements in the correspondence information. (((14))) The information processing system described in (((12))), wherein the processor obtains, as the identifiability information, information based on the probability of any layer associated with the last element of the plurality of elements in the correspondence information. (((15))) The information processing system described in (((12)))), wherein the processor obtains, as the identifiability information, information based on the maximum probability among the probabilities of any layer associated with any of the plurality of elements in the correspondence information. (((16))) The processor: The identifiability information includes: a probability of any layer associated with any element other than the last element among the plurality of elements in the correspondence information; a probability of any layer associated with the last element of the plurality of elements in the correspondence information; and In the correspondence information, the maximum probability of any layer associated with any of the plurality of elements is The information processing system according to (((12)))), which acquires information based on the above. (((17))) On the computer, The function to obtain input data; A function of acquiring output data generated by AI (Artificial Intelligence) in accordance with the input data; A function to output the output data and error probability information indicating the degree of possibility that the elements of the output data contain errors made by the AI; A program to achieve this.

[0102] According to the invention of (((1))), it is possible to make the user aware of the degree to which elements of the output data are likely to contain errors caused by AI. According to the invention of (((2))), it becomes easy for the user to recognize the correspondence between the elements of the output data and the degree of possibility that the data may contain errors caused by the AI. According to the invention of (((3))), the user can recognize at a glance the correspondence between the elements of the output data and the degree of possibility that the data contains errors made by the AI. According to the invention of (((4))), it is possible to make the user aware of the degree of possibility that each element of the output data contains an error made by AI. According to the invention (((5))), it is possible to make the user aware of the degree of possibility that each element of a predetermined type of output data contains an error made by AI. According to the invention (((6))), it is possible to allow the user to recognize the degree of possibility that each element of the output data, which is a predetermined part of speech or a numerical element, may contain an error made by the AI. According to the invention (((7))), for elements other than the predetermined types of elements in the output data, the user can be made aware of the degree of possibility that each element may contain an error made by AI. According to the invention (((8))), for elements of the output data other than elements of predetermined parts of speech, the user can be made aware of the degree of possibility that each element may contain an error made by the AI. According to the invention (((9))), it is possible to allow the user to recognize the degree of possibility that each element other than at least one of the antonym element and the translation element of the output data may contain an error made by AI. According to the invention of (((10))), it is possible to make the user aware of the degree to which elements of the output data are likely to contain errors made by AI, without the need for an external database. According to the invention of (((11))), when it is unlikely that a knowledge neuron corresponding to an element can be identified in a large-scale language model, the user can be made aware that there is a high possibility that the element of the output data contains an error made by AI. According to the invention of (((12))), it becomes easy to recognize the degree of possibility of identifying knowledge neurons corresponding to elements in a large-scale language model. According to the invention of (((13))), by focusing on the probability of the layer associated with an element other than the last element of the output data, it is possible to recognize the degree of possibility of identifying a knowledge neuron corresponding to an element in a large-scale language model. According to the invention of (((14))), by focusing on the probability of the layer associated with the last element of the output data, it is possible to recognize the degree of possibility of identifying a knowledge neuron corresponding to an element in a large-scale language model. According to the invention of (((15))), by focusing on the maximum probability among the probabilities of the layers associated with the elements of the output data, it is possible to recognize the degree of possibility of identifying a knowledge neuron corresponding to an element in a large-scale language model. According to the invention of (((16))), it is possible to recognize the degree of possibility of identifying a knowledge neuron corresponding to an element in a large-scale language model by focusing on the probability of the layer associated with an element other than the last element of the output data, the probability of the layer associated with the last element of the output data, and the maximum probability among the probabilities of the layers associated with the elements of the output data. According to the invention of (((17))), it is possible to make the user aware of the degree to which elements of the output data are likely to contain errors caused by AI. [Explanation of symbols]

[0103] 1...Sentence generation system, 10...User terminal, 21...Operation reception unit, 22...Transmission unit, 23...Reception unit, 24...Display control unit, 40...Server, 51...Reception unit, 52...Sentence acquisition unit, 53...Knowledge neuron analysis unit, 54...Index calculation unit, 55...Hallucination determination unit, 56...Transmission unit

Claims

1. a processor; The processor: Get the input data, Obtaining output data generated by AI (Artificial Intelligence) in accordance with the input data; outputting the output data and error probability information indicating the degree of probability that an element of the output data includes an error caused by the AI; Information processing system.

2. The information processing system according to claim 1 , wherein the processor outputs the output data and the error probability information in a state where the error probability information is added to the element.

3. the output data is text data, The information processing system according to claim 2 , wherein the processor adds the error probability information to the element by setting an attribute of a character corresponding to the element of the text data to an attribute corresponding to the error probability information.

4. The information processing system according to claim 1 , wherein the elements are all elements of the output data.

5. The information processing system according to claim 1 , wherein the element is an element of a predetermined type among a plurality of elements of the output data.

6. the output data is text data, The information processing system according to claim 5 , wherein the predetermined type of element is at least one of an element of a predetermined part of speech and an element consisting of a number.

7. The information processing system according to claim 1 , wherein the element is an element other than an element of a predetermined type among the plurality of elements of the output data.

8. the output data is text data, The information processing system according to claim 7 , wherein the predetermined type of element is an element of a predetermined part of speech.

9. the output data is text data, The information processing system according to claim 7 , wherein the predetermined type of element is at least one of an antonym element and a translation element.

10. The processor: As the output data, data generated by the AI ​​based on a large-scale language model is obtained; 2. The information processing system according to claim 1, wherein the error probability information is generated based on identifiability information indicating a degree of possibility that a knowledge neuron corresponding to the element can be identified in the large-scale language model.

11. 11. The information processing system according to claim 10, wherein the processor generates the error probability information indicating that the element is likely to contain an error made by the AI ​​when the identifiability information indicates that it is unlikely that a knowledge neuron corresponding to the element can be identified in the large-scale language model.

12. The information processing system according to claim 10, wherein the processor acquires the identifiability information based on correspondence information that associates each element of the plurality of elements of the output data with a probability that neurons in a plurality of layers of the element will contribute to determining a next element of the plurality of elements.

13. The information processing system according to claim 12 , wherein the processor acquires, as the identifiability information, information based on the probability of any layer associated with any element other than a last element among the plurality of elements in the correspondence information.

14. The information processing system according to claim 12 , wherein the processor acquires, as the identifiability information, information based on a probability of any layer associated with a last element of the plurality of elements in the correspondence information.

15. The information processing system according to claim 12 , wherein the processor acquires, as the identifiability information, information based on a maximum probability among probabilities of any layer associated with any of the plurality of elements in the correspondence information.

16. The processor: The identifiability information includes: a probability of any layer associated with any element other than the last element among the plurality of elements in the correspondence information; a probability of any layer associated with the last element of the plurality of elements in the correspondence information; and In the correspondence information, the maximum probability of any layer associated with any of the plurality of elements is The information processing system according to claim 12 , wherein the information is obtained based on the above.

17. On the computer, The function to obtain input data; a function of acquiring output data generated by AI (Artificial Intelligence) in accordance with the input data; A function to output the output data and error probability information indicating the degree of probability that an element of the output data includes an error caused by the AI; A program to achieve this.

Images