Answer output device and answer output system

Abstract

The invention provides a better artificial intelligence answer output technology. According to the invention, it is possible to contribute to "9 industry, innovation and infrastructure" and "11 sustainable cities and communities" of sustainable development objectives (SDGs). An answer output device includes: a control unit that acquires, from a large-scale language model, an answer given to a cue input to the large-scale language model; a display unit; and a sound output unit, in which, in a control state of the control unit, there is a state in which an answer from the large-scale language model is output via the display unit or the sound output unit.

Description

Technical Field

[0001] This invention relates to a response output device and a response output system. Background Technology

[0002] Regarding response output technologies that utilize artificial intelligence such as language models, for example, patent document 1 has been disclosed.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Publication No. 2019-528512 Summary of the Invention

[0006] The technical problem that the invention aims to solve

[0007] However, the content disclosed in Patent Document 1 does not fully consider what structure to adopt to better provide users with answer output technology implemented using artificial intelligence.

[0008] The purpose of this invention is to provide a better response output technology.

[0009] Technical means to solve the problem

[0010] To address the aforementioned problems, for example, the structure described in the claimed technical solution can be employed. This application includes various technical solutions to solve the aforementioned problems. One example is a response output device, which may be configured to include: a control unit that obtains responses from a large-scale language model in response to prompts input to the large-scale language model; a display unit; and a sound output unit, wherein, in the control state of the control unit, there exists a state in which the response from the large-scale language model is output via the display unit or the sound output unit.

[0011] Invention Effects

[0012] According to the present invention, a better response output technology can be provided. Other technical problems, technical features, and technical effects will become clear in the following description of the embodiments. Attached Figure Description

[0013] Figure 1A This is a diagram illustrating an example of an artificial intelligence response output device and system according to an embodiment of the present invention.

[0014] Figure 1B This is a diagram illustrating an example of an artificial intelligence response output device according to an embodiment of the present invention.

[0015] Figure 1C This is a diagram illustrating an example of the operation of an artificial intelligence response output device and system according to an embodiment of the present invention.

[0016] Figure 2A This is an illustrative diagram illustrating an example of a role-playing device and role-playing system according to an embodiment of the present invention.

[0017] Figure 2B This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0018] Figure 2C This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0019] Figure 2D This is an explanatory diagram illustrating an example of a session in a role-based conversation device and role-based conversation system, which is an embodiment of the present invention.

[0020] Figure 2E This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0021] Figure 2F This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0022] Figure 2G This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0023] Figure 2H This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0024] Figure 2I This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0025] Figure 2J This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0026] Figure 2K This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0027] Figure 2L This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0028] Figure 3A This is an illustrative diagram illustrating an example of a role-playing device and role-playing system according to an embodiment of the present invention.

[0029] Figure 3BThis is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0030] Figure 3C This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0031] Figure 3D This is an explanatory diagram illustrating an example of a session in a role-based conversation device and role-based conversation system, which is an embodiment of the present invention.

[0032] Figure 3E This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0033] Figure 3F This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0034] Figure 3G This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0035] Figure 3H This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0036] Figure 3I This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0037] Figure 4A This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0038] Figure 4B This is an explanatory diagram illustrating an example of the operation of a role-playing device and a role-playing system according to an embodiment of the present invention.

[0039] Figure 5A This is an explanatory diagram illustrating an example of the operation of an artificial intelligence response output device according to an embodiment of the present invention.

[0040] Figure 5B This is an explanatory diagram of an example of a display example of an artificial intelligence answer output device according to an embodiment of the present invention.

[0041] Figure 5C This is an explanatory diagram of an example of a display example of an artificial intelligence answer output device according to an embodiment of the present invention.

[0042] Figure 5D This is an explanatory diagram of an example of a display example of an artificial intelligence answer output device according to an embodiment of the present invention.

[0043] Figure 6 This is an explanatory diagram illustrating an example of the answer generation process of an artificial intelligence answer output device according to an embodiment of the present invention.

[0044] Figure 7A This is an explanatory diagram illustrating an example of the answer generation process of an artificial intelligence answer output device according to an embodiment of the present invention.

[0045] Figure 7B This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention.

[0046] Figure 7C This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention.

[0047] Figure 7D This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention.

[0048] Figure 7E This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention.

[0049] Figure 8A This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention.

[0050] Figure 8B This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention.

[0051] Figure 8C This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention.

[0052] Figure 9A This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention.

[0053] Figure 9B This is an explanatory diagram illustrating an example of the processing of an artificial intelligence response output device according to an embodiment of the present invention. Detailed Implementation

[0054] The embodiments of the present invention are described in detail below based on the accompanying drawings. However, the present invention is not limited to the description of the embodiments, and those skilled in the art can make various changes and modifications within the scope of the technical concept disclosed in this specification. Furthermore, in all the drawings used to illustrate the present invention, parts having the same function are labeled with the same reference numerals, and sometimes repeated descriptions are omitted.

[0055] Furthermore, the AI ​​response output device in the various embodiments of the present invention, when it has a display screen, can also be called a display device. The AI ​​response output device, when it has a sound output function, can also be called a sound output device. The AI ​​response output device can also be simply referred to as an information processing device. The system including the AI ​​response output device and the large-scale language model server that maintains the large-scale language model can be called an AI response output system. Furthermore, when the AI ​​response output device provides users with response services based on the large-scale language model of the AI, offering assistance to the user, the AI ​​response output device or its display output can act as an AI assistant for the user. Therefore, in this case, the AI ​​response output device can also be called an AI assistant device or an AI assistant display device. Similarly, in this case, the system including the AI ​​response output device and the large-scale language model server that maintains the large-scale language model can be called an AI assistant system or an AI assistant display system. Furthermore, in this case, since the AI ​​response output device is the interface between the user and the AI, it can also be called an AI interface device. In this case, the system including the AI ​​response output device and the large-scale language model server that maintains the large-scale language model can be called an AI interface system.

[0056] <Example 1>

[0057] As an embodiment 1 of the present invention, an artificial intelligence answer output device and system thereof that outputs answers from large-scale language model artificial intelligence will be described.

[0058] use Figure 1A An example of the artificial intelligence response output device 10010 of the present invention will be described. Furthermore, regarding the case where the artificial intelligence response output device 10010 collaborates with a large-scale language model server 19001 via communication or the like, an example of a system including a large-scale language model server 19001 and / or a multimodal large-scale language model server 20001 will be described.

[0059] Figure 1A In the example, the artificial intelligence answer output device 10010 has a display unit 10011. Figure 1AIn this example, the display unit 10011 can be a flat panel display, a screen that projects images from the back, or an aerial levitation image that forms an optical image in the air. If the display unit 10011 is a flat panel display, it can be a liquid crystal display (LCD) with a liquid crystal panel and a backlight. Alternatively, the display unit 10011 can be a plasma display. The display unit 10011 can also be an organic EL display with self-emissive pixels. Furthermore, the display unit 10011 can be equipped with a touch input sensor, thus configuring it as a touch panel.

[0060] Figure 1A In the example, the AI ​​response output device 10010 includes a sound output unit 1140 composed of a speaker. Additionally, the AI ​​response output device 10010 includes a microphone 1139 capable of capturing the user's voice. Through sound input from this microphone 1139 and user operation input via the operation input unit described later, the AI ​​response output device 10010 can obtain user input, which becomes the source of prompts for the AI, i.e., a large-scale language model.

[0061] The AI ​​response output device 10010 may have a local large-scale language model. In this case, the response from the large-scale language model can be used as the display output of the display unit 10011 and / or the sound output of the sound output unit 1140.

[0062] Alternatively, the AI ​​response output device 10010 may not have a local large-scale language model, but may communicate with an external large-scale language model server 19001 and output the responses received from the large-scale language model server 19001 as the display output of the display unit 10011 and / or the sound output of the sound output unit 1140.

[0063] Alternatively, the AI ​​response output device 10010 may be configured to have a local large-scale language model and also communicate with an external large-scale language model server 19001 that has a large-scale language model or an external large-scale language model server 20001 that has a multimodal large-scale language model. In this case, the response from the local large-scale language model and the response received from the large-scale language model of the large-scale language model server 19001 or the multimodal large-scale language model of the multimodal large-scale language model server 20001 can be switched, and either one can be output as the display output of the display unit 10011 and / or the sound output of the sound output unit 1140. Alternatively, a response may be generated based on both the response from the local large-scale language model and the response received from the large-scale language model of the large-scale language model server 19001 or the multimodal large-scale language model of the multimodal large-scale language model server 20001, and the generated response can be output as the display output of the display unit 10011 and / or the sound output of the sound output unit 1140.

[0064] The structure of the AI ​​response output device 10010 in the case of communicating and cooperating with an external large-scale language model server 19001 or large-scale language model server 20001 is as follows. The AI ​​response output device 10010 can communicate with the communication device 19011 connected to the Internet 19000 via the communication unit 1132. Figure 1A The example shown illustrates wireless communication between communication unit 1132 and communication device 19011, but wired communication is also possible. The communication path from communication unit 1132 to communication device 19011 can include both wired and wireless components, and may also pass through routers or repeaters. Similarly, the communication path from communication unit 1132 to the Internet 19000 can also include both wired and wireless components, and may also pass through routers or repeaters. The AI ​​response output device 10010 can communicate with the large-scale language model server 19001 via communication device 19011 and the Internet 19000. Furthermore, the AI ​​response output device 10010 can communicate with the large-scale language model server 19001 or large-scale language model server 20001, and a second server 19002 (different from these servers) via communication device 19011 and the Internet 19000. The structure including the AI ​​response output device 10010 and the large-scale language model server 19001 or large-scale language model server 20001 can be considered as a system.

[0065] In the following description, unless otherwise specified, the term "large-scale language model" can be considered to include the local large-scale language model possessed by the AI ​​response output device 10010, the large-scale language model possessed by the large-scale language model server 19001, and the multimodal large-scale language model possessed by the large-scale language model server 20001.

[0066] Figure 1A In the example shown, the display unit 10011 displays various elements in two display areas: a prompt display area 10051 for user input of prompts to artificial intelligence, i.e., a large-scale language model, and an artificial intelligence answer display area 10061 for displaying answers from the large-scale language model. Figure 1A The example illustrates an instance in which the prompt display area 10051 displays an icon 10052 representing the user, text 10053 consisting of natural language or software code as components of the prompt, an image 10054 consisting of components of the prompt, a video 10055 consisting of components of the prompt, etc. Figure 1A The example illustrates an instance where, in the AI ​​response display area 10061, an icon 10062 representing artificial intelligence or an AI assistant, text 10063 including natural language or software code as components of the AI ​​response, an image 10064 as a component of the AI ​​response, and a video 10065 as a component of the AI ​​response are displayed. Additionally, Figure 1A The display example of the display unit 10011 of the artificial intelligence response output device 10010 shown is just one example. Based on the implementation example using the artificial intelligence response output device 10010, further modifications can be made. Figure 1A The examples shown are displayed differently.

[0067] Here, large-scale language models are explained. Large-scale language models are also referred to as LLM (Large Language Model). Specifically, various models such as GPT-1, GPT-2, GPT-3, InstructGPT, and ChatGPT are disclosed. These techniques can be used in this embodiment. These large-scale language models are artificial intelligence models generated through large-scale pre-training on the natural language contained in a large number of documents and texts existing in human society. The number of parameters in these artificial intelligence models exceeds one hundred million. Furthermore, in addition to these, there are models that implement reinforcement learning based on human feedback. An example of a basic model is a model called Transformer. For example, reference 1 discloses a training example for these models.

[0068] [Reference 1]

[0069] Long Ouyang, et. al. "Training language models to follow instructions with human feedback", https: / / arxiv.org / pdf / 2203.02155.pdf

[0070] These large-scale language models are capable of translating, correcting, and summarizing natural language texts. More advanced models can even answer questions (also known as dialogues or conversations) in natural language, generate suggestions, and generate programming code. Because these AI models have a very large number of parameters, training them requires enormous amounts of data and computational resources. Therefore, training at this level of AI for specific purposes is extremely inefficient. For this reason, foundation models are generated through large-scale pre-training to serve as versatile applications. For example, Figure 1A The large-scale language model server 19001 shown can be configured to possess such a large-scale language model and can be used through various terminals via an API (Application Programming Interface). Furthermore, Figure 1A The AI ​​response output device 10010 shown can be configured to have a local large-scale language model for its own use. For any large-scale language model, its training can be performed separately on a large scale to generate the model, and the generated large-scale language model can be copied and set up in the large-scale language model server 19001 or the AI ​​response output device 10010, etc. In this way, instead of pre-training according to purpose or terminal, a large-scale pre-training is performed to generate a large-scale language model as the base model, and this large-scale language model is copied for use on various servers and terminals. Therefore, the resource consumption used in training can be shared, resulting in good resource efficiency.

[0071] Furthermore, even large-scale language models that are generated as the base model through large-scale pre-training can be additionally trained on various servers or devices according to their intended use or purpose, such as through transfer learning.

[0072] Furthermore, large-scale language models can pre-train natural language models to perform input-output processing on natural language. Therefore, multimodal large-scale language model artificial intelligence, capable of processing information other than natural language text information, can also be applied to embodiments of the present invention. Figure 1AThe diagram illustrates a large-scale language model server 20001 with a multimodal large-scale language model. Specifically, GPT-4 (see Reference 2) and Gato (see Reference 3) are disclosed as examples of multimodal large-scale language model artificial intelligence. These techniques can be used in this embodiment. Furthermore, these multimodal large-scale language models are artificial intelligence models generated through large-scale pre-training on a vast amount of natural language and other types of information (e.g., images, videos, sounds, etc.) contained in documents and texts existing in human society. In addition, models implementing reinforcement learning based on human feedback also exist. Hereinafter, information other than natural language text, such as images, videos, and sounds, can be referred to as non-natural language information sources.

[0073] [Reference 2]

[0074] Open AI "GPT-4 Technical Report", https: / / cdn.openai.com / papers / gpt-4.pdf

[0075] [Reference 3]

[0076] Scott Reed, et. al. "A Generalist Agent", https: / / arxiv.org / pdf / 2205.06175.pdf

[0077] Next, use Figure 1B An example of the structure of the artificial intelligence response output device 10010 will be described. This artificial intelligence response output device 10010 accepts input from users of artificial intelligence such as large-scale language models, and outputs the responses given by the artificial intelligence such as large-scale language models to the user's input.

[0078] The AI ​​response output device 10010 includes a display unit 10011, a control unit 1110, a memory 1109, a non-volatile memory 1108, an external power input interface 1111, an operation input unit 1107, a power supply 1106, a secondary battery 1112, a storage unit 1170, an image control unit 1160, a posture sensor 1113, a communication unit 1132, an audio output unit 1140, a microphone 1139, an image signal input unit 1131, an audio signal input unit 1133, and a camera unit 1180. The AI ​​response output device 10010 may, for example, have a large screen such as a monitor or television.

[0079] Display unit 10011 can be a flat panel display, a screen that projects images from the back, or a suspended image that forms an optical image in the air. When display unit 10011 is a flat panel display, it can be a liquid crystal display (LCD) with a liquid crystal panel and a backlight. Alternatively, display unit 10011 can be a plasma display. Display unit 10011 can also be an organic EL display with self-emissive pixels. When display unit 10011 is a panel, it can also be called a display panel. A touch operation input sensor can also be provided in display unit 10011, configured to accept touch operation input from the user 230's finger. In this case, display unit 10011 can be configured as a touch panel. Through the user's operation input via this touch panel, the artificial intelligence response output device 10010 can obtain the user input, which becomes the source of prompts for artificial intelligence, i.e., a large-scale language model.

[0080] The communication unit 1132 can be configured using Wi-Fi, Bluetooth (registered trademark), or mobile communication interfaces such as 4G and 5G. The communication unit 1132 of the AI ​​response output device 10010 can use these communication methods to communicate with the communication device 19011 connected to the Internet 19000. Furthermore, the communication path from the communication unit 1132 to the communication device 19011 can include both wired and wireless components, and may also pass through a router or repeater. In the wired case, the communication unit 1132 can have an Ethernet connection interface and communicate using a LAN-based communication method. Therefore, the AI ​​response output device 10010 can communicate with various servers connected to the Internet 19000.

[0081] The artificial intelligence response output device 10010 includes a control unit 1110 such as a CPU and a memory 1109. The control unit 1110 controls the display unit 10011 and the communication unit 1132, etc.

[0082] Power supply 1106 converts AC current input from the outside via external power input interface 1111 into DC current, supplying the required DC current to each part of the artificial intelligence response output device 10010. Secondary battery 1112 stores the power (electrical power) supplied by power supply 1106. Furthermore, when no external power is supplied via external power input interface 1111, secondary battery 1112 supplies power to the parts requiring electricity.

[0083] The operation input unit 1107 is, for example, a signal receiving unit or an infrared light receiving unit such as an operation button or a remote control, used to input signals for operations different from the touch operations performed by the user on the touch operation input sensor of the display unit 10011. Besides users performing touch operations on the touch operation input sensor of the display unit 10011, the operation input unit 1107 can also be used, for example, by an administrator to operate the artificial intelligence response output device 10010. Through user operation input via this operation input unit 1107, the artificial intelligence response output device 10010 can obtain user input, which becomes the source of prompts for artificial intelligence, i.e., large-scale language models. Alternatively, a variation can be adopted in which the touch operation input sensor of the display unit 10011 is also part of the operation input unit 1107.

[0084] The video signal input unit 1131 connects to an external video output device to input video data. The video signal input unit 1131 can be various digital video input interfaces. For example, it can be a video input interface conforming to the HDMI (High-Definition Multimedia Interface) standard, a video input interface conforming to the DVI (Digital Visual Interface) standard, or a video input interface conforming to the DisplayPort standard. Alternatively, analog video input interfaces such as analog RGB and component video can also be provided. The video signal input unit 1131 can also be various USB interfaces, etc.

[0085] The audio signal input unit 1133 connects to an external audio output device to input audio data. The audio signal input unit 1133 can be configured as an HDMI standard audio input interface, an optical digital terminal interface, or a coaxial digital terminal interface, etc. The audio signal input unit 1133 can also be various USB interfaces, etc. When using an HDMI standard interface, the video signal input unit 1131 and the audio signal input unit 1133 can be configured as an interface integrating terminals and cables.

[0086] The audio output unit 1140 can output sound based on the audio data input to the audio signal input unit 1133. The audio output unit 1140 can also output sound based on the audio data stored in the storage unit 1170. The audio output unit 1140 can be configured as a speaker. Additionally, the audio output unit 1140 can also output built-in operation tones and error warning tones. Alternatively, the audio output unit 1140 can be configured to output digital signals as audio signals to external devices, similar to the Audio Return Channel function specified in the HDMI standard. Alternatively, the audio output unit 1140 can be configured to output analog signals as audio signals to external devices such as headphones.

[0087] Microphone 1039 is a microphone that collects ambient sounds from the artificial intelligence response output device 10010 and converts them into signals to generate sound signals. Alternatively, the microphone may record the voice of a user or other person (speech), and the generated sound signal is processed by the control unit 1110 (described later) for voice recognition processing (speech recognition processing) to obtain text information from the sound signal. By providing sound input through microphone 1139, the artificial intelligence response output device 10010 can obtain user input, which becomes the source of prompts for artificial intelligence, i.e., large-scale language models.

[0088] The camera unit 1180 is a camera equipped with an image sensor. The camera can be installed on the front side of the display unit 10011 of the artificial intelligence response output device 10010, or on the back side of the display unit 10011. Both a front-facing camera and a back-facing camera can be installed. In this embodiment, the camera unit 1180 will be described with both a front-facing camera and a back-facing camera.

[0089] Storage unit 1170 is a storage device that records various types of data and information, such as image data, audio data, etc. Storage unit 1170 can be composed of a magnetic recording medium recording device such as a hard disk drive (HDD) or a semiconductor device memory such as a solid-state drive (SSD). For example, image data, audio data, and other various types of data and information can be pre-recorded in storage unit 1170 at the time of product shipment. In addition, storage unit 1170 can also record image data, audio data, and other various types of data and information obtained from external devices and external servers via communication unit 1132. The image data and other data recorded in storage unit 1170 are output to display unit 10011. The image data and other data recorded in storage unit 1170 can also be output to external devices and / or external servers via communication unit 1132.

[0090] The image control unit 1160 performs various controls on the image signals input to the display unit 10011. The image control unit 1160 can also be called an image processing circuit, and can be constructed from hardware such as an ASIC, FPGA, or video processor. Alternatively, the image control unit 1160 can also be called an image processing unit or an image processing unit. For example, the image control unit 1160 controls image switching, switching which image signal (image data) stored in the memory 1109 or input to the image signal input unit 1131 is input to the image display unit 10011. Furthermore, the image control unit 1160 can also control image processing on the image signals input from the image signal input unit 1131 and the image signals stored in the memory 1109. Image processing includes, for example, scaling (enlarging, reducing, and distorting the image), brightness adjustment (changing brightness), contrast adjustment (changing the image's contrast curve), and Retinex processing (decomposing the image into light components and changing the weights of each component).

[0091] The attitude sensor 1113 is a sensor composed of a gravity sensor, an acceleration sensor, or a combination thereof, capable of detecting the attitude of the artificial intelligence response output device 10010. The control unit 1110 can control the operation of the connected components based on the attitude detection results of the attitude sensor 1113.

[0092] Non-volatile memory 1108 stores various data used in the artificial intelligence response output device 10010. The data stored in non-volatile memory 1108 includes, for example, various operational data to be displayed on the display unit 10011 of the artificial intelligence response output device 10010, display icons, data on objects for user operation, layout information, etc. Memory 1109 stores image data displayed on the display unit 10011 and / or device control data, etc. The control unit 1110 can read various software from the storage unit 1170, load it, and store it in memory 1109.

[0093] The local LLM processing unit 10028 has memory capable of storing a large-scale language model (LLM) and can perform inference of the large-scale language model under the control of the control unit 1110. As hardware, it can be constructed using a GPU (Graphics Processing Unit) or similar device. The local LLM processing unit 10028 can perform both inference and training. Furthermore, in cases where large-scale language model inference is not required in a local environment where the artificial intelligence response output device 10010 is not needed, the local LLM processing unit 10028 is not required.

[0094] The control unit 1110 controls the operation of each connected unit. Furthermore, the control unit 1110 can cooperate with programs stored in memory 1109 to perform calculations based on information obtained from each unit within the artificial intelligence response output device 10010. For example, the control unit 1110 may be in a state where, via the display unit 10011 or a speaker (i.e., the sound output unit 1140), it outputs a response from the local LLM processing unit 10028 (large-scale language model), or a response from the large-scale language model server 19001 or a multimodal large-scale language model server 20001 (large-scale language model obtained via the communication unit 1132).

[0095] Furthermore, when a user inputs information via the aforementioned touch panel, microphone 1139, or operation input unit 1107, the control unit 1110 can generate prompts based on the input and send them to the local large-scale language model of the local LLM processing unit 10028 of the artificial intelligence response output device 10010, the large-scale language model of the large-scale language model server 19001, or the multimodal large-scale language model of the large-scale language model server 20001, and obtain responses from these large-scale language models.

[0096] Additionally, the storage unit 1170 may also store a database of predefined answer statements (also referred to as a database of predefined answer statements DB), which is used to output predefined statements (fixed statements) as the responses of the AI ​​response output device 10010 to prompts. The control unit 1110 can control the generation of the responses to be output using the data stored in the database of predefined answer statements. Figure 1C The image shows an example of a database that responds to definite statements. Figure 1C In the example, for each condition marked with a condition number, the answers to the pre-defined statements output by the AI ​​response output device 10010 are stored. For example, as shown in condition number 1, if the user inputs "Good morning" via the touch panel, microphone 1139, or operation input unit 1107, "Good morning" or "Today is ○ month ○ day." is used as the pre-defined response statement. The "○" part, such as "○ month ○ day," can be generated using information stored in the memory 1109 or the like in the AI ​​response output device 10010.

[0097] in addition, Figure 1CIn the example of the answer formatting statements in the database shown, when multiple answer formatting statements separated by " / " are stored, the control unit 1110 can control the selection of a specific answer formatting statement randomly using a random number or the like and output the answer. This eliminates and improves the monotony of answers under the same conditions. The examples for conditions 2, 3, and 4 are explained in the same way as the example for condition 1. The control unit 1110 can control the selection of the answer formatting statements. Figure 1C The conditions in the examples shown are used Figure 1C The responses to the examples shown are output as fixed statements.

[0098] Next, regarding Figure 1C The example shown is condition number 5. In the example shown for condition number 5, control is performed such that, if the control unit 1110 fails to understand the meaning of user input obtained via the touch panel, microphone 1139, or operation input unit 1107 in natural language, or if there are obvious grammatical errors in the user input, the control unit 1110 outputs a response using a predefined statement such as "I didn't quite understand" or "I might not understand this." This response prompts the user to re-enter the information and allows for waiting for corrected user input.

[0099] Next, regarding Figure 1C The example shown is condition number 6. Condition number 6 is an example where, in this case, the control unit 1110 detects a constituent... Figure 1B When any part of the AI ​​response output device 10010 malfunctions (is in an abnormal state), user input occurs via the touch panel, microphone 1139, or operation input unit 1107. In this case, the control unit 1110 takes control and outputs a response using the phrase "The state seems not very good." This response informs the user that the AI ​​response output device 10010 is not functioning properly and prompts the user to resolve the error.

[0100] In the AI ​​response output device 10010, the response can replace the response generated by the large-scale language model, such as the local large-scale language model of the AI ​​response output device 10010, the large-scale language model of the large-scale language model server 19001, and the multimodal large-scale language model of the large-scale language model server 20001, and instead output the response generated by the large-scale language model that is used in the AI ​​response output device 10010. Figure 1C The output is the answer from the Response Type Statement Database (Response Type Statement DB). Alternatively, the output can be a combination of the answers from these large-scale language models and the answers from the Response Type Statement Database (Response Type Statement DB).

[0101] In addition, the above explanation Figure 1CThe database of definitive answer statements (answer definitive statement DB) is stored in the storage unit 1170, and the control unit 1110 of the artificial intelligence answer output device 10010 can use it to generate answers. However, Figure 1C The answer specification statement database (answer specification statement DB) shown can also be set on either the large-scale language model server 19001 or the large-scale language model server 20001. In this case, the control unit of the large-scale language model server 19001 or the large-scale language model server 20001 can generate an answer using the answer specification statement database (answer specification statement DB). In the control unit of the large-scale language model server 19001 or the large-scale language model server 20001, the answer generated using the answer specification statement database (answer specification statement DB) can be sent to the artificial intelligence answer output device 10010 instead of the answer generated by the large-scale language model stored in each server. In this way, even if the artificial intelligence answer output device 10010 does not have an answer specification statement database (answer specification statement DB), it is possible to generate an answer using the answer specification statement database (answer specification statement DB).

[0102] Furthermore, the above description indicates that the artificial intelligence response output device 10010 has a display panel that uses a display image with fixed pixels. This concept may also include a projection-type image display device (projector), in which a projection optical system is provided behind the display panel that uses a display image with fixed pixels to project an optical image of the display panel onto a screen or wall.

[0103] in addition, Figure 1A and Figure 1B The example described illustrates an AI response output device 10010 equipped with a display unit 10011. However, the AI ​​response output device 10010 of the embodiments of the present invention may also omit the display unit 10011. For example, even without the display unit 10011, it can be configured to accept user input to the AI ​​via the sound signal input unit 1133 or the microphone 1139, and output the AI's response to the user's input, such as a large-scale language model, via the sound output unit 1140.

[0104] According to the above description of Embodiment 1 of the present invention, the artificial intelligence answer output device and artificial intelligence answer output system can accept user input on artificial intelligence such as large-scale language models, and output an answer to the user's input generated by artificial intelligence through reasoning, such as a large-scale language model possessed by a server device on the network or a local large-scale language model possessed by the artificial intelligence answer output device itself.

[0105] <Example 2>

[0106] Next, as Embodiment 2 of the present invention, an example is described in which the artificial intelligence response output device 10010 described in Embodiment 1 is connected to the Internet and operates by connecting to a server equipped with a large-scale language model artificial intelligence via the Internet. The differences from Embodiment 1 are described in this embodiment, and repeated descriptions of structures identical to those in these embodiments are omitted.

[0107] use Figure 2A An example of the connection state between the AI ​​response output device 10010 of Embodiment 2 of the present invention and the large-scale language model server 19001 will be described. The AI ​​response output device 10010 of Embodiment 2 can also be referred to as a role-playing device. Furthermore, the system including the AI ​​response output device 10010 of Embodiment 2 and the large-scale language model server 19001 can be referred to as a role-playing system. An image of a role 19051 is displayed on the display unit 10011 of the AI ​​response output device 10010. The image of role 19051 is generated by rendering a 3D model of the role in virtual space.

[0108] Furthermore, the role in this embodiment can provide users with artificial intelligence, i.e., large-scale language model services, to assist them. Therefore, this role can act as an artificial intelligence (AI) assistant for the user. In this case, the role conversation device and role conversation system in this embodiment can also be referred to as an AI assistant conversation device, an AI assistant display device, an AI assistant response output device, an AI assistant conversation system, an AI assistant display system, and an AI assistant response output system.

[0109] Figure 2A In this example, the AI ​​response output device 10010 includes a sound output unit 1140 composed of a speaker. Additionally, the AI ​​response output device 10010 includes a microphone 1139 capable of capturing the user's voice. The AI ​​response output device 10010 can communicate with a communication device 19011 connected to the Internet 19000 via a communication unit 1132. Figure 2AThe example shown illustrates wireless communication between communication unit 1132 and communication device 19011, but wired communication is also possible. The communication path from communication unit 1132 to the Internet 19000 can contain both wired and wireless components. The AI ​​response output device 10010 can communicate with the large-scale language model server 19001 via communication device 19011 and the Internet 19000. Furthermore, the AI ​​response output device 10010 can communicate with a second server 19002, which is different from the large-scale language model server 19001, via communication device 19011 and the Internet 19000. The structure including the AI ​​response output device 10010 and the large-scale language model server 19001 can be considered as a system.

[0110] Next, use Figure 2B An example of the operation of the role-based conversation device (artificial intelligence response output device 10010) of Embodiment 2 of the present invention will be described. This can also be described as an example of the operation of a role-based conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 19001. Furthermore, Figure 2B Omitted Figure 2A The diagram shows communication paths such as Internet 19000. Figure 2B The image also shows user 230 of the AI-powered answer output device 10010.

[0111] Here, a series of procedures for the operation of the AI ​​response output device 10010 are explained. Furthermore, the AI ​​response output device 10010 loads a character action program stored in the storage unit 1170, etc., into the memory 1109, and the control unit 1110 executes the character action program, thereby enabling the various processes described below.

[0112] First, the AI ​​response output device 10010 includes a microphone 1139. When user 230 speaks to character 19051, the microphone 1139 captures the user's voice (the user's speech) and converts it into an audio signal. Here, the character action program executed by the control unit 1110 extracts the text of the speech spoken by user 230 from the audio signal. This text is natural language. Furthermore, the extraction of the text of the speech spoken by user 230 can be performed continuously on all speech, or it can begin when the user speaks within a predetermined period following a triggering keyword. For example, the triggering keyword could be the user saying the character's name after "Hello." For example, assuming the character 19051's name is "Koto," "Hello, Koto!" could be used as the triggering keyword.

[0113] The AI ​​response output device 10010's action program generates a prompt based on the text of the user 230's spoken words and sends the prompt to the large-scale language model server 19001 via an API. Here, the prompt may be metadata containing information described using syntaxes such as Markup (a markup language) using tags, Markdown (a markup format) using defined symbols, or JSON (a scripting language) using object syntax. The prompt contains natural language text information as the main message. The types of prompts sent from the AI ​​response output device 10010 to the large-scale language model server 19001 include setting prompts that store initial settings and other instructions, and user prompts that reflect the user's instructions. Information identifying whether a prompt is a setting prompt or a user prompt may also be stored in a portion other than the main message of that prompt. When the AI ​​response output device 10010's action program generates a prompt based on the text of the user 230's spoken words, it generates a user prompt and sends it to the large-scale language model server 19001.

[0114] Next, the large-scale language model server 19001 performs inference based on the prompt words sent from the artificial intelligence response output device 10010, and generates a response containing natural language text information based on the result. The large-scale language model server 19001 sends the response to the artificial intelligence response output device 10010 using an API. In this response, the natural language text information is stored as the main message. Here, the response may be metadata storing information described using the same syntax as the prompt words (such as the Markup format of markup languages ​​using tags, the Markdown format using specified symbols, or the object syntax of specified scripts such as JSON). In this response, when using the same format as the prompt words, to indicate that it is information of a different kind from the initial setting prompt words and user prompt words, category identification information can be stored in a part other than the main message. For example, information indicating that it is a response from the large-scale language model can be stored.

[0115] Next, the AI ​​response output device 10010 receives the response from the large-scale language model server 19001 and extracts the natural language text information stored as the main message from the response. Based on the natural language text information extracted from the response, the AI ​​response output device 10010's character action program uses voice synthesis (speech synthesis) technology to generate a natural language voice (speech) as a response to the user, which is output from the speaker, i.e., the sound output unit 1140, in a manner that sounds as if it were the voice of the character 19051. This process can be described as the character "speaking".

[0116] exist Figure 2C In the conversation examples 1 to 5, a specific example is shown of the response voice of character 19051 corresponding to the user 230's speech, obtained through processing by the artificial intelligence response output device 10010 and the large-scale language model server 19001 described above. In this way, the user 230 can have a conversation as if character 19051 were a real person.

[0117] Based on the above explanation Figure 2B The AI ​​response output device 10010, or a system including the AI ​​response output device 10010, does not require the AI ​​response output device 10010 itself to carry a large-scale language model that requires massive data and computing resources for training. Furthermore, it can utilize the advanced natural language processing capabilities of the large-scale language model via API, enabling better responses and conversations when the user speaks to the character.

[0118] Next, use Figure 2D This section describes an example of the operation of the role-playing conversation device (artificial intelligence response output device 10010) according to Embodiment 2 of the present invention. This can also be described as an example of the operation of a role-playing conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 19001. Specifically, Figure 2D This is an example of the natural language text of the main message of the prompt word sent from the AI ​​response output device 10010 to the large-scale language model server 19001, and the natural language text of the main message of the server response as its answer. The conversation between the character 19051 and the user 230 displayed on the AI ​​response output device 10010 is based on this.

[0119] in addition, Figure 2D The diagram illustrates the sequential interaction between prompts and responses, from the initial set prompt, the first user prompt and its response, up to the fourth user prompt and its response.

[0120] like Figure 2DAs shown, as an initial setting, the AI-powered large-scale language model of the large-scale language model server 19001 can be instructed with a set prompt word, including the large-scale language model's own name, the role it will play, and conversational characteristics. Additionally, as an initial setting, it can also understand the user's name. Thus, the large-scale language model generates responses for the first and subsequent rounds while maintaining that role. In this way, when the user hears the voice of role 19051 based on the first and subsequent responses, it feels as if role 19051 is the character's setting and personality described in the set prompt word. Furthermore, the large-scale language model server 19001 in this embodiment is configured to have a memory that stores the content of conversations until the end of a series of conversations, generating responses based on a series of stored user prompt words and their responses. Therefore, it is possible to achieve... Figure 2D The session shown.

[0121] Next, use Figure 2E This section describes an example of the operation of the role-playing conversation device (artificial intelligence response output device 10010) according to Embodiment 2 of the present invention. This can also be described as an example of the operation of a role-playing conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 19001. Specifically, Figure 2E This is an example of the natural language text of the main message of the prompt word sent from the AI ​​response output device 10010 to the large-scale language model server 19001, and the natural language text of the main message of the server response as its answer. The conversation between the character 19051 and the user 230 displayed on the AI ​​response output device 10010 is based on this.

[0122] Figure 2E Indicates in Figure 2D This is an example of a situation where, after a series of conversations and after the end of that series of conversations, user 230 speaks to character 19051 again and a new conversation begins. Figure 2E The diagram illustrates the sequential interaction between prompts and responses, from the first prompt and its response to the third prompt and its response.

[0123] Here, "end" in "the continuation of a series of sessions" refers to the process by which the large-scale language model server 19001 clears the memory of the sessions stored during the continuation of the series of sessions, under certain conditions. One example of these conditions is when the AI ​​response output device 10010 indicates the "end" of the series of sessions to the large-scale language model server 19001 via a prompt word. Another example is when a predetermined time has elapsed since the AI ​​response output device 10010 stopped sending prompt words to the large-scale language model server 19001 for that series of sessions (timeout). Furthermore, it is also possible that when the above-mentioned prompt word and response interaction occurs based on authentication processing in the connection between the AI ​​response output device 10010 and the large-scale language model server 19001, the authentication processing fails due to communication interruption, power-off of the AI ​​response output device 10010, or other reasons.

[0124] Additionally, when the "continuation of a series of sessions" "ends," the large-scale language model server 19001 clears the session memory maintained during the continuation of the series of sessions from the large-scale language model server 19001. Thus, Figure 2E The session shown is in Figure 2D The series of sessions shown are followed by the server response corresponding to the user prompt, but the server response is... Figure 2D The setup prompts shown include answers to questions about the character's name, the role to be played, conversational characteristics, and the user's name—all elements of a state where the user has no memory of the settings in the large-scale language model. Similarly, Figure 2E The conversation shown is about Figure 2D The series of conversations shown are responses to content from a state where there is absolutely no memory. That is, because... Figure 2D The "series of consecutive sessions" shown has "ended". Figure 2E The session begins from the state after the large-scale language model of the AI ​​server 19001 is initialized.

[0125] This causes user 230 to feel as if character 19051 has lost their memories of themselves or has become someone else. User 230 will feel strongly uncomfortable with the character's responses and experience feelings of loneliness and disappointment. Under such circumstances, there is a problem that the settings and memories of character 19051, such as their name, identity, conversational characteristics, and personality, displayed on the AI ​​response output device 10010 cannot be guaranteed to be consistent.

[0126] Next, use Figure 2FThis section describes an example of the operation of the role-playing conversation device (artificial intelligence response output device 10010) according to Embodiment 2 of the present invention. This can also be described as an example of the operation of a role-playing conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 19001. Specifically, Figure 2F This is an example of the natural language text of the main message of the prompt word sent from the AI ​​response output device 10010 to the large-scale language model server 19001, and the natural language text of the main message of the server response as its answer. The conversation between the character 19051 and the user 230 displayed on the AI ​​response output device 10010 is based on this.

[0127] Figure 2F Indicates in Figure 2D This is an example of a situation where, after a series of conversations and the end of that series of conversations, user 230 speaks to character 19051 again, thus starting a new conversation. (The last sentence appears to be incomplete and possibly contains errors.) Figure 2E The treatments are different. Figure 2F In the processing, when a new session begins, the AI ​​response output device 10010 sends a predefined prompt as the initial prompt to the large-scale language model server 19001. This predefined prompt stores information related to... Figure 2D The initial setup prompt is the same natural language text as the setup prompt. This can be expressed as reset text. Following this setup prompt is natural language text explaining past conversation history. This can be expressed as conversation history text. Regarding past conversation history, you can... Figure 2D During the duration of a series of conversations as described above, the AI-generated response output device 10010 records the history of the conversations as natural language text information in the storage unit 1170, associating it with the date and time information of the conversations. In the case of conversations with different dates, the history of the conversations can be accumulated by associating each conversation with its date and time information separately. When generating a... Figure 2F When setting the initial prompt for a future session, the natural language text information of the session and the date and time of the session recorded in the storage unit 1170 can be read to generate the setting prompt.

[0128] Furthermore, when generating the specified prompts using natural language text information from past conversation history, the format can be determined to some extent because the data is sent to a large-scale language model. Figure 2FAs shown, it is possible to prepare prefixes or suffixes in natural language such as "I said these things on ○ month ○ day." or "You said these things on ○ month ○ day." and process them by integrating them with the natural language text information of the recorded conversation to generate the text for the setting prompt. In addition, the date and time information of the conversation read from the storage unit 1170 can be integrated with the aforementioned "○ month ○ day" part and used as part of the text for the setting prompt.

[0129] Even if, after a series of conversations and after the end of that series of conversations, user 230 speaks to character 19051 again to start a new conversation, the above explanation will apply. Figure 2F By optimizing the generation and transmission of prompts, subsequent user responses will reflect the character's identity, name, conversational traits, personality, and / or other conversational characteristics from previous conversations, along with the conversation history. Therefore, from the user's perspective, the consistency between the character's identity, name, conversational traits, or personality from previous conversations and their memory is better ensured, making this a more preferable approach.

[0130] Next, use Figure 2G This section describes an example of the operation of the role-playing conversation device (artificial intelligence response output device 10010) according to Embodiment 2 of the present invention. This can also be described as an example of the operation of a role-playing conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 19001. Specifically, Figure 2G This is an example of the natural language text of the main message of the prompt word sent from the AI ​​response output device 10010 to the large-scale language model server 19001, and the natural language text of the main message of the server response as its answer. The conversation between the character 19051 and the user 230 displayed on the AI ​​response output device 10010 is based on this.

[0131] Figure 2G express Figure 2F The example shown is a series of conversations following the initial setting prompt, from the first user prompt and its response to the third user prompt and its response. Figure 2G The image shows a sequential interaction between prompts and responses. The content of the prompts is set because of... Figure 2F Since the information shown is the same, repeated entries are omitted.

[0132] like Figure 2F The table shows the natural language text of the server response, using... Figure 2FThe setting prompts shown indicate that the server response of the large-scale language model artificial intelligence of the large-scale language model server 19001 will be able to reflect the character's identity, name, conversation characteristics, or personality settings and conversation history from previous conversations. Therefore, from the user's perspective, the consistency between the user's understanding of the character's identity, name, conversation characteristics, or personality settings from previous conversations and their memory is better ensured, making it even better. Furthermore, from the user's perspective, the character can be perceived as the same person, which can also be referred to as class consistency of the character from the user's perspective.

[0133] In addition, from the user's perspective, being able to share memories with the character can lead to a more enjoyable character conversation experience.

[0134] Next, use Figure 2H This section describes an example of the operation of the role-playing conversation device (artificial intelligence response output device 10010) according to Embodiment 2 of the present invention. This can also be described as an example of the operation of a role-playing conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 19001. Specifically, Figure 2H This describes an example of switching between multiple candidate characters to display on the display unit 10011 of the AI ​​response output device 10010. The character action program executed by the control unit 1110 of the AI ​​response output device 10010 can switch the displayed character based on operation input input to the operation input unit 1107 or operations detected by the touch operation input sensor of the display unit 10011.

[0135] Figure 2H In the examples, except Figures 2A to 2G In addition to character 19051 (named "Koto") used in the description, characters 19052 (named "Tom") and 19053 (named "Necco") are also shown. Characters 19051 (named "Koto") and 19052 (named "Tom") are human characters, and character 19053 (named "Necco") is a cat character. Regarding the display switching of the characters displayed on display unit 10011, the display unit 10011 can switch the display of images generated by rendering characters in different virtual 3D spaces for each character.

[0136] Furthermore, when the character action program executed by the control unit 1110 switches the display of the character shown on the display unit 10011, it is preferable to also change the synthesized voice (synthetic speech) used in the "speech" of each character. For this purpose, the data of the synthesized voice with the timbre associated with each character can be saved in the storage unit 1170 in advance, and the synthesized voice change processing can also be performed when the character display is switched.

[0137] in addition, Figure 2H The example is that user 230 can have a conversation with any role. Figure 2H The AI-powered response output device 10010 sets different identities, names, conversational characteristics, or personalities for each of these characters. Furthermore, the memories of each character based on their conversation history are managed as separate data for each character.

[0138] Therefore, the artificial intelligence response output device 10010 is constructed in the storage unit 1170. Figure 2I The database shown is used to manage character settings and character session history.

[0139] Next, use Figure 2I This section describes an example of the operation of the role-playing conversation device (artificial intelligence response output device 10010) according to Embodiment 2 of the present invention. This can also be described as an example of the operation of a role-playing conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 19001. Specifically, Figure 2I This is an explanatory diagram of a database 19200 used to manage the settings of multiple characters and their conversation history displayed on the display unit 10011 of the artificial intelligence response output device 10010.

[0140] The role action program executed by the control unit 1110 of the AI ​​response output device 10010, for example, constructs the database 19200 in the storage unit 1170. The role ID is an identifier that identifies each of the multiple roles that can be displayed in the AI ​​response output device 10010; it can be a natural number or a letter. The name is data representing the individual names of the multiple roles that can be displayed in the AI ​​response output device 10010.

[0141] The initial setting prompt is natural language text information describing the character identity, name, conversational characteristics, or personality settings of the multiple characters that can be displayed in the AI ​​response output device 10010. This initial setting prompt is the natural language text information of the master data of the setting prompt sent from the AI ​​response output device 10010 to the large-scale language model server 19001. Therefore, it is preferable to use content that can be directly read by the AI ​​large-scale language model of the large-scale language model server 19001.

[0142] Conversation history 1, 2, etc., are records of conversations between each role and the user, and are recorded separately for each role. This conversation history is contained in the natural language text information of the master data of the set prompt words sent from the artificial intelligence response output device 10010 to the large-scale language model server 19001. Therefore, it is preferable to use the recorded content that can be directly read by the artificial intelligence large-scale language model of the large-scale language model server 19001.

[0143] The character action program executed by the control unit 1110 of the AI ​​response output device 10010, when switching the character displayed on the display unit 10011 of the AI ​​response output device 10010, uses... Figure 2I The database 19200, in a manner corresponding to the character displayed on the display unit 10011 of the AI ​​response output device 10010, selects and switches the initial setting prompt words and conversation history used in the natural language text information of the master data of the setting prompt words sent from the AI ​​response output device 10010 to the large-scale language model server 19001. Additionally, the character action program records the history of the conversation each time the user 230 engages in a conversation with the character. Figure 2I The area in database 19200 that corresponds to the role displayed on display unit 10011.

[0144] The character action program executed by the control unit 1110 of the AI ​​response output device 10010 utilizes the database 19200 in this way. Although the conversation between the user 230 and the character is established by obtaining the character's speech using the responses from the same AI large-scale language model based on the same large-scale language model server 19001, from the user's perspective, it is perceptible that the uniqueness of each character's personality and other settings is maintained, and each character retains the memory of their different conversations. From the user's perspective, it is more preferable because it is recognized that the consistency of the character's identity, name, conversational characteristics, or personality settings and memories from previous conversations is better ensured for each character. This can be expressed as ensuring class consistency of the characters from the user's perspective for each character.

[0145] Therefore, even if the AI ​​response output device 10010 is configured to switch between multiple character candidates to be displayed on the display unit 10011, in this case, through the actions of using the database 19200 described above, the user will feel less unnatural in conversations with each character, and can share memories with multiple characters separately, resulting in a more pleasant character conversation experience.

[0146] Furthermore, by preventing users from editing the initial setup prompts for multiple characters, the settings for each character's identity, name, conversational traits, or personality can be maintained in a state close to the intent of the provider of the AI ​​response output device 10010 or the content creator of the character. Conversely, users can also edit the initial setup prompts for characters by inputting through the operation input unit 1107, etc. In this case, the character's identity, name, conversational traits, or personality can be set according to the user's preferences, allowing the user to converse with a uniquely configured character. In this case, the type of the character's 3D model or its rendered image, as well as the type of the character's synthesized voice, can also be changed accordingly.

[0147] Next, use Figure 2J This section describes an example of the operation of the role-playing device (artificial intelligence response output device 10010) according to Embodiment 2 of the present invention. This can also be described as an example of the operation of a role-playing system including the artificial intelligence response output device 10010 and the large-scale language model server 19001. Specifically, a method for providing role-playing services at a lower cost will be described, wherein the role-playing service is implemented through a role-playing device based on the artificial intelligence response output device 10010, and / or a role-playing system based on the artificial intelligence response output device 10010 and the large-scale language model server 19001.

[0148] like Figure 2B As explained, for large-scale language models, limiting AI training to a specific purpose is extremely inefficient in terms of resources. Therefore, a more efficient approach is to generate models through large-scale training as foundational models applicable to various uses, and then access them via APIs (Application Programming Interfaces) through various terminals. In this way, providers of large-scale language models typically recoup the training costs from end-users through API fees. In this case, natural language models often charge API fees based on the number of language units (called tokens) processed, derived from the text's structure.

[0149] Therefore, in the AI ​​response output device 10010 of Embodiment 2 of the present invention, by reducing the number of lexical units in the natural language text information transmitted using API between the AI ​​response output device 10010 and the large-scale language model server 19001, it is possible to provide users with role-based conversation services implemented through the role-based conversation device based on the AI ​​response output device 10010 and / or the role-based conversation system based on the AI ​​response output device 10010 and the large-scale language model server 19001 at a lower cost.

[0150] For example, by adopting such Figure 2J The processing and structure shown in Examples 1 to 3 can technically reduce the number of lexical units in natural language text information transmitted via API between the AI ​​response output device 10010 and the large-scale language model server 19001.

[0151] Example 1 illustrates a method for reducing the number of lexical terms in the conversation history text saved and transmitted in the API's setting prompts. This method uses document summarization to shorten the conversation history text and reduce the number of lexical terms. For instance, the natural language of the conversation history with the role recorded in storage unit 1170 is summarized and then re-recorded. While document summarization can be performed at the start of the next conversation, it is more feasible to perform it at the end of a "series of conversations."

[0152] Alternatively, article summarization can be delegated to the large-scale language model itself on the large-scale language model server 19001. However, the savings in lexical count are relatively small in this case. Therefore, for example, if natural language article summarization is provided in the second server 19002 via API at a lower cost than the large-scale language model on the large-scale language model server 19001, the article summarization can be delegated to the second server 19002 via API, and the article summary of the conversation history can be stored in the set prompt words of the large-scale language model server 19001 and transmitted.

[0153] Alternatively, if only article summarization is required, it can be performed on the terminal side. The control unit 1110 can execute the document summarization program loaded into the memory 1109 of the artificial intelligence response output device 10010 to perform article summarization. In this case, the saving effect on the number of lexical units is relatively high. In addition, even if the conversation history becomes longer, as long as the upper limit of the number of summarized characters is specified in the article summarization process, the upper limit of the article length of the conversation history can be determined, so the upper limit of the lexical unit can be specified, thus saving lexical units.

[0154] In addition, the initial text information of a character, such as identity, name, conversation characteristics, or personality, does not increase like the conversation history. Therefore, maintaining the text information of the character's initial setting prompts and reducing the number of lexical items in the text information of the conversation history is efficient and preferred.

[0155] The processing described in Example 1 can be controlled by the role action program executed by the control unit 1110 to control each part.

[0156] Example 2 is another example of reducing the number of lexical units in the conversation history text stored and transmitted in the API's setting prompts. For example, in the conversation history with the character recorded in storage unit 1170, the number of lexical units is reduced by clearing from older history records. If an upper limit for the number of characters in the conversation history is specified, the upper limit for the length of the conversation history text can be determined, so an upper limit for lexical units can be specified, thus saving lexical units. Alternatively, a method can be used to specify a period for the conversation history and delete conversation history that exceeds that period. In this case, lexical units can also be saved. Furthermore, in Example 2, the character's initial setting text information, such as the character's identity, name, conversation characteristics, or personality, does not increase like the conversation history. Therefore, maintaining the recording of the character's initial setting prompt text information and reducing the number of lexical units in the conversation history text information is efficient and preferred.

[0157] The processing described in Example 2 can be controlled by the role action program executed by the control unit 1110 to control each part.

[0158] Example 3 is a method to reduce the number of lexical units by decreasing the frequency of sending setting prompts via API. Specifically, after the device is powered on and the displayed character is switched, even after the image settings and synthesized voice settings of the displayed character are completed, setting prompts are not sent in advance. Instead, setting prompts are only sent to the large-scale language model server 19001 when the control unit 1110 determines that the natural language text information contained in the user's voice collected by the microphone 1139 is text information that requires the use of an artificial intelligence large-scale language model. This reduces the frequency of sending setting prompts to the large-scale language model server 19001 and thus reduces the number of lexical units.

[0159] Specifically, for example, after the device is powered on and a character switching operation input is received, the display processing of the display unit 10011 is controlled by the character action program executed by the control unit 1110, such as... Figure 2H The character 19051 (whose name is "Koto") is displayed on the display unit 10011. At this time, if, for example, a synthesized voice for the appearance of character 19051 is saved and prepared in the storage unit 1170 or similar, the synthesized voice for the appearance, such as "Good morning. I am Koto.", "Hello. I am Koto.", "Good evening. I am Koto.", can be output from the sound output unit 1140, i.e., the speaker. At this time, the image of character 19051 has been set as the image of the character displayed on the display unit 10011, and the synthesized voice corresponding to character 19051 has been set as the synthesized voice output from the sound output unit 1140, i.e., the speaker.

[0160] Here, for the inference processing of the artificial intelligence large-scale language model in the already described large-scale language model server 19001, the longer the prompt word, the more time it takes. In particular, when the setting prompt word contains textual information about past conversation history, the number of lexical units in the prompt word increases, so the inference processing time will be extended. The setting prompt word itself and its answer are not output to the user 230. Starting from the user's answer to the setting prompt word, the synthesized voice of the character's "speech" is output from the sound output unit 1140, i.e., the speaker. Thus, if the setting prompt word is sent from the artificial intelligence answer output device 10010 to the large-scale language model server 19001 in advance, and the inference processing of the large-scale language model for the setting prompt word is completed in advance, the output response of the synthesized voice of the character 19051's "speech" after the user 230 speaks to the character 19051 is faster, so it seems superior at first glance.

[0161] However, even if the setting prompt is sent to the large-scale language model server 19001 before the user 230 speaks, and the large-scale language model performs inference processing on the setting prompt in advance, there may still be situations where, for example, the user 230 operates the touch operation input sensor of the operation input unit 1107 or the display unit 10011 to turn off the power of the artificial intelligence response output device 10010, or the user 230 operates the touch operation input sensor of the operation input unit 1107 or the display unit 10011 to switch the displayed role from role 19051 to another role. In these cases, the number of lexical units that are sent to the large-scale language model server 19001 in advance and processed by the large-scale language model's inference processing will become a waste of usage fees. This would hinder the provision of role conversation services to users at a lower cost through a role conversation device based on the artificial intelligence response output device 10010, and / or a role conversation system based on the artificial intelligence response output device 10010 and the large-scale language model server 19001.

[0162] Therefore, the AI ​​response output device 10010 preferably maintains a state of not sending setting prompt words to the large-scale language model server 19001 after the device is powered on and after the operation input for displaying character switching is received, even after the character action program executed by the control unit 1110 sets the image of character 19051 as the image of character displayed on the display unit 10011 and sets the synthesized voice corresponding to character 19051 as the synthesized voice output from the sound output unit 1140, i.e., the speaker, until the moment when the user 230 speaks to character 19051 is recognized.

[0163] Here, the moment when user 230 speaks to character 19051 is identified, for example, by detecting... Figure 2B The timing of triggering keywords as described in the text, or the timing of text extraction of the utterances spoken by user 230, etc. By adopting this approach, the number of lexical units processed that incurs wasted on usage fees can be reduced, and role-based conversation services can be provided to users at a lower cost through a role-based conversation device based on the AI ​​response output device 10010, and / or a role-based conversation system based on the AI ​​response output device 10010 and a large-scale language model server 19001.

[0164] Furthermore, even after the moment when user 230 speaks to character 19051 has passed, for example, if the text information extracted from user 230's voice collected from microphone 1139 corresponds to a preset keyword that does not require inference processing by a large-scale language model, it is preferable to maintain the state of not sending a preset prompt word to the large-scale language model server 19001. Specifically, examples of preset keywords include "jump around" or "dance," which are keywords that user 230 requests character 19051 to respond to (e.g., animation of character 19051's movement or the emission of a synthesized sound). In this case, the character action program executed by control unit 1110 can read the action data, animation images, and / or synthesized sound data corresponding to character 19051 stored in storage unit, and use this data to perform image generation processing on display unit 10011 and output processing of synthesized sound from sound output unit 1140, i.e., speaker.

[0165] Such processing does not necessarily require inference processing of the large-scale language model on the large-scale language model server 19001. If, after this processing, the user 230 turns off the power to the AI ​​response output device 10010 by operating the touch input sensor of the operation input unit 1107 or the display unit 10011, and / or, for example, the user 230 switches the displayed role from role 19051 to another role by operating the touch input sensor of the operation input unit 1107 or the display unit 10011, if the setting prompt words have been sent to the large-scale language model server 19001 in advance and processed by inference processing of the large-scale language model, then the number of lexical units processed becomes a waste of usage fees.

[0166] Therefore, preferably, even after the moment when user 230 is identified speaking to character 19051, the state of not sending the preset prompt word to the large-scale language model server 19001 is maintained until, for example, when it is necessary to determine whether the text information extracted from the voice of user 230 collected from microphone 1139 corresponds to the text information of a preset keyword that does not require large-scale language model inference processing. Preferably, the preset prompt word is sent to the large-scale language model server 19001 and large-scale language model inference processing is performed only when it is determined through this determination that large-scale language model inference processing is required.

[0167] Furthermore, the processing described in Example 3 can be controlled by the role action program executed by the control unit 1110.

[0168] Based on the above explanation Figure 2J The methods for reducing (saving) the number of lexical units processed by large-scale language models can provide users with role-based conversation services at a lower cost through a role-based conversation device based on an AI response output device 10010, and / or a role-based conversation system based on an AI response output device 10010 and a large-scale language model server 19001.

[0169] Next, use Figure 2K An example of the display of the role-playing device (artificial intelligence response output device 10010) of Embodiment 2 of the present invention will be described. Figure 2K The example shows that Figures 2A to 2J The figures illustrate an example of how the large-scale language model's response to user prompts is displayed on the display unit 10011 of a character conversation device (AI response output device 10010). Specifically, it's an example where the text 10063 from the large-scale language model is displayed alongside the image of the character 19051 on the display unit 10011. The text 10063 from the large-scale language model can be displayed as follows: Figure 2K The image is overlaid on the image of character 19051. Alternatively, the response from the large-scale language model, i.e., text 10063, can also be displayed together with the image of character 19051 without being overlaid on it.

[0170] Figure 2K For example, when user 230 operates the touch operation input sensor of operation input unit 1107 or display unit 10011 to adjust the volume of the sound output of the sound output unit 1140 of the role conversation device (artificial intelligence response output device 10010) to the minimum or set the sound output to OFF, user 230 cannot confirm the answer from the large-scale language model by sound.

[0171] Therefore, in this case, the control unit 1110 can perform control to begin executing the response from the large-scale language model, i.e., text 10063, such as... Figure 2K The display mode shown is where the image of character 19051 is displayed together. In this way, even when sound output is to be suppressed, user 230 can better utilize the character conversation device (AI response output device 10010). Alternatively, the user 230 can manually switch the display mode, which displays the response (text 10063) from the large-scale language model along with the image of character 19051, on / off via the operation input unit 1107 or the touch operation input sensor of the display unit 10011.

[0172] Next, regarding Figure 2H and Figure 2I The example described herein is an answer-type statement database (answer-type statement DB) in a role-playing conversation device (artificial intelligence answer output device 10010) capable of displaying multiple characters, using... Figure 2L Please provide an explanation. Figure 2L In the example, the condition number and condition content are the same as Figure 1C Same. For these conditions, Figure 2L In the example, individual pre-defined response statements were set for each of the multiple roles. For example, for... Figure 2H and Figure 2I The document describes three roles: Koto (role 1), Tom (role 2), and Necco (role 3), each storing the corresponding definitive answer statements for each condition. The output control of these definitive answer statements is related to... Figure 1C Since they are the same, repeated descriptions are omitted.

[0173] Figure 2L In the example, the control unit 1110 can select the corresponding answer-defined statement from the answer-defined statement database (answer-defined statement DB) based on the role displayed in the role conversation device (artificial intelligence answer output device 10010) and the current conditions, and use it for output control as the answer made by the role. For example, in Figure 2L In the example of the answer-prefix statement database (answer-prefix statement DB), even under the same conditions, the answer-prefix statements change to expressions or content corresponding to the character's personality. Therefore, the character dialogue device (AI answer output device 10010) can provide the user with a dialogue corresponding to the displayed character's personality. The user can perceive that each character possesses a more consistent personality. Thus, a character dialogue device (AI answer output device 10010) that makes multiple characters feel more real can be realized.

[0174] In addition, the above explanation Figure 2L The database of definitive answer statements (answer definitive statement DB) is stored in the storage unit 1170, and the control unit 1110 of the artificial intelligence answer output device 10010 can use it to generate answers. However, Figure 2L The answer specification statement database (answer specification statement DB) shown can also be set on the side of the large-scale language model server 19001. In this case, the control unit of the large-scale language model server 19001 can generate an answer using the answer specification statement database (answer specification statement DB). In the control unit of the large-scale language model server 19001, the answer generated using the answer specification statement database (answer specification statement DB) can be sent to the artificial intelligence answer output device 10010 instead of the answer generated by the large-scale language model stored in each server. In this way, even if the artificial intelligence answer output device 10010 does not have an answer specification statement database (answer specification statement DB), it is possible to generate an answer using the answer specification statement database (answer specification statement DB).

[0175] The role-conversation device and system of Embodiment 2 described above can further reduce the unnaturalness experienced by the user in conversations with the character displayed on the AI ​​response output device 10010. Furthermore, the role-conversation device and system of Embodiment 2 can provide role-conversation services to users at a lower cost.

[0176] Furthermore, in the above description of Embodiment 2, an example was described using a large-scale language model possessed by the large-scale language model server 19001 as a large-scale language model. In contrast, the role-based conversation device (artificial intelligence response output device 10010) could also be configured to possess... Figure 1B The local LLM processing unit 10028 shown replaces the large-scale language model of the large-scale language model server 19001 with the large-scale language model of the local LLM processing unit 10028. In this case, as described above in Embodiment 2, the large-scale language model of the large-scale language model server 19001 can be changed to the large-scale language model of the local LLM processing unit 10028 of the role conversation device (artificial intelligence response output device 10010).

[0177] In this case, the unnaturalness experienced by the user during their conversation with the character displayed on the AI ​​response output device 10010 can be further reduced. Furthermore, when using a large-scale language model in the local LLM processing unit 10028 instead of the large-scale language model in the large-scale language model server 19001, although there is little need to consider usage fees corresponding to the number of processed lexical units, even with the large-scale language model in the local LLM processing unit 10028, the power and other resource consumption for inference can be reduced by decreasing the number of processed lexical units. In this case, a character conversation service with lower power consumption can be provided to the user.

[0178] Furthermore, in the above description of Embodiment 2, an example was described where the user's and character's conversation history was stored in the storage unit 1170 of the character conversation device (AI response output device 10010). In contrast, the user's and character's conversation history can also be recorded and stored in a second server 19002 or other cloud server connected to the Internet 19000. In this case, when the user and character begin a new conversation, the character conversation device (AI response output device 10010) can communicate with the second server 19002 or other cloud server to obtain (download) the past conversation history between the character and the user, and store it in the storage unit 1170 or memory 1109 of the character conversation device (AI response output device 10010) for generating prompt words for the large-scale language model. The specific method for generating prompt words for the large-scale language model using past conversation history is the same as that described in the figures of Embodiment 2, so repeated descriptions are omitted.

[0179] Furthermore, the character conversation device (AI response output device 10010) can send (upload) the character's conversation history up to the specified time, such as each time the user engages in a conversation with the character, or at the end of the conversation. That is, the character conversation device (AI response output device 10010) can upload the user's conversation history to the second server 19002 or other cloud server at specified times. When the user begins a conversation with the character, the character conversation device (AI response output device 10010) downloads the latest conversation history from the second server 19002 or other cloud server to generate prompts for the large-scale language model. Using this method, even if the character conversation device (AI response output device 10010) used by the user the previous day is a different device than the one the user is currently using, the same character can still be displayed. When the same character engages in multiple conversations with the user at different times across different devices, it allows for a perception of inheriting the character's memory from the previous conversation, which is more preferable for the user.

[0180] The above-described process, in which the character conversation device (AI response output device 10010) uploads or downloads the user's conversation history with the character from the second server 19002 or other cloud servers and perceptually inherits the character's memory, is used... Figure 2H and Figure 2I The database 19200, which contains session history for multiple roles as described in the description, is also valid. That is, it can be configured to upload or download to a second server 19002 or other cloud servers. Figure 2I The database 19200 described herein can achieve a conversation that allows the user to intuitively inherit the memories of each character from the previous conversation, not only for one character, but also for multiple characters, when the user has multiple conversations with each character's device at different times and from different individuals. This is even better for the user.

[0181] <Example 3>

[0182] Next, Embodiment 3 of the present invention improves upon the role-conversation device (artificial intelligence response output device 10010) and role-conversation system illustrated in the figures of Embodiment 2. The differences from Embodiment 2 are described in this embodiment, and repeated descriptions of structures identical to those in these embodiments are omitted.

[0183] Similar to Example 2, the role in Example 3 can provide users with artificial intelligence, i.e., large-scale language model services, to assist them. Therefore, this role can act as an artificial intelligence (AI) assistant for the user. In this case, the role conversation device and role conversation system in this embodiment can also be referred to as an AI assistant conversation device, an AI assistant display device, an AI assistant response output device, an AI assistant conversation system, an AI assistant display system, and an AI assistant response output system.

[0184] use Figure 3A An example of the role-playing device and role-playing system of Embodiment 3 of the present invention will be described. In the role-playing system of Embodiment 3, instead of Figure 2A The large-scale language model server 19001 has a large-scale language model server 20001 and is connected to the Internet 19000.

[0185] Here, the large-scale language model server 20001 is a server equipped with large-scale language model artificial intelligence. However, it is a multimodal large-scale language model artificial intelligence that can process not only natural language text information but also other types of information besides natural language text information, in addition to the natural language text information that the large-scale language model server 19001 can process.

[0186] Furthermore, it is assumed that the role-conversation device, i.e., the artificial intelligence response output device 10010, has the same structure as the role-conversation device (artificial intelligence response output device 10010) of Embodiment 2 for the description.

[0187] Similarly, in Example 3, the role-based conversation device, i.e., the AI ​​response output device 10010, can use an API to communicate with the large-scale language model of the large-scale language model server 20001 via the Internet 19000.

[0188] In the role-based session system of Example 3, there is a mobile information processing terminal 20010 used by user 230. The mobile information processing terminal 20010 is a so-called smartphone or tablet information processing terminal.

[0189] Here, use Figure 3B An example of a mobile information processing terminal 20010 will be described. The mobile information processing terminal 20010 includes a touch operation input panel (display panel) 20011, a control unit 20012, an external power input interface 20013, a power supply 20014, a secondary battery 20015, a storage unit 20016, an image control unit 20017, a posture sensor 20018, a communication unit 20020, a sound output unit 20021, a microphone 20022, an image signal input unit 20023, a sound signal input unit 20024, and a camera unit 20025.

[0190] Display panel 20011 is equipped with a touch input sensor, which can accept touch input from user 230's finger. Display panel 20011 uses a liquid crystal panel or an organic EL panel for display, and can display images. Display panel 20011 can also be referred to as a display unit.

[0191] The communication unit 20020 can be configured using a Wi-Fi communication interface, a Bluetooth communication interface, or mobile communication interfaces such as 4G and 5G. The communication unit 20020 of the mobile information processing terminal 20010 can use these communication methods to communicate with the communication unit 1132 of the role-interaction device (artificial intelligence response output device 10010). The mobile information processing terminal 20010 includes a control unit such as a CPU and a memory, which controls the display panel 20011 and the communication unit 20020. Furthermore, using any of the communication methods of the communication unit 20020, it can communicate with a communication device 19011 connected to the Internet 19000. Therefore, the mobile information processing terminal 20010 can communicate with various servers connected to the Internet 19000.

[0192] Power supply 20014 converts AC current input from the outside via external power input interface 20013 into DC current, supplying the required DC current to each part of mobile information processing terminal 20010. Secondary battery 20015 stores the power (electrical power) supplied by power supply 20014. Furthermore, when no external power is supplied via external power input interface 20013, secondary battery 20015 supplies power to the parts requiring electricity.

[0193] The video signal input unit 20023 connects to an external video output device to input video data. The video signal input unit 20023 can be various digital video input interfaces. For example, it can be a video input interface conforming to the HDMI (High-Definition Multimedia Interface) standard, a video input interface conforming to the DVI (Digital Visual Interface) standard, or a video input interface conforming to the DisplayPort standard. Alternatively, it can be an analog video input interface such as analog RGB or component video. The video signal input unit 20023 can also be various USB interfaces, etc.

[0194] The audio signal input unit 20024 connects to an external audio output device to input audio data. The audio signal input unit 20024 can be configured with an HDMI standard audio input interface, an optical digital terminal interface, or a coaxial digital terminal interface, etc. The audio signal input unit 20024 can also be various USB interfaces, etc. When using an HDMI standard interface, the video signal input unit 20023 and the audio signal input unit 20024 can be configured as an interface integrating terminals and cables.

[0195] The audio output unit 20021 can output audio based on the audio data input to the audio signal input unit 20024. The audio output unit 20021 can also output audio based on the audio data stored in the storage unit 20016. The audio output unit 20021 can be configured as a speaker. Additionally, the audio output unit 20021 can also output built-in operation tones and error warning tones. Alternatively, the audio output unit 20021 can be configured to output digital signals to external devices, as specified in the HDMI standard's Audio Return Channel function.

[0196] Microphone 20022 is a microphone that collects ambient sound from the mobile information processing terminal 20010 and converts it into a signal to generate an audio signal. Alternatively, the microphone may be configured to record the voice of a user or other person (speech), and the generated audio signal may be processed by the control unit 20012 (described later) for voice recognition processing (speech recognition processing) to obtain text information from the audio signal.

[0197] The camera unit 20025 is a camera equipped with an image sensor. The camera can be installed on the front side of the display panel 20011 of the mobile information processing terminal 20010, or on the back side of the display panel 20011. Both a front-facing camera and a rear-facing camera can be installed. In this embodiment, the camera unit 20025 will be described with both a front-facing camera and a rear-facing camera.

[0198] Storage unit 20016 is a storage device that records various types of data and information, such as image data, audio data, etc. Storage unit 20016 can be composed of magnetic recording media recording devices such as hard disk drives (HDDs) or semiconductor memory devices such as solid-state drives (SSDs). In storage unit 20016, for example, various types of data and information, such as image data, audio data, etc., can be pre-recorded at the factory. Additionally, storage unit 20016 can also record various types of data and information, such as image data, audio data, etc., obtained from external devices and external servers via communication unit 20020. The image data and audio data recorded in storage unit 20016 are output to display panel 20011. The image data and audio data recorded in storage unit 20016 can also be output to external devices and / or external servers via communication unit 20020.

[0199] The image control unit 20017 performs various controls on the image signals input to the display panel 20011. The image control unit 20017 can also be called an image processing circuit, and may be constructed from hardware such as an ASIC, FPGA, or video processor. Alternatively, the image control unit 20017 can also be called an image processing unit or an image processing unit. For example, the image control unit 20017 controls image switching, switching which image signal (image data) from the image signal stored in the memory 20026 or the image signal input to the image signal input unit 20023 is input to the display panel 20011. Furthermore, the image control unit 20017 can also control image processing on the image signals input from the image signal input unit 20023 and the image signals stored in the memory 20026. Image processing includes, for example, scaling operations such as enlarging, reducing, and distorting the image; brightness adjustment operations such as changing the brightness; contrast adjustment operations such as changing the contrast curve of the image; and Retinex processing such as decomposing the image into light components and changing the weights of each component.

[0200] The attitude sensor 20018 is a sensor composed of a gravity sensor, an acceleration sensor, or a combination thereof, capable of detecting the attitude of the mobile information processing terminal 20010. The control unit 20012 can control the actions of connected components based on the attitude detection results from the attitude sensor 20018.

[0201] Non-volatile memory 20027 stores various data used in mobile information processing terminal 20010. The data stored in non-volatile memory 20027 includes, for example, various operational data, display icons, data on objects for user operation, layout information, etc., displayed on the display panel 20011 of mobile information processing terminal 20010. Memory 20026 stores image data displayed on display panel 20011 and / or device control data, etc. Control unit 20012 can read various software from storage unit 20016, load it, and store it in memory 20026.

[0202] The control unit 20012 controls the operation of each connected unit. In addition, the control unit 20012 can cooperate with the program stored in the memory 20026 to perform calculations based on information obtained from each unit in the mobile information processing terminal 20010.

[0203] Next, use Figure 3C Here is an example of the operation of the role-conversation device (artificial intelligence response output device 10010) according to Embodiment 3 of the present invention. This can also be described as an example of the operation of a role-conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 20001. Similarly, in Embodiment 3, the role-conversation device (artificial intelligence response output device 10010) loads the role action program stored in the storage unit 1170, etc., into the memory 1109, and the control unit 1110 executes the role action program, thereby enabling the various processes described below.

[0204] In Embodiment 2, the action initiated by user 230 to the role-playing device (AI response output device 10010) is primarily a call made using user 230's voice. In the role-playing device (AI response output device 10010) of Embodiment 2, a series of actions are performed starting from the processing of user 230's voice using a microphone. In contrast, in the role-playing device (AI response output device 10010) of Embodiment 3, the series of actions described in Embodiment 2, starting from the processing of user 230's voice using a microphone, can also be executed. Furthermore, in the role-playing device (AI response output device 10010) of Embodiment 3, user 230 can... Figure 1B The user operation performed by the operation input unit 1107 initiates an action on the role-based conversation device (artificial intelligence response output device 10010). Here, as... Figure 1B Examples of operation input units 1107 include mice, keyboards, and touch panels.

[0205] Furthermore, in the role-playing conversation device (artificial intelligence response output device 10010) of Embodiment 3, through the ability to... Figure 1B The touch operation input sensor of the display unit 10011 detects the user's touch operation, and the user 230 can initiate its actions to the role conversation device (artificial intelligence response output device 10010).

[0206] In addition, user 230 can communicate with role-playing device (artificial intelligence response output device 10010) from mobile information processing terminal 20010 by operating mobile information processing terminal 20010, and can also input user 230's operation input to role-playing device (artificial intelligence response output device 10010).

[0207] Alternatively, the display panel 20011 of the mobile information processing terminal 20010 can also display a stored image, such as a QR code, containing the information the user wants to convey to the role-conversation device (artificial intelligence response output device 10010). This image is stored in the form of a QR code or similar information. Figure 1B The camera unit 1180 captures the display. The control unit 1110 of the role-conversation device (AI response output device 10010) can extract information from the stored image such as the QR code captured by the camera unit 1180 to obtain the information. Alternatively, the image that the user wants to convey to the role-conversation device (AI response output device 10010) can also be displayed on the display panel 20011 of the mobile information processing terminal 20010, provided that the role-conversation device (AI response output device 10010) has... Figure 1B The camera unit 1180 captures the display. The control unit 1110 of the character conversation device (artificial intelligence response output device 10010) can perform image recognition processing on the image captured by the camera unit 1180 and obtain the result of the image recognition processing.

[0208] Thus, in the role-conversation device (AI response output device 10010) of Embodiment 3, compared with the role-conversation device (AI response output device 10010) described in Embodiment 2, the user 230 can initiate more types of actions on the role-conversation device (AI response output device 10010). Therefore, the role-conversation device (AI response output device 10010) of Embodiment 3 can obtain the results of actions initiated by the user 230 other than the user's voice, and generate prompts to be sent to the large-scale language model server 20001 based on these results. This allows for better inclusion of information other than natural language text information extracted from the user's voice in the prompts sent to the large-scale language model server 20001. Information other than natural language text information extracted from the user's voice includes, for example, images, videos, and sounds.

[0209] Next, the role-based conversation device (AI response output device 10010) in this embodiment sends prompt words to the large-scale language model server 20001 using an API. Similarly, in this embodiment, the prompt words can be metadata containing information described by the following syntax: syntax using tags in Markup format of markup languages, syntax using defined symbols in Markdown format, or object syntax of defined scripts such as JSON. Similarly, in this embodiment, the types of prompt words include setting prompt words that store initial settings and other instructions, and user prompt words that reflect user instructions. The type identification information, indicating whether a prompt word is a setting prompt word or a user prompt word, can also be stored in a part other than the main message of the prompt word. In this case, the prompt word, as the main message, can contain natural language text information. Furthermore, in this embodiment, in addition to natural language text information, the main message of the prompt word can also contain non-natural language information sources such as images, videos, or sounds. The specific method for including non-natural language information sources in the prompt words will be described later.

[0210] The large-scale language model server 20001 in this embodiment has a multimodal large-scale language model, capable of processing natural language text information as well as non-natural language information sources. The large-scale language model server 20001 receives prompt words from a role-based conversation device (artificial intelligence response output device 10010). The multimodal large-scale language model performs reasoning based on the prompt words, generating a response containing natural language text information with the reasoning results. Here, the artificial intelligence of the large-scale language model server 20001 is a multimodal large-scale language model, so the response can include non-natural language information sources such as images, videos, or sounds in addition to natural language text information.

[0211] The role-conversation device (AI response output device 10010) receives a response from the large-scale language model server 20001 and extracts natural language text information and non-natural language information sources such as images, videos, or sounds from the response, which are stored as the main message. The role action program of the role-conversation device (AI response output device 10010) can generate a natural language voice (speech) as a response to the user based on the natural language text information extracted from the above response, using voice synthesis (speech synthesis) technology, and output it from the speaker, i.e., the sound output unit 1140, in a manner that sounds as if it were the voice of the character 19051 displayed on the screen.

[0212] Furthermore, the character action program of the character dialogue device (AI response output device 10010) can also display natural language text as a response to the user on the display screen of the character dialogue device (AI response output device 10010) based on the natural language text information extracted from the above-mentioned responses. At this time, the text can be displayed together with the character 19051, or it can be superimposed on the image of the character 19051, or it can be displayed in place of the image of the character 19051. These specific processes can be performed by the image control unit 1160.

[0213] Furthermore, the character action program of the character dialogue device (AI response output device 10010) can also display the image extracted from the non-natural language information source of the aforementioned response on the display screen of the character dialogue device (AI response output device 10010) to present it to the user. At this time, the image can be displayed together with the character 19051, superimposed on the image of the character 19051, or displayed in place of the image of the character 19051. These specific processes can be performed by the image control unit 1160.

[0214] Furthermore, the character action program of the character dialogue device (AI response output device 10010) can also display the video, based on the non-natural language information source extracted from the above-mentioned response, on the display screen of the character dialogue device (AI response output device 10010) to present it to the user. At this time, the video can be displayed together with the character 19051, or it can be superimposed on the image of the character 19051, or it can be displayed in place of the image of the character 19051. These specific processes can be performed by the image control unit 1160.

[0215] In addition, the character action program of the character dialogue device (artificial intelligence response output device 10010) can also generate sound based on the sound information of the non-natural language information source extracted from the above response, and output it from the speaker, i.e., the sound output unit 1140.

[0216] Based on the above explanation Figure 3C The role-based conversation device (AI response output device 10010), or a role-based conversation system including the role-based conversation device (AI response output device 10010) and a large-scale language model server 20001, does not require the large-scale language model itself, which requires massive data and computing resources for training, to be integrated into the role-based conversation device (AI response output device 10010). Furthermore, it can utilize the advanced natural language processing and non-natural language information processing capabilities of the multimodal large-scale language model via API. In addition to responses based on natural language text, it can also provide responses based on non-natural language information sources to user actions on the role, enabling better conversations.

[0217] Next, use Figure 3D This section describes an example of the operation of the role-based conversation device (artificial intelligence response output device 10010) according to Embodiment 3 of the present invention. This can also be described as an example of the operation of a role-based conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 20001. Specifically, Figure 3D This example illustrates a non-natural language information source, such as natural language text and images, of the main message containing prompts sent from a role-based conversation device (AI response output device 10010) to a large-scale language model server 20001, and an example of a non-natural language information source, such as natural language text and images, of the main message of the server's response. In this embodiment, the non-natural language information source can use images, videos, sounds, etc., but... Figure 3D The image is shown as an example of a non-natural language information source.

[0218] in addition, Figure 3D The diagram illustrates the sequential interaction between the prompt and the response, from the initial prompt, the first user prompt and its response, to the second user prompt and its response. Figure 3D The prompts and answers shown include those from Example 2. Figure 2D Non-natural language information sources 20061 and 20062 are not shown in the text. Figure 3D In the example, both non-natural language information source 20061 and non-natural language information source 20062 are images.

[0219] Here, Figure 3DFor simplicity, the example shown depicts a state where an image of the non-natural language information source 20061 is pasted into the prompt. However, there are various methods for transmitting or specifying data of the non-natural language information source 20061 in the prompt sent from the role-conversation device (artificial intelligence response output device 10010) to the large-scale language model server 20001. The role-conversation device (artificial intelligence response output device 10010) can use any of these methods or switch between them. An example of each method is described below.

[0220] A first method for transmitting or specifying non-natural language information source data in a prompt word is used, for example, in the following situation: the non-natural language information source to be specified is a non-natural language information source existing in a place such as a server connected to a network such as the Internet. Specifically, the first method uses information such as tags and symbols within the prompt word, along with location information (such as a URL) and filename from the Internet or other network, to specify the non-natural language information source file existing on the Internet or other network.

[0221] For example, image tags can be specified using markup languages, etc. The **** section describes the location and filename information of the image file, thus specifying images located on the internet or other networks. Alternatively, tags in markup languages ​​can be used to specify video. <video src="****">The **** section describes the location and filename information of the video file, thus specifying videos located on the internet or other networks. Additionally, tags from markup languages ​​can be used to specify audio. <audio src="****">The prompt word specifies the location and filename of the audio file, indicating that the audio exists on the internet or other networks. Alternatively, if using JSON syntax, you can specify an image file by preparing keys such as `img_src` and specifying its location and filename in the value. Separate key-value pairs can also be prepared for video and audio files. This specific example is just one illustration; other unique formats can also be used. In either case, simply store the location and filename information of the source file for the non-natural language information in the prompt word.

[0222] In the first method, where the location and filename information of the specified non-natural language information source file are stored in the prompt word, it is not necessary to store the non-natural language information source file data itself in the prompt word. Therefore, the amount of data in the prompt word can be reduced. In the first method, the large-scale language model server 20001 that receives the prompt word specifying the non-natural language information source data can use the location and filename information of the non-natural language information source file stored in the prompt word to obtain the non-natural language information source file located in a place such as a server connected to a network such as the Internet.

[0223] This section explains how to input location information and filename information when the character conversation device (artificial intelligence response output device 10010) uses a first method to specify non-natural language information source data in the prompt words. Figure 3C As explained earlier, in this embodiment, the types of actions that user 230 can initiate with the role-based conversation device (AI response output device 10010) are increased compared to embodiment 2, in addition to user 230's voice. Therefore, for example, user 230 can initiate actions via... Figure 1B The operation input unit 1107 performs user operations (e.g., mouse, keyboard, touch panel) and inputs location information such as the URL of the non-natural language information source data and file name information.

[0224] Furthermore, in the role-playing device (AI response output device 10010), the control unit 1110 can cooperate with the memory 1109 to execute a web browser program and display the GUI of the web browser program on the display screen of the role-playing device (AI response output device 10010). The user's actions on the GUI of the web browser program can be accepted through user operations via the operation input unit 1107 (e.g., mouse, keyboard, touch panel) or touch operations detected by the touch operation input sensor of the display unit 10011. The non-natural language information source data, such as images, videos, and sounds selected on the browser screen of the web browser program, will be used as data for the specified object in the prompt. In this case, the web browser program can obtain the location information and file name information of the non-natural language information source data and pass it to the role-playing program.

[0225] Additionally, user 230 can also communicate with the role-playing device (AI response output device 10010) from the mobile information processing terminal 20010 by operating the mobile information processing terminal 20010, thereby inputting location information such as the URL of the non-natural language information source data into the role-playing device (AI response output device 10010). Alternatively, it can also use... Figure 3C The method described herein allows input of location information such as the URL and filename information for specifying non-natural language information source data. In this method, a QR code and other information storage image are displayed on the display panel 20011 of the mobile information processing terminal 20010. The image captured by the camera unit 1180 of the role-conversation device (artificial intelligence response output device 10010) is processed for image recognition, and the result of the image recognition is obtained.

[0226] Furthermore, the first method of transmitting or specifying non-natural language information source data in the prompt is not limited to situations where the non-natural language information source file pre-exists in a location such as a server connected to a network like the Internet. For example, when it is necessary to include non-natural language information source data such as images, videos, and sounds stored in the storage unit 1170 of the role-conversation device (AI response output device 10010) in the prompt, the role-conversation device (AI response output device 10010) can upload the non-natural language information source data to a second server 19002 via the Internet 19000, and include the location information (such as a URL) and filename of the uploaded non-natural language information source data on the Internet in the prompt. In this case, the second server 19002 functions as an intermediate server.

[0227] Similarly, when it is necessary to include non-natural language information source data such as images, videos, and sounds stored in the storage unit 20016 of the mobile information processing terminal 20010 in the prompt, the mobile information processing terminal 20010 can upload the non-natural language information source data to the second server 19002 via the Internet 19000. The location information (such as URL) and filename of the non-natural language information source data on the Internet from the mobile information processing terminal 20010 or the second server 19002 can be sent to the role-conversation device (artificial intelligence response output device 10010). The role action program of the role-conversation device (artificial intelligence response output device 10010) includes the location information (such as URL) and filename of the non-natural language information source data uploaded to the second server 19002 in the prompt.

[0228] Furthermore, the role action program of the role-conversation device (AI response output device 10010) can also cooperate with the memory 1109 and the storage unit 1170 to build a media server within the role-conversation device (AI response output device 10010) that can be accessed from other servers via the Internet 19000. In this case, when the role-conversation device (AI response output device 10010) specifies non-natural language information source data in the prompt word using the first method, it can store the Internet location information (such as a URL) representing the media server built within the role-conversation device (AI response output device 10010) itself and the file name of the non-natural language information source data in the aforementioned prompt word.

[0229] Next, a second method for transmitting or specifying non-natural language information source data in the prompt is, for example, simply storing (attaching) the non-natural language information source data itself in the prompt and sending it. Generally, non-natural language information source data such as images, videos, and sounds have a larger data volume compared to natural language text information. Therefore, in this case, the data volume of the prompt itself is larger than that of the first method. The role action program of the role conversation device (artificial intelligence response output device 10010) can first store the non-natural language information source data to be stored (attached) in the prompt in memory 1109, and when sending the prompt, store (attach) it from memory 1109 in the prompt via communication unit 1132 and output it to the large-scale language model server 20001. The non-natural language information source data stored in memory 1109 for the character action program of the character conversation device (artificial intelligence response output device 10010) can be obtained by the communication unit 1132 via the Internet 19000, or by the communication unit 1132 from the mobile information processing terminal 20010, or read from the storage unit 1170 and stored in memory 1109.

[0230] Using the methods described above, the role-conversation device (artificial intelligence response output device 10010) can transmit or specify non-natural language information source data through prompt words.

[0231] The Large Scale Language Model Server 20001 is a multimodal large-scale language model capable of processing non-natural language information sources alongside natural language text information. Therefore, it can... Figure 3D The example shows the first user prompt, which obtains images of the swimming pool and poolside as non-natural language information source 20061, as well as natural language text information, as its reasoning result, and outputs the natural language text information shown in the figure as the answer to the first user prompt.

[0232] Furthermore, the Large Scale Language Model Server 20001 is a multimodal large-scale language model capable of processing non-natural language information sources alongside natural language text information, therefore... Figure 3D As shown in the example of the second user prompt word response, the large-scale language model server 20001 is able to include the non-natural language information source 20062 generated by reasoning through the multimodal large-scale language model in the response and send it to the role conversation device (artificial intelligence response output device 10010). Figure 3D The example shown is an image obtained by marking circles on an image of a swimming pool and its edge, which is a non-natural language information source 20061. Furthermore, the non-natural language information source 20062 stored in the response is not limited to... Figure 3D The image shown can also be a video or audio.

[0233] In cases where the response from the large-scale language model server 20001 contains non-natural language information sources other than natural language text information, the method can use the first or second method corresponding to the first method or the second method of transmitting or specifying non-natural language information source data in the prompt words of the aforementioned role conversation device (artificial intelligence response output device 10010) (i.e., the first or second method can be referred to).

[0234] Specifically, as a method corresponding to the first method described above, the large-scale language model server 20001 can save the location and filename information of a specified non-natural language information source file in the response. The non-natural language information source 20062, such as images, videos, and sounds, can be stored in the large-scale language model server 20001, or it can be transferred to the second server 19002, which functions as an intermediate server, for retention. In either case, the large-scale language model server 20001 only needs to save the location and filename information of the specified non-natural language information source file in the response. The role-conversation device (artificial intelligence response output device 10010) that has obtained the response can use the location and filename information of the non-natural language information source file recorded in the prompt to access the large-scale language model server 20001 and the second server 19002 to obtain the non-natural language information source 20062.

[0235] Furthermore, specifically, as a method corresponding to the second method described above, the large-scale language model server 20001 can store (attach) the file data of the non-natural language information source 20062 in the answer and send it to the role-conversation device (artificial intelligence answer output device 10010). The role-conversation device (artificial intelligence answer output device 10010) can obtain the data of the non-natural language information source 20062 stored (attached) in the answer for various outputs to the user 230.

[0236] Based on the above usage Figure 3D The actions of the role-based conversation device (AI response output device 10010) and the role-based conversation system described in Embodiment 3 involve the exchange of prompts and responses between the role displayed on the role-based conversation device (AI response output device 10010) and the user 230 to realize a conversation using non-natural language information, namely images, videos, and sounds. Thus, it is possible to achieve... Figure 3D The messages displayed provide a more advanced and natural conversation.

[0237] Next, use Figure 3E This section describes an example of the operation of the role-based conversation device (artificial intelligence response output device 10010) according to Embodiment 3 of the present invention. This can also be described as an example of the operation of a role-based conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 20001. Specifically, Figure 3E This is an example of the main message of the prompt word sent from the AI ​​response output device 10010 to the large-scale language model server 20001 and the main message of the server response as its answer, based on which the conversation between the character 19051 and the user 230 displayed on the AI ​​response output device 10010 is implemented.

[0238] Figure 3E Indicates in Figure 3D This is an example of a situation where, after a series of conversations and after the end of that series of conversations, user 230 speaks to character 19051 again and a new conversation begins. Figure 3E In the example, the procedure described in Example 2 was not performed. Figure 2F , Figure 2G , Figure 2I The explanation uses session history processing. Therefore, Figure 3E Compared with Example 2 Figure 2E Similarly, the response obtained was completely unremembered about the large-scale language model's own name, the role it was to play, conversational features, the user's name, the history of the conversation, and other information contained in the set prompts.

[0239] Next, use Figure 3F This section describes an example of the operation of the role-based conversation device (artificial intelligence response output device 10010) according to Embodiment 3 of the present invention. This can also be described as an example of the operation of a role-based conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 20001. Specifically, Figure 3F This is an example of the main message of the prompt word sent from the AI ​​response output device 10010 to the large-scale language model server 20001 and the main message of the server response as its answer, based on which the conversation between the character 19051 and the user 230 displayed on the AI ​​response output device 10010 is implemented.

[0240] Figure 3F Indicates in Figure 3D This is an example of a situation where, after a series of conversations and the end of that series of conversations, user 230 speaks to character 19051 again, thus starting a new conversation. Here, Figure 3F In the middle, the second example will be... Figure 2F The method described in the example for saving messages explaining past conversation history in the set prompt words is also applicable to the role-based conversation device (AI response output device 10010) of Example 3. Specifically, the method described in the example is to save messages explaining past conversation history in the set prompt words. Figure 3D The message containing the set prompt words Figure 3F Save it as a reset message. After the reset message, save it as a message describing the past session history as a session history message.

[0241] Because the large-scale language model server 20001 in Example 3 is a multimodal large-scale language model capable of processing non-natural language information sources along with natural language text information, there are instances where non-natural language information source data was transmitted or specified in past prompts and responses. Therefore, Figure 3F The example in this session history message reflects not only the natural language text information in past prompts and responses, but also the transmission or designation of non-natural language information source data in past prompts and responses. Figure 3F The specific method of transmitting or specifying non-natural language information source data in the prompt words, and such as Figure 3D Since the transmission or specification of the non-natural language information source data described is the same, repeated descriptions are omitted.

[0242] Figure 3D In the examples, the methods for transmitting or specifying non-natural language information source data include cases where the non-natural language information source data itself is stored (attached) in the prompt word, and cases where the non-natural language information source data is not stored (attached) in the prompt word. Regarding this, Figure 3F The prompt words are the same.

[0243] Next, use Figure 3G This section describes an example of the operation of the role-based conversation device (artificial intelligence response output device 10010) according to Embodiment 3 of the present invention. This can also be described as an example of the operation of a role-based conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 20001. Specifically, Figure 3G This is an example of the main message of the prompt word sent from the AI ​​response output device 10010 to the large-scale language model server 20001 and the main message of the server response as its answer, based on which the conversation between the character 19051 and the user 230 displayed on the AI ​​response output device 10010 is implemented.

[0244] Figure 3G express Figure 3F The example shown is a series of conversations following the initial setting prompt, from the first user prompt and its response to the third user prompt and its response. Figure 3G The image shows a sequential interaction between prompts and responses. The content of the prompts is set because of... Figure 3F Since the information shown is the same, repeated entries are omitted.

[0245] As explained above, in the case of using the large-scale language model server 20001 of Embodiment 3, which is capable of processing multimodal large-scale language models of non-natural language information sources along with natural language text information, similarly, even if user 230 speaks to character 19051 again to start a new conversation after a series of sessions and the end of that series of sessions, by performing Figure 3F The generation and sending of the specified prompt words will determine the subsequent user response to the prompt words. Figure 3G As shown, this will be able to reflect the character's identity, name, conversation characteristics, personality, and / or conversation history from previous conversations. Therefore, from the user's perspective, the consistency between the character's identity, name, conversation characteristics, or personality settings and memories from previous conversations is better ensured, making this method even better.

[0246] Next, use Figure 3H This section describes an example of the operation of the role-based conversation device (artificial intelligence response output device 10010) according to Embodiment 3 of the present invention. This can also be described as an example of the operation of a role-based conversation system including the artificial intelligence response output device 10010 and the large-scale language model server 20001. Specifically, Figure 3H This is an explanatory diagram of a database 20200 used to manage character settings and conversation history for multiple characters displayed on the display unit 10011 of a character conversation device (artificial intelligence response output device 10010). Here, Figure 3H In this example, the settings of multiple characters displayed on the display unit 10011 of the character dialogue device (artificial intelligence response output device 10010) utilize the method described in Embodiment 2. Figure 2H Examples are provided below. Therefore, repeated explanations regarding the settings of multiple characters, etc., are omitted.

[0247] in addition, Figure 3H The database 20200 shown, used for managing character settings and character session history, has the same features as in Example 2. Figure 2I The database shown is in the same format as database 19200. Figure 3H China only to its Figure 2I The differences in database 19200 shown are explained. Additionally, the content regarding the character "Koto" within the database is explained, while content related to other characters is omitted.

[0248] Here, as described above, the large-scale language model server 20001 in Embodiment 3 is a multimodal large-scale language model capable of processing non-natural language information sources along with natural language text information. Therefore, the prompts from the role-conversation device (artificial intelligence response output device 10010) and the responses from the large-scale language model server 20001 include not only natural language text information but also the transmission or specification of non-natural language information source data. Therefore, Figure 3H In the database 20200 shown, the session history data records not only the natural language text information contained in these prompts and responses, but also the transmission or specified information of non-natural language information source data. The specific methods of transmitting or specifying non-natural language information source data in this session history record are related to... Figure 3D Since the transmission or specification of non-natural language information source data described herein is the same, repeated descriptions are omitted.

[0249] Figure 3D In the examples, the methods for transmitting or specifying non-natural language information source data include cases where the non-natural language information source data itself is stored (attached) in the prompt word, and cases where the non-natural language information source data is not stored (attached) in the prompt word. Regarding this, Figure 3H The conversation history is the same. However, Figure 3H In the session history, when specifying the location and filename information of the non-natural language information source file on a server (a second server 19002 acting as an intermediary server, or other cloud servers) located on a network such as the Internet, there is a possibility that the non-natural language information source file on that server may be deleted as the session history lengthens. Thus, even if the location and filename information are used, there is a possibility that the non-natural language information source file may no longer be available in the future, and the session record information may be lost.

[0250] To prevent this, when the role-based conversation device (AI response output device 10010) transforms the prompt and response messages into conversation history and records them, it can use the location information and filename information to retrieve the non-natural language information source file specified in the prompt and response from a server on the network and save it in the storage unit 1170. Furthermore, the role action program of the role-based conversation device (AI response output device 10010) can rewrite the location information and filename of the non-natural language information source file to represent the location information (such as a URL) on the Internet within the media server built inside the role-based conversation device (AI response output device 10010), and then record it in the conversation record. In this way, as long as the role-based conversation device (AI response output device 10010) itself does not delete the non-natural language information source file from the storage unit 1170, the non-natural language information source will not be lost from the conversation record information, so the preservation of the conversation record is better.

[0251] By using the above instructions Figure 3H In the case where the role dialogue device (artificial intelligence response output device 10010) is configured to switch between multiple role candidates to display the role to be displayed on the display unit 10011, the database of Embodiment 2 can be obtained. Figure 2I The effect is that, from the user's perspective, there is less unnaturalness in conversations with each character, and the user can share memories with multiple characters separately, resulting in a more enjoyable character conversation experience. This effect is also achieved when the large-scale language model server 20001 is a multimodal large-scale language model capable of processing non-natural language information sources along with natural language text information.

[0252] In addition, in the role-conversation device (artificial intelligence response output device 10010) or role-conversation system of Embodiment 3, a multimodal large-scale language model artificial intelligence is used in the large-scale language model server 20001, which is capable of processing non-natural language information in addition to natural language text information.

[0253] Here, the character conversation device (AI response output device 10010) communicates with the large-scale language model server 20001 using an API. In multimodal large-scale language models, there may be a form of payment that, in addition to charging for the number of language units processed from the text, called tokens, based on the number of natural language text information sources, also requires payment for API usage based on the amount of data from non-natural language information sources.

[0254] Therefore, in order to provide users with role-session services implemented by the role-session system of this embodiment at a lower cost, the following variations can be used.

[0255] As the first variation example, in Figure 3H The database's session history records also document the transmission or specified information of non-natural language information sources. However, when roles and users interact using natural language text information to conduct conversations about this non-natural language information source data, the content is also recorded as natural language text information. Thus, in Figure 3H Even if the transmission or specified information of the non-natural language information source data is omitted from the database's session history records, the session itself regarding that non-natural language information source data is still recorded to some extent as natural language text information. Therefore, if a certain degree of information reduction is allowed, then in Figure 3H In the database's session history records, the transmission or specified information of the non-natural language information source data can also be omitted. In this case, from Figure 3F The transmission or specified information of the non-natural language information source is also omitted from the session history messages of the setting prompt words. This reduces the amount of data from non-natural language information sources used in API communication.

[0256] Next, the second variation is an example in which, in Figure 3H Instead of recording the transmission or specified information of non-natural language information source data, the database's session history records natural language text information describing the content of the non-natural language information source data. This natural language text information describing the content of the non-natural language information source data can be obtained, for example, by establishing a session between the large-scale language model of the large-scale language model server 20001 and the role's session device (artificial intelligence response output device 10010) outside of a session with the role, specifying a character limit, and allowing the large-scale language model server 20001 to describe the content of the non-natural language information source data. Alternatively, it can be obtained by using another large-scale language model from another server that can be used at a lower cost than the large-scale language model of the large-scale language model server 20001, and by having a session with it, specifying a character limit, and allowing it to describe the content of the non-natural language information source data. Furthermore, if alternative text data is prepared at the time of obtaining the non-natural language information source data, this alternative text data can be used as natural language text information describing the content of the non-natural language information source data. A specific example of alternative text data for non-natural language information source data is tags in a markup language. , <video src="" alt="****"> 、 <audio src="" alt="****">The text information described in the **** section, etc.

[0257] Additionally, if JSON syntax is used, the key and value of the substitute text data itself can be stored in an object that is associated with the key and value representing the location information of the non-natural language information source data (i.e., the location information and file name information of the non-natural language information source data).

[0258] In this case, Figure 3H In the database's session history records, it is also possible to omit recording the transmission or specified information of the non-natural language information source data, from... Figure 3F The transmission or specified information of the non-natural language information source is also omitted from the session history messages of the setting prompt words. This reduces the amount of data from non-natural language information sources used in API communication.

[0259] Next, the third variation is an example in which, in Figure 3D At the moment of the first user prompt, instead of storing the transmitted or specified information of the non-natural language information source data in the user prompt, it replaces it with natural language text information describing the content of the non-natural language information source data. For example, it can be configured such that, in Figure 3D In the initial user prompt, instead of the information transmitted or specified by the non-natural language information source data 20061, a descriptive statement such as "This is an image of a swimming pool, poolside lounge chairs, and a parasol. There is water in the pool. There are drinks on the table next to the lounge chairs" is stored in the user prompt as natural language text information. This descriptive statement can also be obtained by using other large-scale language models on other servers that can be used at a lower cost than the large-scale language model of the large-scale language model server 20001, and by engaging with them and specifying a prescribed word count limit, thus describing the content of the non-natural language information source data. Alternatively, this descriptive statement can also be obtained from servers that can obtain summaries and descriptions of the content of non-natural language information source data such as images, videos, and audio. Furthermore, if alternative text data is prepared at the time of obtaining the non-natural language information source data, this alternative text data can be used as natural language text information describing the content of the non-natural language information source data.

[0260] Next, use Figure 3I Here, an example of the display of the role-playing conversation device (artificial intelligence response output device 10010) of Embodiment 3 of the present invention will be described. Figure 3I The example shows that Figures 3A to 3H The following is an example of how the large-scale language model's response to the user's prompts is displayed on the display unit 10011 of the character conversation device (artificial intelligence response output device 10010). Specifically, it is an example where the response from the large-scale language model—text 10063 from natural language information source data, image 10064 from non-natural language information source data, and / or video 10065 from non-natural language information source data—is displayed together with the image of the character 19051 on the display unit 10011. The response from the large-scale language model—text 10063, image 10064, and / or video 10065—can be displayed as follows: Figure 3I The image is overlaid in front of the character 19051 as shown.

[0261] Additionally, responses from a large-scale language model, namely text 10063, image 10064, and / or video 10065, can also be displayed alongside the image of character 19051 without being overlaid on it. Figure 3I For example, when user 230 operates the touch operation input sensor of operation input unit 1107 or display unit 10011, and adjusts the volume of the sound output of the sound output unit 1140 of the character conversation device (artificial intelligence response output device 10010) to the minimum or sets the sound output to OFF, user 230 cannot confirm the answer from the large-scale language model by sound. Therefore, in this case, control unit 1110 can control the process to begin executing the output of the answer from the large-scale language model, i.e., text 10063, image 10064, and / or video 10065, as shown in the image. Figure 3I The display mode shown is displayed together with the image of character 19051.

[0262] In this manner, even when sound output is to be suppressed, user 230 can better utilize the character conversation device (AI response output device 10010). Alternatively, it can be configured such that user 230 can manually switch the display mode—which displays responses from the large-scale language model, i.e., text 10063, images 10064, and / or video 10065, along with the image of character 19051—on / off via the operation input unit 1107 or the touch operation input sensor of the display unit 10011. Figure 3I For example, in a multimodal role-playing conversation device (AI response output device 10010), it is able to output responses from a large-scale language model more effectively.

[0263] The role-based conversation device and system of Embodiment 3 described above, in addition to the effects of the role-based conversation device and system of Embodiment 2, can also utilize a multimodal large-scale language model to provide users with a more advanced conversational experience that includes non-natural language information in addition to natural language information. Furthermore, the role-based conversation device and system of Embodiment 3 can provide role-based conversation services to users at a lower cost.

[0264] Furthermore, in the above description of Embodiment 3, an example was described using a large-scale language model possessed by the large-scale language model server 20001 as a large-scale language model. In contrast, a role-based conversation device (artificial intelligence response output device 10010) could also be included. Figure 1B The local LLM processing unit 10028 shown uses the multimodal large-scale language model it possesses. In this case, the multimodal large-scale language model of the local LLM processing unit 10028 can be used instead of the multimodal large-scale language model of the large-scale language model server 20001.

[0265] In this case, as described above in Embodiment 3, the multimodal large-scale language model of the large-scale language model server 20001 can be replaced with the multimodal large-scale language model of the local LLM processing unit 10028 of the role-conversation device (artificial intelligence response output device 10010). In this case, the multimodal large-scale language model can also be used to provide users with a more advanced conversational experience that includes non-natural language information in addition to natural language information. Furthermore, when using the multimodal large-scale language model of the local LLM processing unit 10028 instead of the multimodal large-scale language model of the large-scale language model server 20001, although there is little need to consider the usage fees corresponding to the number of processed lexical units and the amount of data from non-natural language information sources, even with the multimodal large-scale language model of the local LLM processing unit 10028, the power and other resource consumption for inference can be reduced by decreasing the number of processed lexical units and the amount of data from non-natural language information sources. In this case, a role-conversation service with lower power consumption can be provided to the user.

[0266] Furthermore, Example 2 describes the structure for uploading or downloading user and role conversation history, including data in a database containing user and role conversation history, to a second server 19002 or other cloud servers. This structure can also be used in the example using a multimodal large-scale language model described in Example 3. In this case, for one or more roles, when a user has multiple conversations with each of the roles at different times and on different individual devices, it is possible to achieve a conversation where the user perceives that they have inherited the memories of each role from the previous conversation, which is more preferable for the user.

[0267] <Example 4>

[0268] Next, Embodiment 4 of the present invention improves upon the artificial intelligence response output device 10010, the role-playing device, or its system illustrated in the figures of Embodiment 2 or Embodiment 3. Differences from Embodiment 2 or Embodiment 3 are described in this embodiment, and repeated descriptions of structures identical to those embodiments are omitted.

[0269] Similar to the embodiments described above, the AI ​​response output device 10010 can be referred to as an AI response output device, an AI assistant device, an AI assistant display device, or an AI interface device. A system including the AI ​​response output device 10010 and a large-scale language model server can also be referred to as an AI response output system, an AI assistant system, an AI assistant display system, or an AI interface system.

[0270] use Figure 4A An example of an action performed using a database in the role-conversation device (artificial intelligence response output device 10010) of Embodiment 4 of the present invention will be described. Figure 4A The database pair shown in Example 4 Figure 2I or Figure 3I The database described in the document has been expanded. Specifically, Figure 4A The database shown envisions a scenario where multiple different users use the same role conversation device (AI response output device 10010) or the same role conversation system, storing the initial setting prompts and conversation history corresponding to each user and role in the database.

[0271] Figure 4A In the example, for user 1 with user ID 1, the initial setup prompts and session history for each of the following characters are saved: Koto (character ID 1), Tom (character ID 2), and Necco (character ID 3). Similarly, for user 2 with user ID 2 and user 3 with user ID 3, the initial setup prompts and session history for each of the following characters are also saved: Koto (character ID 1), Tom (character ID 2), and Necco (character ID 3).

[0272] These initial setup prompts and session history data are stored as separate data in areas that differ for each combination of user and role. Figure 4A For illustrative purposes, the data stored in each area is recorded and represented as data 11, 12, 13, 21, 22, 23, 31, 32, and 33. The control unit 1110 of the role-conversation device (AI response output device 10010) uses the initial setting prompts and conversation history stored in areas that differ for each combination of user and role, based on the user currently using (logining) the role-conversation device (AI response output device 10010) or its system. This allows for better maintenance of consistency in the character's personality and persistence of memory for different users.

[0273] Specifically, consider the scenario where User 1 first uses the role-based conversation device (AI response output device 10010) to have a conversation with the role Tom, and User 2 is unaware of this conversation. Then, if the AI ​​response output device 10010 uses initial preset prompts or a database of conversation history without user identification, the response output by the AI ​​response output device 10010 is based on a conversation history that does not exist in User 2's memory. This raises the possibility that User 2's conversation with the role of the AI ​​response output device 10010 may not match (i.e., effective communication may be impossible).

[0274] In contrast, even under the same conditions, by using Figure 4A The database shown indicates that the control unit 1110 of the role-based conversation device (AI response output device 10010) can identify users based on their IDs, store initial setting prompts and conversation history in different areas for each user, and use the initial setting prompts and conversation history stored in different areas for each user to generate AI responses. Therefore, the initial setting prompts and conversation history used to generate AI responses for each user are managed separately from the user's operations or conversation context, based on that user's actions or conversation context. This allows for better matching of each user's conversation history with the roles of the AI ​​response output device 10010.

[0275] in addition, Figure 4A The database of initial setting prompts and / or conversation history described herein can be stored in the storage unit 1170 of the AI ​​response output device 10010 and used by the control unit 1110. Alternatively, the database of initial setting prompts and / or conversation history can also be stored on a server on a network. For example, when the AI ​​response output device 10010 generates AI responses using a large-scale language model from a large-scale language model server 19001 or a multimodal large-scale language model from a large-scale language model server 20001, the database can be stored on these servers themselves. Figure 4A The database of initial setup prompts and / or conversation history described herein. This approach eliminates the need for the AI ​​response output device 10010 to re-include the initial setup prompts and conversation history in the prompts before sending them to these servers, thus saving on the number of tokens sent in the use of large-scale language models.

[0276] Save Figure 4A In the case of the initial setup prompts and / or a database of session history as described in the instructions, the AI ​​response output device 10010 simply sends the user ID, role ID, and user prompts for subsequent sessions to these servers. The large-scale language models on these servers use the user ID and role ID obtained from the AI ​​response output device 10010 to... Figure 4A The system retrieves the corresponding initial setting prompts and conversation history from a database of initial setting prompts and / or conversation history. The large-scale language models on these servers can use these initial setting prompts and conversation history, along with user prompts sent from the AI ​​response output device 10010 for subsequent conversations, to perform inference, generate AI responses, and send them to the AI ​​response output device 10010. This approach saves on the number of lexical units sent in the use of large-scale language models, while also achieving better consistency in character personality and memory persistence for different users.

[0277] Next, use Figure 4B An example of an action performed using a database in the role-conversation device (artificial intelligence response output device 10010) of Embodiment 4 of the present invention will be described. Figure 4B The database pair shown in Example 4 Figure 1C or Figure 2L The database described in the document has been expanded. Specifically, Figure 4B The database shown envisions a scenario where multiple different users use the same role conversation device (AI response output device 10010) or the same role conversation system, storing data of the response templates corresponding to each user and role in the database.

[0278] Figure 4B In the example, for user 1 with user ID 1, the answer format data for each of the following roles is stored: Koto (role ID 1), Tom (role ID 2), and Necco (role ID 3). Similarly, for user 2 with user ID 2 and user 3 with user ID 3, the answer format data for each of the following roles is also stored: Koto (role ID 1), Tom (role ID 2), and Necco (role ID 3).

[0279] The data for these answer-specific statements is stored as separate data in areas that differ for each combination of user and role. Figure 4B For illustrative purposes, the data stored in each area will be denoted as response statement data 101, 102, 103, 201, 202, 203, 301, 302, and 303. For example, in response statement data 101, the data will be... Figure 2L Role 1: Koto's answer statements corresponding to condition numbers 1-7 are saved as tables or other database entries. Figure 4B In data 201, Figure 2L Role 2: Tom's answer statements corresponding to condition numbers 1 to 7 are saved as tables or other database entries.

[0280] exist Figure 4B In data 301, Figure 2L Role 3: Necco's answer statements corresponding to condition numbers 1-7 are saved as tables or other database entries. Figure 4B Data 102, 202, and 302 are stored in the same format, containing the modified response statements for user 2. Figure 4B In data 103, 203, and 303, the modified answer format statements for user 3 are stored in the same format. The control unit 1110 of the role conversation device (artificial intelligence answer output device 10010) uses the answer format statement data stored in areas that are different for each combination of user and role, based on the user currently using (login) the role conversation device (artificial intelligence answer output device 10010) or its system.

[0281] This approach allows even the same user character to respond using pre-defined answer phrases tailored to each user. That is, even with the same user character, it's best to adjust the content of the pre-defined answer phrases based on the relationship between the character and the user. For example, depending on the relationship between the character's set age and the ages of users registered in the AI ​​response output device 10010 or the system, there may be cases where the user is older than the character, the same age, or younger. In these cases, changing the content of the pre-defined answer phrases for older users, users of the same age, and younger users respectively will result in a better or more natural conversation between the user and the character. In other words, by using... Figure 4B The database's actions can vary the content of the response statements based on each relationship between the role and the user, creating a better or more natural conversation.

[0282] In addition, the above explanation Figure 4B The database of definitive answer statements (answer definitive statement DB) is stored in the storage unit 1170, and the control unit 1110 of the artificial intelligence answer output device 10010 can use it to generate answers. However, Figure 4B The answer specification statement database (answer specification statement DB) shown can also be set on either the large-scale language model server 19001 or the large-scale language model server 20001. In this case, the control unit of the large-scale language model server 19001 or the large-scale language model server 20001 can generate an answer using the answer specification statement database (answer specification statement DB). In the control unit of the large-scale language model server 19001 or the large-scale language model server 20001, the answer generated using the answer specification statement database (answer specification statement DB) can be sent to the artificial intelligence answer output device 10010 instead of the answer generated by the large-scale language model stored in each server. In this way, even if the artificial intelligence answer output device 10010 does not have an answer specification statement database (answer specification statement DB), it is possible to generate an answer using the answer specification statement database (answer specification statement DB).

[0283] The role-based conversation device and system described in Embodiment 4 above can create better or more natural conversations based on the relationship between the role and the user, conversation history, etc.

[0284] <Example 5>

[0285] Next, Embodiment 5 of the present invention improves upon the artificial intelligence answer output device 10010 or artificial intelligence answer output system illustrated in the figures of Embodiments 1, 2, and 3. Specifically, in an example where processing is performed, the answer generation process of the artificial intelligence answer output device 10010 is switched from answer generation processing based on a large-scale language model on the network to answer generation processing based on a local large-scale language model (such as a local LLM processing unit 10028) possessed by the artificial intelligence answer output device 10010, or answer generation processing based on an answer-defined statement database. The differences from these embodiments are described in this embodiment, and repeated descriptions of structures identical to those embodiments are omitted.

[0286] Similar to the embodiments described above, the AI ​​response output device 10010 can be referred to as an AI response output device, a role-based conversation device, an AI assistant device, an AI assistant display device, or an AI interface device. A system including the AI ​​response output device 10010 and a large-scale language model server can also be referred to as an AI response output system, a role-based conversation system, an AI assistant system, an AI assistant display system, or an AI interface system.

[0287] use Figure 5A An example of the switching process for the answer generation process in the artificial intelligence answer output device 10010 of Embodiment 5 of the present invention will be described. Figure 5A The table shows examples 1 to 9 of the switching process for the answer generation process in the artificial intelligence answer output device 10010. Figure 5A The table shows a summary of the switching process for each example in the "Switching Summary" column. The "State Before Switching to LLM (API-Connected LLM) on the Network" column shows the state before the response generation process of large-scale language models (using API connections) on the network, such as the large-scale language model possessed by the large-scale language model server 19001 and the multimodal large-scale language model possessed by the large-scale language model server 20001 (based on Figure 1), is switched to another response generation process. The "Switching Occurrence Conditions" column shows the conditions under which the response generation process switches. The "Target to Switch From LLM (API-Connected LLM) on the Network" column shows the switching target to which the response generation process of the artificial intelligence response output device 10010 switches from the large-scale language model possessed by the large-scale language model server 19001 and the multimodal large-scale language model possessed by the large-scale language model server 20001 (based on Figure 1) to. The control unit 1110 of the artificial intelligence response output device 10010... Figure 5A If the conditions shown in "Switchover Conditions" occur in the "Pre-Switchover State of LLM (API-Connected LLM) on the Network" state, control can be exercised to switch to the large-scale language model, database, or countermeasure shown in "Target to Switch From LLM (API-Connected LLM) on the Network".

[0288] The following is about Figure 5A The examples shown in the table will be explained. As shown in the "Switching Summary", Example 1 is an example of switching according to the network connectivity status of the AI ​​response output device 10010. In Example 1, as "State before switching LLM (API-connected LLM) on the network", the network connectivity status of the AI ​​response output device 10010 is shown to be a connectable state. Here, in Example 1, as "Switching Occurrence Condition", "Network becomes unconnectable" is shown. That is, this means that the AI ​​response output device 10010 and the large-scale language model (using API connection) on the network can no longer connect via the network. Specifically, this unconnectivity may be caused by the connection path from the AI ​​response output device 10010 to the Internet 19000 becoming uncommunicable. Alternatively, the unconnectivity may also be caused by the uncommunicable state of the Internet 19000. Alternatively, the unconnectivity may also be caused by the large-scale language model (using API connection) on the network itself being unable to connect to the Internet 19000. Furthermore, in Example 1, "local LLM" is shown as the target to which "switching from an LLM (API-connected LLM) on the network" is made. Specifically, this means performing a switching process to switch to the answer generation process based on the local LLM processing unit 10028 of the AI ​​answer output device 10010. That is, in Example 1, if for some reason it is impossible to connect to a large-scale language model (using API connection) on the network and the answer generation process based on the large-scale language model (using API connection) cannot be used, the switch is made to the answer generation process based on the local LLM processing unit 10028 of the AI ​​answer output device 10010. Thus, although there is a performance difference in the large-scale language model, the answer generation process using the large-scale language model can continue.

[0289] Next, regarding Figure 5A Example 2 will be used to illustrate this. Example 2 changes the target of "switching from an LLM (API-connected LLM) on the network" in Example 1 from "local LLM" to "responding to a defined statement DB (database)". The response generation process based on this "responding to a defined statement DB (database)" will then be... Figure 1C , Figure 2L or Figure 4B The processing described in the example is the same, so repeated explanations are omitted. That is, in Example 2, if for some reason it is impossible to connect to a large-scale language model on the network (a large-scale language model connected via API) and the answer generation processing based on the large-scale language model on the network (a large-scale language model connected via API) cannot be used, the answer generation processing using the answer formatting statement database is switched to, thereby enabling the generation of an answer through simpler processing and outputting the answer to the user.

[0290] Next, regarding Figure 5A Example 3 will be used as an example. In Example 3, the "target to which the user switches from the LLM (API-connected LLM) on the network" in Example 1 is changed from "local LLM" to "no response". This "no response" means that even if the user makes user input requesting a response from the large-scale language model via the touch panel, microphone 1139, or operation input unit 1107, no response will be generated for that input, or no response will be output for the user input requesting a response from the large-scale language model. That is, in Example 3, in the case where, for some reason, the connection to the large-scale language model on the network (the large-scale language model using API connection) cannot be established, and the response generation processing based on the large-scale language model on the network (the large-scale language model using API connection) cannot be used, the handling in that situation can be simplified.

[0291] Next, regarding Figure 5A Example 4 will be used as an example. As shown in the "Switching Summary", Example 4 is an example of switching based on the delay of the response from an LLM on the network. In Example 4, the state before switching from the LLM (API-connected LLM) on the network is shown as the state where a response from the LLM on the network is received within a specified time. Here, in Example 4, the case where no response from the LLM on the network is received within the specified time is shown as the "Switching Condition", and the specified time has exceeded the specified time is shown. In addition, in Example 4, the "target to which to switch from the LLM (API-connected LLM) on the network" is shown as the "local LLM". The "local LLM" of the switching target is the same as in Example 1, so the repeated explanation is omitted. That is, in Example 4, if for some reason the response from the LLM (large-scale language model using API connection) on the network exceeds the specified time and the response generation processing based on the LLM (large-scale language model using API connection) on the network cannot be used smoothly, the process also switches to the response generation processing based on the local LLM processing unit 10028 of the artificial intelligence response output device 10010. Therefore, despite the performance differences of large-scale language models, it is still possible to continue to perform response generation processing using large-scale language models.

[0292] Next, regarding Figure 5A Example 5 will be used as an example. Example 2 changes the target of "switching from an LLM (API-connected LLM) on the network" in Example 4 from "local LLM" to "responding to a defined statement DB (database)". The response generation process based on this "responding to a defined statement DB (database)" is then... Figure 1C , Figure 2L or Figure 4B The processing described in the example is the same, so repeated explanations are omitted. That is, in Example 5, if for some reason the answer from the LLM (Large-Scale Language Model connected via API) on the network exceeds the specified time and the answer generation processing based on the LLM (Large-Scale Language Model connected via API) on the network cannot be used smoothly, the process switches to answer generation processing using the answer formatting statement database, thereby enabling the generation of the answer through simpler processing and outputting the answer to the user.

[0293] Next, regarding Figure 5A Examples 6 through 9 will be used for illustration. As shown in the "Switching Summary," Examples 6 through 9 are examples of switching based on API usage or reaching a usage fee cap. Here, as explained in Example 2, providers of large-scale language models mostly recoup the cost of training the large-scale language model from end-users in the form of API usage fees. In this case, natural language models mostly require payment of API usage fees based on the number of language units processed, called tokens, derived from the text. Here, various payment methods and limiting methods can be considered for API usage fees. As one approach, consider using the number of tokens processed to define an upper limit on the amount of services a user can normally accept from a large-scale language model.

[0294] In this case, before reaching the usage limit (or the corresponding usage fee), users can accept the service of using the large-scale language model at the specified API usage fee. When the upper limit of usage (or the corresponding usage fee) is reached, certain restrictions can be expected, such as no longer being able to accept the service provided by the large-scale language model in the usual state (performance or frequency).

[0295] Figure 5A Examples 6 to 9 illustrate the switching control of the response generation process performed by the control unit 1110 of the artificial intelligence response output device 10010 when such limitations arise in the use of large-scale language models. Specifically, in Example 6, the "pre-switching state of LLM (API-connected LLM) on the network" is a state where API usage or API usage fees have not reached the prescribed limit. This indicates that the usage of the LLM (API-connected LLM) on the network has not yet reached the prescribed limit. At this time, the user can use the LLM (API-connected LLM) on the network in its normal state.

[0296] Here, in Example 6, the "switching condition" illustrates a situation where API usage or API usage fees reach a predetermined limit. This indicates that the usage of an LLM (API-connected LLM) on the network has reached its predetermined limit. Additionally, Example 6 shows a second LLM on the network, different from the LLM used in the normal state (which can be called the first LLM), as the "target to switch from the LLM (API-connected LLM) on the network." Examples of second LLMs on the network could include LLMs with lower costs compared to the first LLM used in the normal state. Because it is a lower-cost service, the performance of the second LLM can be considered lower than that of the first LLM. In this case, it is highly advantageous to be able to use a large-scale language model at a low cost even after reaching the usage / usage fee limit of the first LLM.

[0297] Next, regarding Figure 5A Example 7 illustrates this. Example 7 changes the "target from the LLM (API-connected LLM) on the network" in Example 6 from a second LLM on a different network than the LLM used normally (which can be called the first LLM) to a "local LLM". In Example 7, when API usage or API usage fees reach a predetermined limit—that is, when the usage of the LLM (API-connected LLM) on the network reaches a predetermined limit—switching to a response generation process based on a local LLM, which is not limited by the usage of the LLM on the network, API usage, or API usage fees, allows continued response generation processing using a large-scale language model.

[0298] Next, regarding Figure 5A Example 8 will be used as an example. Example 8 changes the target of "switching from an LLM (API-connected LLM) on the network" in Example 7 from "local LLM" to "responding to a defined statement DB (database)". The response generation process based on this "responding to a defined statement DB (database)" is then... Figure 1C , Figure 2L or Figure 4B The processing described in Example 8 is the same, so repeated explanations are omitted. Specifically, in Example 8, when API usage or API fees reach the specified limit (i.e., when the usage of LLMs (API-connected LLMs) on the network reaches the specified limit), the process switches to answer generation processing based on a predefined statement database, which is not limited by the usage of LLMs on the network, API usage, or API fees. This allows for simpler processing to generate and output the answer to the user.

[0299] Next, regarding Figure 5A Example 9 illustrates this further. Example 9 changes the target of "switching from the LLM (API-connected LLM) on the network" in Example 7 from "local LLM" to "no response". "No response" means neither generating nor outputting a response to the user. Example 9 simplifies the handling of situations where response generation processing based on a large-scale language model on the network (using API connections) cannot be used because API usage or API fees have reached a certain limit—that is, the usage of the LLM (API-connected LLM) on the network has reached a certain limit.

[0300] Based on the above explanation Figure 5A The switching control of the answer generation process of the artificial intelligence answer output device 10010 shown in Examples 1 to 9 can perform better switching or response measures corresponding to each situation when answer generation processing based on network-based LLM (large-scale language model connected via API) cannot be used as usual.

[0301] in addition, Figure 5A The switching control of Examples 1 to 9 can also be implemented by combining multiple examples. For example, the switching control of Examples 1 to 3 can be combined with any one of the controls in Examples 4 to 9. Similarly, the control of Example 4 or Example 5 can be combined with any one of the controls in Examples 1 to 3, or Examples 6 to 9. Likewise, the control of Examples 6 to 9 can be combined with any one of the controls in Examples 1 to 5.

[0302] Next, use Figures 5B to 5D This describes an example of an AI assistant or role display when the AI ​​response output device 10010 of Embodiment 5 is configured as an AI assistant device or a role conversation device.

[0303] first, Figure 5B Is to carry out Figure 5A In the case of switching control in Example 3, the display of the AI ​​assistant or role in the AI ​​response output device 10010 is an example. Figure 5B In the example, the display status of the AI ​​assistant or character is changed accordingly based on whether the network connection status of the AI ​​response output device 10010 is available or unavailable. The network connectivity status and network inaccessibility status of the AI ​​response output device 10010 are related to... Figure 5A The explanation is the same as that in the previous section, so it will not be repeated.

[0304] Figure 5B In the example, the AI ​​response output device 10010 displays the AI ​​assistant or character in a normal, awake state when the network is connected, and in a "sleep" state when the network is not connected. Figure 5A In the switching control of Example 3, if the network of the AI ​​response output device 10010 is unconnected, no response will be generated or output even if the user inputs a prompt. In this case, if the AI ​​assistant or character displayed by the AI ​​response output device 10010 is in a normally awake state, the user will feel unnatural. However, if the AI ​​assistant or character displayed by the AI ​​response output device 10010 is in a sleeping state, the user can understand that "the AI ​​assistant or character is not responding because it is sleeping," which can further reduce the unnaturalness felt by the user.

[0305] In addition, Figure 5B In case (2), it is preferable that the user understands "the AI ​​assistant or character is not responding because it is sleeping" before the user inputs a request for a response from a large-scale language model via the touch panel, microphone 1139, or operation input unit 1107 of the AI ​​response output device 10010. Therefore, in Figure 5B (2) When the network is not connected, the timing for displaying the status of the AI ​​assistant or character in a "sleep" state is preferably after the control unit 1110 of the artificial intelligence answer output device 10010 has just determined that the network is not connected, and before the user makes user input requesting a large-scale language model to provide an answer.

[0306] Next, as another example, regarding Figure 5C The following example will be used to illustrate this. Figure 5C The example shown is in Figure 5A In the switching control, an example is given of changing the display status of the AI ​​assistant or character accordingly based on the status of "target switched from LLM (API-connected LLM) on the network" in the table. Specifically, Figure 5C The diagram shows an example of an AI assistant or role in a state where (1) the AI ​​response output device 10010 is able to connect to a large-scale language model (connected via API) on the network and can use response generation processing based on the large-scale language model on the network (referred to as the normal state in this diagram); (2) an example of an AI assistant or role in a state where the AI ​​response output device 10010 switches to response generation processing based on an LLM or response type statement database, which has lower performance than the large-scale language model on the network (connected via API); and (3) an example of an AI assistant or role in a state where the AI ​​response output device 10010 switches to... Figure 5A The example shown is of an AI assistant or character in a "no response" state.

[0307] Figure 5C In the example, for instance, when the AI ​​response output device 10010 is in "normal state" (1), the AI ​​response output device 10010 displays the AI ​​assistant or character without any particular question. Additionally, Figure 5C The "normal state" in this context can be considered as a state other than states (2) and (3). Additionally, for example, when the AI ​​response output device 10010 switches to a response generation process based on an LLM or response formula database with lower performance than a large-scale language model on the network (a large-scale language model connected via API), the AI ​​response output device 10010 displays the AI ​​assistant or character in a "drowsy" state. Furthermore, "displaying the AI ​​assistant or character in a 'drowsy' state" can also be expressed as "a display indicating that the AI ​​assistant or character is drowsy."

[0308] The response generation process in (2) is less efficient than the response generation process in (1) which is based on a large-scale language model on the internet (a large-scale language model connected via API). Therefore, by displaying the AI ​​assistant or character in a "drowsy" state, it is possible to imply to the user that the AI ​​assistant or character's response performance (response performance) is low. This further reduces the unnaturalness that users may feel towards low-performance responses. In addition, the switching condition for the AI ​​response output device 10010 to switch to response generation processing based on an LLM or response formula database, which has lower performance than a large-scale language model on the internet (a large-scale language model connected via API), is different from that of the AI ​​assistant or character. Figure 5A The same explanations are provided in the text, so repeated explanations are omitted.

[0309] In addition, Figure 5C In case (2), it is preferable to suggest to the user that the AI ​​assistant or character's response performance is low before the user makes user input requesting a response from a large-scale language model via the touch panel, microphone 1139, or operation input unit 1107 of the AI ​​response output device 10010. Therefore, Figure 5C (2) The timing of starting to display the state of the AI ​​assistant or character as "drowsy" is preferably after the AI ​​response output device 10010 has just switched to response generation processing based on an LLM or response formula database that has lower performance than a large-scale language model on the network (a large-scale language model connected via API), and before the user makes user input requesting a response from the large-scale language model.

[0310] Additionally, for example, in (3) the artificial intelligence answer output device 10010 switches to Figure 5A In the "no response" state described above, the AI ​​response output device 10010 displays the AI ​​assistant or character in a "sleep" state. For example... Figure 5B As explained, by displaying the AI ​​assistant or character in a "sleeping" state through the AI ​​response output device 10010, the user can understand that "the AI ​​assistant or character is not responding because it is sleeping," further reducing the unnatural feeling experienced by the user. Additionally, the AI ​​response output device 10010 switches to... Figure 5A The conditions for switching to "no response" as explained in the text are the same as those for switching to "no response". Figure 5A The same explanations are given in Examples 3 or 9, so repeated explanations are omitted. Additionally, in Figure 5C In case (3), it is preferable that the user understands "the AI ​​assistant or character is not responding because it is sleeping" before the user inputs a request for a response from a large-scale language model via the touch panel, microphone 1139, or operation input unit 1107 of the AI ​​response output device 10010. Therefore, Figure 5C (3) The start time for displaying the state of the AI ​​assistant or character in "sleep" state is preferably when the artificial intelligence response output device 10010 has just switched to the "sleep" state. Figure 5A This refers to the moment after the "no response" statement and before the user provides user input that requires a large-scale language model to provide an answer.

[0311] in addition, Figure 5C In the example provided, the AI ​​response output device 10010 does not directly provide a technical explanation of its state regarding response generation processing to the user. Instead, it subtly reflects changes in the state of the AI ​​assistant or character through its display. This further reduces the unnaturalness experienced by the user compared to directly providing a technical explanation of the AI ​​response output device 10010's state regarding response generation processing. Furthermore, it further reduces the unnaturalness experienced by the user compared to situations where the display state of the AI ​​assistant or character remains the same even if the state of the AI ​​response output device 10010 regarding response generation processing changes.

[0312] However, depending on the user, there may be situations where they want more accurate technical status descriptions for each state. Therefore, for display examples that address such users, use... Figure 5D Please provide an explanation. Figure 5D The rows in the table shown are the device status, the rows describing the display status, and... Figure 5C They are exactly the same, so repeated descriptions are omitted. Additionally, the rows showing examples of AI assistants or characters are identical to... Figure 5C They are largely the same, except that a question mark (?) is displayed in the explicit example. This question mark (?) is a marker used when the user requests clarification from the AI ​​response output device 10010, and can be called a help marker.

[0313] Figure 5D In the example, when the user via Figure 1B When a user selects the question mark (?) through an operation input unit 1107 or a touch panel on the display unit 10011, the display of the AI ​​assistant or character in the artificial intelligence response output device 10010 changes to the display example shown in the row of the user operation display example. Specifically, regardless of whether the device state is one of (1), (2), or (3), a technical status description for each state is displayed. For example, Figure 5D In the example, if the device's state is (1) normal state, it can be displayed as "normal state," indicating that it is in a normal state without any particular technical limitations. Alternatively, if the device's state is (2) using a low-performance LLM or responding to a categorical statement database, it can be displayed as "low-performance mode," indicating that it is in a low-performance state. The purpose of this display can be considered as explaining why the AI ​​assistant or character is displayed in a "drowsy" state.

[0314] In this case, a more detailed technical explanation can also be provided. Specifically, a "low-performance LLM usage mode" or "fixed-statement response mode" can be displayed. In addition, if the device's state is (3) "no response", a technical explanation of the reason for switching to "no response" such as "network unconnected state" can be displayed. If the reason for switching to "no response" is that the response from an LLM (Large-Scale Language Model) on the network exceeds the specified time, a message such as "LLM response is delayed" can be displayed. In addition, if the reason for switching to "no response" is that the usage of LLM, API usage, or API usage fee on the network has reached the limit, a message such as "LLM usage has reached the limit", "API usage has reached the limit", or "API usage fee has reached the specified amount" can be displayed. The purpose of these displays can be considered as explaining why the AI ​​assistant or character is displayed in a "sleep" state.

[0315] Based on the above explanation Figure 5D For example, even if there are technical limitations in the answer generation process of the AI ​​answer output device 10010, the user is not directly informed. Instead, the device's status is implied through changes in the display state of the AI ​​assistant or character, thereby further reducing any unnaturalness experienced by the user. This display is more suitable for users who do not require technical explanations. Furthermore, by displaying an operation marker for providing technical status explanations, the status of the answer generation process in the AI ​​answer output device 10010 (normal state or state with technical limitations) is displayed to users who have operated the marker. This provides a better display for users who want to accurately understand the technical status.

[0316] in addition, Figure 5B , Figure 5C , Figure 5D In the example shown, the "sleeping" state is displayed as a way to indicate that the AI ​​assistant or character is not responding. However, this is just one example, and the implementation is not limited to this. Instead of the "sleeping" state, other display states such as "resting" that imply an inability to respond can also be used. Furthermore, Figure 5C , Figure 5D In the example shown, the "drowsy" state is illustrated as an example of the display state of an AI assistant or character when using a low-performance LLM or a database that responds to definite statements. However, this is just one example, and the approach in this embodiment is not limited to this. Other display states, such as "hungry," which imply low response performance of the AI ​​assistant or character, could also be used instead.

[0317] The AI ​​response output device and AI response output system of Embodiment 5 described above can better switch the response generation processing used by the AI ​​response output device based on factors such as the connection status between the large-scale language model on the network and the AI ​​response output device, the response latency status from the large-scale language model on the network, or the usage of the large-scale language model on the network. Furthermore, when the AI ​​response output device of Embodiment 5 is configured as an AI assistant device or a role-playing conversation device, it can provide less unnatural and less visually unappealing displays for the user.

[0318] <Example 6>

[0319] Next, Embodiment 6 of the present invention improves upon the artificial intelligence answer output device 10010 or artificial intelligence answer output system described in the figures of Embodiments 1 to 5. Specifically, in the answer generation process of the artificial intelligence answer output device 10010, an example is given where answer generation processing based on a large-scale language model on the network or answer generation processing based on a local large-scale language model (such as the local LLM processing unit 10028) provided by the artificial intelligence answer output device 10010 is better combined with answer generation processing based on an answer formula database to generate the answer output. In this embodiment, the differences from these embodiments are described, and repeated descriptions of the same structure as these embodiments are omitted.

[0320] Similar to the embodiments described above, the AI ​​response output device 10010 can be referred to as an AI response output device, a role-based conversation device, an AI assistant device, an AI assistant display device, or an AI interface device. A system including the AI ​​response output device 10010 and a large-scale language model server can also be referred to as an AI response output system, a role-based conversation system, an AI assistant system, an AI assistant display system, or an AI interface system.

[0321] use Figure 6 An example of the answer generation process in the artificial intelligence answer output device 10010 of Embodiment 6 of the present invention will be described. Figure 6 Referring to Embodiment 6, an example of a flowchart of the answer generation process in the artificial intelligence answer output device 10010 of Embodiment 6 of the present invention is shown. Specifically, a timeline progressing from top to bottom, a processing flow, and an example of answer output are shown. The output of the answer shown in the example can be performed via the display of the display unit 10011 of the artificial intelligence answer output device 10010 or the sound output of the sound output unit 1140.

[0322] Figure 6 In the example, first at time t0, the user inputs a request for a response from a large-scale language model via the touch panel, microphone 1139, or operation input unit 1107 of the artificial intelligence response output device 10010. The control unit 1110 of the artificial intelligence response output device 10010 acquires this user input (step 600). Next, at time t1, the control unit 1110 begins preparing to output a response using the response formatting statement database stored in the storage unit 1170, and begins outputting a response using the response formatting statement database (step 601). Figure 6 In the example, at time t2, the output of responses using the definite statement database has begun, as shown in the figure. The definite statement responses are being output, but the output is not yet complete. The "morning" in the figure represents the output of an article like "Good morning" along the way.

[0323] At time t3, before the response output from the response formula database is completed, the control unit 1110 generates a prompt word based on the user input obtained in step 600, and sends the generated prompt word to the large-scale language model on the network or the local large-scale language model (such as the local LLM processing unit 10028) equipped by the artificial intelligence response output device 10010, thus initiating a request for a response from the large-scale language model (step 602). Then, at time t4, before the response output from the response formula database is completed, the control unit 1110 begins to obtain a response from the large-scale language model (step 603).

[0324] At time t5, an example of the response output after the response output using the response-defined statement database has been completed is shown. For example, Figure 6 At time t5, an example is shown where the response output "Good morning. Today is ○ month ○ day." is displayed using a fixed phrase stored in the response fixed phrase database and date information stored in memory. Here, at time t4, before the response output using the response fixed phrase database is completed, the control unit 1110 has already started acquiring the response from the large-scale language model. Therefore, at time t6, after the response output using the response fixed phrase database is completed at time t5, the control unit 1110 starts outputting the response from the large-scale language model (step 604) after the response output using the response fixed phrase database is output (step 604). Then, at time t7, after the response output using the response fixed phrase database is output, the response from the large-scale language model is output. When the response output from the large-scale language model is completed, Figure 6 The output of the processing flow shown is complete (step 605).

[0325] Next, the present invention will be discussed. Figure 6 The effects of the processing flow shown are explained. Processing large-scale language models requires significant computational resources. Generally, inference requires fewer computational resources than training, but even when using GPUs (Graphics Processing Units) for inference, there are instances where it can take several to tens of seconds from the control unit requesting a response from the large-scale language model to receiving that response. This period is related to... Figure 6 The period from time t3 to time t4 is shown. Furthermore, from time t0 when user input occurs to time t4, the control unit 1110 cannot obtain the response output from the large-scale language model, and therefore cannot output a response from the large-scale language model to the user.

[0326] Therefore, if the processing flow does not exist Figure 6 In step 601, which involves preparing to use the answer formatting database and starting to output the answer using the answer formatting database, the user may have to wait for several to tens of seconds between the time t0 when the user input was made and the time t4. For example, if the AI ​​answer output device 10010 is configured as an AI assistant device or a role-playing conversation device, this waiting time may make the user feel uncomfortable.

[0327] In contrast, Figure 6 In the processing flow of Embodiment 6 of the present invention, the control unit 1110 begins processing based on the answer type statement database, which requires less computational resources compared to processing based on the large-scale language model, before obtaining answers from the large-scale language model. Therefore, the user will not be in a state of continuous waiting without an answer from the artificial intelligence answer output device 10010 during the period from time t0 to time t4. For the user, both the answer output based on the answer type statement database and the answer output from the large-scale language model are answers from the artificial intelligence answer output device 10010.

[0328] therefore, Figure 6 In the processing flow shown, by setting step 601 before step 603, the AI ​​response output device 10010 can perceive the user's response as being delivered earlier. This further reduces the unnaturalness to the user caused by long waiting times. Furthermore, by having the response from the large-scale language model continue to be output after the response based on the response pattern database in step 604, the user can perceive these outputs as a more natural series of responses.

[0329] According to the AI ​​response output device and AI response output system of Embodiment 6 described above, the user can shorten the response waiting time of the AI ​​response output device and further reduce the unnaturalness felt by the user.

[0330] <Example 7>

[0331] Embodiment 7 of the present invention improves upon the artificial intelligence response output device 10010 or artificial intelligence response output system illustrated in the figures of Embodiments 1 to 6. The differences from Embodiments 1 to 6 are described in Embodiment 7, and repeated descriptions of structures identical to those in these embodiments are omitted.

[0332] Similar to the embodiments described above, the AI ​​response output device 10010 can be referred to as an AI response output device, a role-based conversation device, an AI assistant device, an AI assistant display device, or an AI interface device. A system including the AI ​​response output device 10010 and a large-scale language model server can also be referred to as an AI response output system, a role-based conversation system, an AI assistant system, an AI assistant display system, or an AI interface system.

[0333] Here, in the AI ​​response output device 10010 or AI response output system, prompt words are generated based on user input requesting an answer from a large-scale language model, which is input via the AI ​​response output device 10010. These prompt words are then sent to the large-scale language model to obtain the answer. At this point, there is a possibility that the prompt words contain inappropriate content. Furthermore, there is a possibility that the answer from the large-scale language model contains inappropriate content. In these cases, a technique can be considered to replace the answer content from the large-scale language model with prescribed content to implement filtering and prevent the output of inappropriate answers.

[0334] exist Figure 7A The image shows an example of a replaced answer when filtering is implemented to prevent the output of inappropriate responses. For example, Figure 7A (1) is an example of a natural language article output as a replacement answer when the prompt word sent from the artificial intelligence answer output device 10010 to the large-scale language model contains a record requiring an answer to violence-related content, and when the answer generated by the large-scale language model contains violence-related content. Figure 7A (2) is an example of a natural language article output as a replacement answer when the prompt words sent from the artificial intelligence answer output device 10010 to the large-scale language model contain a record of the required answer to sexual content, and when the answer generated by the large-scale language model contains sexual content. Figure 7A (3) is an example of a natural language article output as a replacement answer when the prompt words sent from the artificial intelligence answer output device 10010 to the large-scale language model contain a record of a request to answer content that endangers public safety, and when the answer generated by the large-scale language model contains content that endangers public safety. Figure 7A (4) is an example of a natural language article output as a replacement answer when the prompt words sent by the AI ​​answer output device 10010 to the large-scale language model contain content that infringes on the privacy of a specific individual, and when the answer generated by the large-scale language model contains content that infringes on the privacy of a specific individual. In this way, the large-scale language model outputs a replacement article (replacement statement) to the AI ​​answer output device 10010 to filter and prevent the output of inappropriate answers, and the AI ​​answer output device 10010 can output the replacement article as an answer to the user.

[0335] However, there are also cases where it is not the best choice, such as... Figure 7A The alternative text shown is directly output to the user as an answer. For example, consider the case where, in the AI ​​answer output device 10010 or AI answer output system illustrated in Figures 1-6, the AI's large-scale language model is initially set with its own name, role, conversational characteristics, etc., so that the answer from the large-scale language model appears to play a role with a consistent personality. Figure 7A The alternative text, which identifies itself as AI (Artificial Intelligence) and explains the situation, is directly output as an answer to the user. In this case, even if the user has previously perceived the answers from the AI ​​answer output device 10010 or the AI ​​answer output system as answers from a character with a consistent personality, they will be surprised by the output. Figure 7A The user may realize that the previous series of answers were ultimately just AI (artificial intelligence) responses. In such a situation, the consistent personality of the character perceived by the user, constructed through the initial set of answers, is completely destroyed. Therefore, the AI ​​response output device 10010 or AI response output system of this embodiment performs actions and processes to output better answers to the user while implementing filtering to prevent large-scale language models from outputting inappropriate answers.

[0336] The specific actions and processes of the artificial intelligence answer output device 10010 or the artificial intelligence answer output system of this embodiment will be described below.

[0337] First of all, Figure 7B The diagram shows an example of a flowchart illustrating the process of preventing inappropriate responses in the AI ​​response output device 10010 or AI response output system of Embodiment 7.

[0338] First, in step 701, when user input is made via the touch panel, microphone 1139, or operation input unit 1107 of the artificial intelligence response output device 10010, the control unit 1110 generates a prompt word 1 such as a question or request based on the user input, and sends the prompt word 1 to a large-scale language model on the network, such as the large-scale language model server 19001 and / or the multimodal large-scale language model server 20001, or a local large-scale language model, such as the local LLM processing unit 10028 (S701). Next, in step S711, the large-scale language model that received the prompt word 1 performs an improper filtering execution determination process (S711) to determine whether improper filtering needs to be performed. Details of the improper filtering execution determination process will be described later.

[0339] Here, after determining whether improper filtering needs to be performed through the improper filtering process in step 711, if the large-scale language model determines that improper filtering needs to be performed, it sends an answer A containing natural language with the improper content filtered to the control unit 1110 of the artificial intelligence answer output device 10010. An example of an answer A containing natural language with the improper content filtered is as described above. Figure 7A The replacement article is shown. At this time, the large-scale language model of this embodiment sends an inappropriate filtering flag B to the control unit 1110 of the artificial intelligence answer output device 10010, indicating that the content is inappropriate and the answer has been filtered, in addition to or in place of the answer A containing natural language whose inappropriate content has been filtered. It can also be said that the inappropriate filtering flag B is a flag indicating that filtering processing has been performed on the answer. In addition, the expression "flag" in the embodiments of the present invention can also be referred to as "control information". Here, in step 702, the control unit 1110 of the artificial intelligence answer output device 10010, which receives the answer A containing natural language whose inappropriate content has been filtered and / or the inappropriate filtering flag B, performs processing corresponding to answer A based on the answer A containing natural language whose inappropriate content has been filtered and / or the inappropriate filtering flag B (S702).

[0340] On the other hand, if the large-scale language model determines, through the improper filtering determination process in step 711, that improper filtering is not required (i.e., the content is appropriate), it sends the generated normal answer C containing natural language to the control unit 1110 of the artificial intelligence answer output device 10010 without improper content filtering. At this time, the large-scale language model in this embodiment can send an improper filtering flag D indicating that there is no improper content to the control unit 1110 of the artificial intelligence answer output device 10010, in addition to the normal answer C containing natural language generated by the large-scale language model. Here, in step 703, the control unit 1110 of the artificial intelligence answer output device 10010, having received the normal answer C containing natural language and / or the improper filtering flag D indicating that there is no improper content generated by the large-scale language model, performs processing corresponding to answer C (S703).

[0341] Based on the above explanation Figure 7B The processing of the AI ​​response output device 10010 or AI response output system in the flowchart is capable of sending a flag from the large-scale language model to the AI ​​response output device 10010 indicating that the content is inappropriate and the response has been filtered, or indicating that the content is not inappropriate, in addition to or in place of the response from the large-scale language model.

[0342] Next, use Figure 7C illustrate Figure 7B This is an example of the improper filtering execution judgment process in step 711. Figure 7C The table showing the decision-making process is provided. Figure 7C In the example of judgment and processing shown, the two conditions of whether the prompt word contains inappropriate keywords and whether the generated data contains inappropriate data are combined to determine whether it is "inappropriate and needs to be filtered" or "not inappropriate and does not need to be filtered". Figure 7C In the example, if neither the prompt nor the generated data contains inappropriate keywords, it is judged as "No inappropriateness, no filtering required." Conversely, if either the prompt contains inappropriate keywords or the generated data contains inappropriate data, it is judged as "Inappropriate, filtering required."

[0343] Based on the above explanation Figure 7C The judgment and processing can better determine the possibility of outputting an inappropriate answer.

[0344] Next, use Figure 7D illustrate Figure 7B This is a specific example of an inappropriate filtering flag. Figure 7D Examples of improper filtering flags, such as Example (1) and Example (2), are shown in the figure.

[0345] First, in, for example Figure 7D In the example of the improper filtering flag in (1), only two states are shown: one is judged as "not improper" and the answer is "not filtered," and the other is judged as "improper" and the answer is "filtered." For example, it can be configured to use a 1-bit flag, where a "0" flag indicates a state of "not improper" and the answer is "not filtered." In this case, it can be configured to indicate a state of "improper" and the answer is "filtered" when the flag is "1."

[0346] Based on the above explanation Figure 7D The improper filtering flag (1) can better identify the state that is judged as "not improper" and the answer is "not filtered" and the state that is judged as "improper" and the answer is "filtered" using 1 bit of data. In addition, it can better judge the possibility of outputting an improper answer.

[0347] In contrast, Figure 7D In the example of the flag in (2), when the status is judged as "inappropriate" and the response is "filtered", the reason for being judged as "inappropriate" can be identified by the flag. Specifically, in the example of the inappropriate filtering flag in (2), it is configured to represent the eight states from "0" to "7" in decimal. When the inappropriate filtering flag in (2) is "0", it is the same state as in (1), so the repeated explanation is omitted.

[0348] Here, when the inappropriate filtering flag in (2) is "1" to "6", the flag indicates a state where the content is judged as "inappropriate" and the response is "filtered". The reason for the judgment as "inappropriate" can be identified by the value of the flag. For example, an inappropriate filtering flag of "1" indicates a state where the prompt or generated data contains content related to "violence", thus being judged as "inappropriate" and the response is "filtered". Similarly, an inappropriate filtering flag of "2" indicates a state where the prompt or generated data contains content related to "sex", thus being judged as "inappropriate" and the response is "filtered". Furthermore, an inappropriate filtering flag of "3" indicates a state where the prompt or generated data contains content that may "endanger public safety", thus being judged as "inappropriate" and the response is "filtered".

[0349] Additionally, an inappropriate filtering flag of "4" indicates that the prompt or generated data contains content that may infringe on "personal privacy," thus being deemed "inappropriate" and the response "filtered." Similarly, an inappropriate filtering flag of "5" indicates that the prompt or generated data contains content that promotes "discrimination," thus being deemed "inappropriate" and the response "filtered." Furthermore, an inappropriate filtering flag of "6" indicates that the prompt or generated data contains content that raises "academic concerns" (e.g., content that promotes academic misconduct or is unscientific), thus being deemed "inappropriate" and the response "filtered." Figure 7D Example (2) also provides a case where the improper filter flag is "7", which indicates a situation that does not conform to the cases where the improper filter flags are "0" to "6" as described above.

[0350] in addition, Figure 7D Example of the flag (2) is constructed using decimal representation of the eight states "0" to "7". Therefore, an improper filtering flag can be constructed using a 3-bit flag.

[0351] Based on the above explanation Figure 7D The improper filtering flag of (2) can not only identify the state that is judged as "not improper" and the answer is "not filtered" and the state that is judged as "improper" and the answer is "filtered", but also identify the reason for being judged as "improper".

[0352] In addition, Figure 7B The LLM shown is a large-scale language model on a network, such as a large-scale language model server 19001 and / or a multimodal large-scale language model server 20001. Figure 7D The flags described herein are sent from a large-scale language model to an artificial intelligence response output device 10010 via a network such as the Internet 19000. The control unit 1110 is able to process and use the improper filtering flags received by the artificial intelligence response output device 10010.

[0353] In addition, Figure 7B The LLM shown is a local large-scale language model in the case of a local LLM processing unit 10028, etc. Figure 7D The flag described herein is sent from the local LLM processing unit 10028 to the control unit 1110 within the artificial intelligence response output device 10010. The control unit 1110 is able to use the received improper filtering flag in processing.

[0354] Next, use Figure 7E This describes an example of the control unit 1110 of the AI ​​response output device 10010 that received an inappropriate filtering flag. Figure 7E Examples of databases, where for Figure 2H Each of the multiple roles shown prepares and saves the answer corresponding to the aforementioned improper filtering flag. The role ID and role name are the same as in Examples 2, 3, or 4, so repeated descriptions are omitted.

[0355] In Embodiments 2, 3, or 4, initial setting prompts, conversation history, or correspondences with user IDs are used to make users perceive that each role has a consistent personality. At this point, it is considered that inappropriate filtering of responses from large-scale language models has occurred; for example, the AI ​​response output device 10010 outputs to the user... Figure 7A The replacement articles after improper filtering are shown in (1) to (4).

[0356] Here, Figure 7A Articles (1) to (4) contain expressions such as "I am AI", and Figure 7A The output contains expressions such as "programming" in (1), (2), and (4). Therefore, users who see this output will strongly realize that... Figure 2H The multiple characters shown are artificial intelligence programmed by software, and may no longer be able to perceive the consistent personalities of the characters built up to this point.

[0357] Therefore, in this embodiment Figure 7E In the example, even if the filtered natural language response generated by the large-scale language model is Figure 7A The content of (1) to (4) is also controlled by the control unit 1110 of the artificial intelligence answer output device 10010. Figure 7E The database shown selects articles (response statements) corresponding to each role and inappropriate filtering flag as answers, and outputs these answers to the user. That is, in this control of control unit 1110, it does not directly output natural language response statements generated by a large-scale language model (e.g., ...) to the user. Figure 7A Instead of the contents of (1) to (4), replace them with Figure 7E The content stored in the database is then output to the user. Figure 7E The database shown can be stored in the storage unit 1170 in Figure 3.

[0358] For example, if the character "Koto" with character ID 1 is displayed on the AI ​​response output device 10010, then... Figure 7B In step 702, the control unit 1110 receives... Figure 7D If the improper filtering flag (indicating that the prompt word or generated data contains "violence" related content) shown in (2) is 1, the control unit 1110 performs control and refers to the data stored in the storage unit 1170. Figure 7E The database selects a response statement like "Koto opposes violence." as the output to the user. By controlling the output of such a response statement, it contrasts with direct output... Figure 7A Compared to the response statement generated by the large-scale language model shown in (1), the user can feel that the response matches the personality of the character displayed on the artificial intelligence response output device 10010, which is preferred.

[0359] Additionally, if, for example, the AI ​​response output device 10010 displays the character "Koto" with character ID 1, it can... Figure 7B In step 702, the control unit 1110 receives... Figure 7D The case shown in (2) is an inappropriate filtering flag with a value of 2 (a flag indicating that the prompt or generated data contains "sex" related content), and the control unit 1110 received... Figure 7D In the case of an inappropriate filtering flag with a value of 3 as shown in (2) (indicating that the prompt or generated data contains content that may "endanger public safety"), the control unit 1110 performs control accordingly, referring to the data stored in the storage unit 1170. Figure 7E The database outputs corresponding response statements to the user. In this way, the user can feel that the response matches the personality of the character "Koto" displayed on the AI ​​response output device 10010, which is even better.

[0360] Similarly, when the character "Tom" with character ID 2 is displayed on the AI ​​response output device 10010, it is possible to... Figure 7B In step 702, the control unit 1110 receives... Figure 7D The control unit 1110 received a value of 1 for the improper filtering flag shown in (2) (a flag indicating that the prompt word or generated data contains "violence" related content). Figure 7D The case shown in (2) is an inappropriate filtering flag with a value of 2 (a flag indicating that the prompt or generated data contains "sex" related content), and the control unit 1110 received... Figure 7D In the case of an inappropriate filtering flag with a value of 3 as shown in (2) (indicating that the prompt or generated data contains content that may "endanger public safety"), the control unit 1110 performs control accordingly, referring to the data stored in the storage unit 1170. Figure 7E The database outputs corresponding response statements to the user. In this way, the user can feel that the response matches the personality of the character "Tom" displayed on the AI ​​response output device 10010, which is even better.

[0361] Similarly, when the character "Necco" with character ID 3 is displayed on the AI ​​response output device 10010, it is possible to... Figure 7B In step 702, the control unit 1110 receives... Figure 7D The control unit 1110 received a value of 1 for the improper filtering flag shown in (2) (a flag indicating that the prompt word or generated data contains "violence" related content). Figure 7D The case shown in (2) is an inappropriate filtering flag with a value of 2 (a flag indicating that the prompt or generated data contains "sex" related content), and the control unit 1110 received... Figure 7D In the case of an inappropriate filtering flag with a value of 3 as shown in (2) (indicating that the prompt or generated data contains content that may "endanger public safety"), the control unit 1110 performs control accordingly, referring to the data stored in the storage unit 1170. Figure 7E The database outputs corresponding response statements to the user.

[0362] in addition, Figure 7E In the example, the character "Necco" received Figure 7D In the case of an inappropriate filtering flag with a value of 1 to 6 as shown in (2), the control unit 1110 controls the output to always display a response such as "Necco doesn't quite understand, meow~". By adopting this method, the artificial intelligence response output device 10010 can give the user the impression that the character "Necco" has a personality that pretends not to understand inappropriate prompts and responses in order to avoid responding. In this case, the user can also feel that the response matches the personality of the character "Necco" displayed on the artificial intelligence response output device 10010, which is even more preferable.

[0363] Based on the above instructions, the following was used. Figure 7E With the control of the database, even if the answer from the large-scale language model is filtered by the large-scale language model, it can be replaced with an answer that matches the personality of the character displayed on the AI ​​answer output device and then output, which is better for the user.

[0364] In addition, the above explanation Figure 7E The database of answer statements is stored in the storage unit 1170, and the control unit 1110 of the artificial intelligence answer output device 10010 can use this database. However, Figure 7E The database of answer statements can also be set on either the large-scale language model server 19001 or the large-scale language model server 20001.

[0365] In this case, the control unit of the large-scale language model server 19001 or the control unit of the large-scale language model server 20001 can obtain the role ID of the role displayed on the artificial intelligence response output device 10010, and replace the response statement generated by the large-scale language model of each server with... Figure 7E The system retrieves the corresponding answer from the answer database and then sends it to the AI ​​answer output device 10010. In this case, the AI ​​answer output device 10010 outputs the answer to the user.

[0366] Using this method, the AI ​​response output device 10010 does not have... Figure 7E In the case of a database of answer statements, it is also possible to use Figure 7E This is an AI-powered answer output system that generates answers from a database of answer statements. In this case, within the large-scale language model server 19001 or 20001, communication between their respective large-scale language models and their respective control units is possible. Figure 7D The symbol shown.

[0367] <Example 8>

[0368] Embodiment 8 of the present invention improves upon the artificial intelligence response output device 10010 or artificial intelligence response output system illustrated in the figures of Embodiments 1 to 7. The differences from Embodiments 1 to 7 are described in Embodiment 8, and repeated descriptions of structures identical to those in these embodiments are omitted.

[0369] Similar to the embodiments described above, the AI ​​response output device 10010 can be referred to as an AI response output device, a role-based conversation device, an AI assistant device, an AI assistant display device, or an AI interface device. A system including the AI ​​response output device 10010 and a large-scale language model server can also be referred to as an AI response output system, a role-based conversation system, an AI assistant system, an AI assistant display system, or an AI interface system.

[0370] In the artificial intelligence response output device 10010 or artificial intelligence response output system of Embodiment 8 of the present invention, the prompt words input by the user are transformed before being sent to a large-scale language model.

[0371] use Figure 8A This section describes an example of a flowchart illustrating a series of processes including the aforementioned transformation process in the artificial intelligence response output device 10010 or artificial intelligence response output system of Embodiment 8 of the present invention. First, in step 801, when user input is made via the touch panel, microphone 1139, or operation input unit 1107 of the artificial intelligence response output device 10010, the control unit 1110 generates prompt words 1 such as questions or requests based on the user input (S801).

[0372] Next, in step 802, the control unit 1110 of the artificial intelligence response output device 10010 transforms the prompt word 1 to generate the prompt word 2, and sends the generated prompt word 2 to a large-scale language model on the network, such as the large-scale language model server 19001 and / or the multimodal large-scale language model server 20001, or to a local large-scale language model, such as the local LLM processing unit 10028 (S802). Details of this transformation process will be described later.

[0373] Next, in step 811, the large-scale language model that received the prompt word 2 performs response generation processing based on the prompt word 2 to generate a response A containing natural language, and sends the response A to the control unit 1110 of the artificial intelligence response output device 10010 (S811).

[0374] Next, in step 803, the control unit 1110 of the AI ​​response output device 10010, which receives the response A containing natural language, performs processing corresponding to response A (S803). The processing corresponding to response A in step 803 refers to processing such as: processing the natural language text contained in the user-output response A, processing that controls various parts of the AI ​​response output device 10010 to perform various actions based on the content of response A, or response statement replacement processing using a database as described in the above embodiments, etc.

[0375] Next, use Figure 8B illustrate Figure 8A An example of the details of the prompt word transformation process in step 802. Figure 8B Examples of different transformation processes are shown in Example 1, Example 2, and Example 3. Figure 8B The "user prompt words" in the table are prompt words generated based on user input via the touch panel, microphone 1139, or operation input unit 1107 of the artificial intelligence response output device 10010, corresponding to... Figure 8A The first prompt word. Additionally... Figure 8B The "transformed prompt words" in the table are prompt words generated by the control unit 1110 of the artificial intelligence response output device 10010 after transforming the "user prompt words," corresponding to... Figure 8A The prompt word 2. The following explains the transformation processes in Examples 1, 2, and 3.

[0376] First of all, Figure 8B In the transformation method of Example 1, auxiliary natural language is added to the natural language of the user prompt word, i.e., auxiliary natural language transformation. For example, when the prompt word 1 is "Please calculate 1+5. Please calculate 10-3. Please calculate the sum of the two results.", the transformation process in Example 1 is as follows: the original natural language of prompt word 1, "Please add an explanation of the answer process." is added as auxiliary natural language. Through this transformation process, prompt word 2 is sent to the large-scale language model in a state that includes auxiliary natural language such as "Please add an explanation of the answer process," which was not included in the user input prompt word 1.

[0377] Here, depending on the large-scale language model, there are cases where the accuracy of the response is improved compared to the case where no such instruction is received due to the aiding prompt words in Example 1. Therefore, even if not intended by the user, the transformation process in Example 1 can improve the response accuracy of such a large-scale language model, more preferably. Furthermore, the content of the aiding natural language is pre-stored in the storage unit 1170, for example, as a fixed phrase, and the control unit 1110 can read and use this fixed phrase of the aiding natural language from the storage unit 1170 during the transformation process.

[0378] exist Figure 8B In the transformation method of Example 2, the auxiliary natural language in the transformation process of Example 2 is changed to English. In the transformation process, even if the natural language of the user prompt word (i.e., prompt word 1) is not English, English auxiliary natural language is added to the non-English natural language of that user prompt word. Figure 8B Example 2 in the table illustrates the case where the same prompt word 1 as in Example 1 is input. For example, the transformation process in Example 1 adds "Please add explanation of the process" in the same language as the user prompt word 1. However, in the transformation process in Example 2, even if the natural language of the user prompt word is not English, the English phrase "Please add explanation of the process" is added as auxiliary natural language.

[0379] Most large-scale language models are trained using existing natural language information accessible via the internet. Because English natural language information constitutes a large proportion of this existing natural language information, high-performance large-scale language models with large training scales tend to have higher adaptability to English prompts and higher response accuracy.

[0380] Therefore, even if the meaning of the auxiliary natural language is the same, when adding English auxiliary natural language as in Example 2, the possibility of obtaining a more accurate answer from the large-scale language model is higher than when adding non-English natural language, and thus it is preferred. Similarly, in Example 2, the content of the auxiliary natural language is, for example, pre-stored as a fixed statement in the storage unit 1170, and the control unit 1110 reads the fixed statement of the auxiliary natural language from the storage unit 1170 and uses it during transformation processing.

[0381] Next, in Figure 8B In the transformation method of Example 3, when the natural language of the user prompt word, i.e., prompt word 1, is not English, it is transformed into English. The transformation of a non-English natural language into English natural language is a so-called translation function. The translation function can utilize existing technology. For example, a program (software) with translation function can be stored in the storage unit 1170 of the artificial intelligence response output device 10010, loaded into the memory 1109, and executed by the control unit 1110. Alternatively, under the control of the control unit 1110, the natural language of the non-English prompt word 1 can be sent to an external translation server via the communication unit 1132, causing the external translation server to perform the process of transforming the natural language of prompt word 1 into English.

[0382] Control unit 1110 can receive the converted English natural language from the server via communication unit 1132 and use it as... Figure 8B Example 3 shows prompt word 2. As a specific example of Example 3, when the natural language of the user prompt word, i.e., prompt word 1, is non-English, such as "Please calculate 1+5. Please calculate 10-3. Please calculate the sum of both results.", the transformed prompt word 2 is in English: "Please calculate 1+5. Please calculate 10-3. Please calculate the sum of both of these results."

[0383] As described in Example 3, the advantages of converting non-English natural language in the prompts to English natural language are largely the same as those explained in Example 2. That is, because English natural language information constitutes a large proportion of existing natural language information, high-performance large-scale language models with large training scales tend to have higher adaptability and higher answer accuracy when using English prompts. Therefore, even if the meaning of the prompts is the same, compared to using non-English prompts, converting them to English prompts as described in Example 3 increases the likelihood of obtaining a higher accuracy answer from the large-scale language model, making it a more preferable approach.

[0384] Based on the above explanation Figure 8B The natural language transformation processing of the prompt words shown in the examples increases the likelihood of obtaining more accurate answers from large-scale language models, and is therefore preferred.

[0385] Here, there also exists a large-scale language model that supports an API in the following format: the data region for the API's prompt words not only has a user prompt word region (data region storing prompt words generated based on user input), but also a system prompt word region (data region different from the user prompt word region) that stores system prompt words. Figure 8C This illustrates the situation when sending prompt words to such a large-scale language model. Figure 8A This is an example of the details of the prompt word transformation process in step 802. Using... Figure 8C The transformation process described Figure 8B Similarly, the transformation process is performed based on the user prompt word (prompt word 1) generated based on the user input to generate the transformed prompt word (prompt word 2). Figure 8C Examples of different transformation processes, such as Example 1 and Example 2, are shown in the figure.

[0386] Figure 8C The transformation process in Example 1 is related to... Figure 8B Similarly, in Example 1, the addition of prompts such as "Please provide further explanation of your answer process" is processed using natural language transformation. However, Figure 8C Example 1 and Figure 8B Unlike Example 1, the additional prompt is not stored in the user prompt area, but in the system prompt area.

[0387] Here, the content of the prompts in the user prompt area is the text (sentence) itself, which is read and processed by the large-scale language model. Therefore, the accuracy of the response to these prompts varies considerably depending on the performance of the large-scale language model. The level of understanding of these prompts itself varies with the performance of the large-scale language model, making it difficult to improve the reliability of the responses. In contrast, the system prompt area can store various contents that are pre-agreed upon between the sending and receiving sides of the API. Thus, the instructions stored in the system prompt area are obtained by the large-scale language model and its peripheral software via the API, and used to perform pre-agreed processing.

[0388] For example, Figure 8C In Example 1, the processing in the peripheral software of the large-scale language model can be pre-defined in the system prompt area as containing "Please add instructions for the response process." Even if the user prompt area contains the natural language phrase "Please add instructions for the response process," but the performance of the large-scale language model prevents it from properly understanding the meaning of the added prompt and performing the required processing, the following can be used: Figure 8C The fact that the system prompts are stored in a pre-defined area increases the likelihood that the requested processing can be performed more reliably.

[0389] Furthermore, the system prompt area can use not only natural language but also software code to record instructions. For example, instructions can be recorded in Python software code within the system prompt area, using JSON format as the metadata format required for the instructions. The Python software code can be considered as pre-defined code for the processing to be executed. The JSON metadata is a pre-defined data format for the meaning of the expression. Thus, Figure 8C In this process, transformation processing is performed to append instructions to the system prompt word area in a pre-defined format, thereby increasing the likelihood of obtaining more accurate answers from large-scale language models, and is therefore preferred.

[0390] Figure 8C Transformation processing of Example 2 and Figure 8C Compared to the transformation process in Example 1, for non-English user prompts (prompt 1), the natural language in the user prompt area remains non-English, but the content of the prompts appended to the system prompt area is changed to English natural language. Other processing is the same as... Figure 8C The transformation process is the same as in Example 1, so repeated explanations are omitted. Furthermore, the advantages of adding English prompts to non-English user prompts are... Figure 8B The same explanation applies, so repeated explanations are omitted. As mentioned above, in Figure 8C In the transformation process of Example 2, by recording the content of the prompt words appended to the system prompt word region in English, the possibility of obtaining a more accurate answer from a large-scale language model can be increased, and therefore it is preferred.

[0391] <Example 9>

[0392] Embodiment 9 of the present invention improves upon the artificial intelligence response output device 10010 or artificial intelligence response output system illustrated in the figures of Embodiments 1 to 8. The differences from Embodiments 1 to 8 are described in Embodiment 9, and repeated descriptions of structures identical to those in these embodiments are omitted.

[0393] Similar to the embodiments described above, the AI ​​response output device 10010 can be referred to as an AI response output device, a role-based conversation device, an AI assistant device, an AI assistant display device, or an AI interface device. A system including the AI ​​response output device 10010 and a large-scale language model server can also be referred to as an AI response output system, a role-based conversation system, an AI assistant system, an AI assistant display system, or an AI interface system.

[0394] In the AI ​​response output device 10010 or AI response output system of Embodiment 9 of the present invention, prompt words are sent to multiple different large-scale language models to obtain the responses of each of the multiple different large-scale language models, and processing is performed based on the responses of each of the models.

[0395] use Figure 9A This describes an example of a flowchart illustrating a series of processes including the aforementioned transformation process in the artificial intelligence response output device 10010 or artificial intelligence response output system according to Embodiment 9 of the present invention. First, in step 901, when user input is made via the touch panel, microphone 1139, or operation input unit 1107 of the artificial intelligence response output device 10010, the control unit 1110 generates a prompt word 1 such as a question or request based on the user input and sends the prompt word 1 to the first large-scale language model (S901). Here, the first large-scale language model can be any of the following: a large-scale language model on a network such as a large-scale language model server 19001 or a multimodal large-scale language model server 20001, or a local large-scale language model such as a local LLM processing unit 10028.

[0396] Next, in step 911, the first large-scale language model that received the prompt word 1 performs response generation processing based on the prompt word 1 to generate a response A containing natural language, and sends the response A generated by the first large-scale language model to the control unit 1110 of the artificial intelligence response output device 10010 (S911). Next, in step 902, the control unit 1110 receives the response A from the first large-scale language model, generates the prompt word 2 based on the received response A, and sends the prompt word 2 to the second large-scale language model (S902). In addition, the second large-scale language model can be a large-scale language model that is different from the first large-scale language model, such as a large-scale language model on the network, such as a large-scale language model server 19001 or a multimodal large-scale language model server 20001, or a local large-scale language model, such as a local LLM processing unit 10028.

[0397] Next, in step 921, the second large-scale language model that received the prompt word 2 performs response generation processing based on the prompt word 2 to generate a response B containing natural language, and sends the response B generated by the second large-scale language model to the control unit 1110 of the artificial intelligence response output device 10010 (S921).

[0398] Next, in step 903, the control unit 1110 receives answer B from the second large-scale language model. The control unit 1110 of the artificial intelligence answer output device 10010 performs processing (S903) based on both answer A received from the first large-scale language model and answer B received from the second large-scale language model. The processing based on answer A and answer B in step 903 may include: comparing answer A and answer B and outputting either one; performing judgment processing using answer B and outputting an answer that includes both answer A and answer B; performing judgment processing using answer B and directly outputting answer A; or performing judgment processing using answer B and outputting an answer with different content instead of answer A, etc.

[0399] Here, use Figure 9B illustrate Figure 9A This is an example of the details of the prompt word 1 sent to the first large-scale language model in step 901, the response A sent by the first large-scale language model to the control unit 1110 in step 911, the prompt word 2 generated by the control unit 1110 and sent to the second large-scale language model in step 902, the response B sent by the second large-scale language model to the control unit 1110 in step 921, and the processing performed by the control unit 1110 based on the responses A and B in step 903.

[0400] Figure 9B Examples 1 and 2 are shown in the figure. Figure 9B The "prompt word 1" in the table corresponds to Figure 8A Prompt word 1. This prompt word is generated based on user input via the touch panel, microphone 1139, or operation input unit 1107 of the artificial intelligence response output device 10010. Figure 9B The table shows "Answer A from the first LLM" as the natural language-containing answer generated by the first large-scale language model based on cue word 1, corresponding to... Figure 9A Answer A. Figure 9B The "cue word 2 for the second LLM" in the table is a cue word generated by the control unit 1110 based on the response A received from the first large-scale language model, corresponding to... Figure 9A The prompt word 2. Figure 9B The "response B from the second LLM" in the table is the natural language-containing response generated by the second large-scale language model based on cue word 2, corresponding to... Figure 9A Answer B. Figure 9B The "processing based on both answer A and answer B" in the table refers to the processing performed by the control unit 1110 based on answer A received from the first large-scale language model and answer B received from the second large-scale language model.

[0401] First of all, Figure 9B Example 1 will be used as an example. Example 1 shows, for instance, how the control unit 1110 sends the prompt word 1, "Please calculate X+Y," generated based on user input, to the first large-scale language model. In order to... Figure 9B The explanation states that "X" and "Y" are replaced with letters, but are actually numbers. Next, an example is shown where a first large-scale language model generates an answer A: "The result of calculating X + Y is Z." In order to... Figure 9B The explanation states that "Z" is represented by a letter, but it is actually a number.

[0402] then, Figure 9B In Example 1, the control unit 1110, which received the answer A, generates a prompt word 2 based on the prompt word "Please answer, for the prompt word "Please calculate X+Y." The prompt word "The result of calculating X+Y is Z. Is it correct? If correct, please answer in the following format: "0: Correct" If incorrect, please answer in the following format: "1: Incorrect" "The reason" and sends it to the second large-scale language model.

[0403] Regarding prompt word 2 in Example 1, new prompt words can be created by inserting prompt word 1 and answer A into the underlined parts of the articles in the table. This process can also be expressed as processing prompt word 1 and answer A with attached articles. Figure 9B In Example 1, prompt word 2 is inserted between the text that asks for verification of whether answer A is the correct answer to prompt word 1 and returns the verification result in the prescribed format.

[0404] Additionally, in the generation and processing of prompt word 2 in Example 1, the output statement to the user is: "Please answer, for the prompt word 'Please calculate X+Y.'" The response is: "The result of calculating X+Y is Z." The user then asks: "Is this correct? If correct, please answer in the following format: '0: Correct.' If incorrect, please answer in the following format: '1: Incorrect'" followed by the reason. Figure 9B The underlined parts in the table indicate whether the prompt words "0: Correct" or "Incorrect" are correct. If correct, please answer in the following format: "0: Correct". If incorrect, please answer in the following format: "1: Incorrect". The part with "reason" is a fixed statement.

[0405] By inserting prompt word 1 and the content of answer A between these fixed statements, prompt word 2 can be generated. Information about these fixed statements is pre-stored in storage unit 1170, and control unit 1110 reads and uses it from storage unit 1170. In this case, the prompt word 2 generation process in Example 1 can be considered as a process of generating prompt word 2 by combining the content of answer A with the fixed statements read by control unit 1110 from storage unit 1170.

[0406] Here, in Figure 9B In the table, the column corresponding to Example 1 under "Processing based on both answer A and answer B" records the cases in Example 1 where the second large-scale language model answered the prompt word 2 as "0: correct" and "1: incorrect" with explanations.

[0407] Specifically, when the second large-scale language model's response B to prompt word 2 is "0: correct", the control unit 1110 outputs "The calculation result of X+Y is Z" to the user. That is, in this example, the control unit 1110 processes the data and, based on the fact that the second large-scale language model's response B is "0: correct", determines that the content of the first large-scale language model's response A is correct, and as a result, directly outputs the content of response A to the user. This processing can be considered as directly outputting response A after performing judgment processing on response B.

[0408] Next, when the second large-scale language model's response B to prompt word 2 is "1: Incorrect" and the reason given, the control unit 1110 can output "Don't know." to the user. That is, in this example, the control unit 1110 processes the data and, based on the fact that the second large-scale language model's response B contains "1: Incorrect," determines that the content of the first large-scale language model's response A is incorrect. As a result, it outputs neither the content of response A nor the content of response B to the user, but instead outputs "Don't know." In this case, the control unit 1110 can control the output of the artificial intelligence response output device 10010 with the policy of not outputting potentially incorrect responses to the user. This processing can be considered as processing that uses response B for judgment processing and outputs a different response instead of the content of response A. In addition, the underlined "Don't know." is pre-stored as a fixed statement in the storage unit 1170, and the control unit 1110 can read and use it from the storage unit 1170. In this case, the processing performed by the control unit 1110 based on both answer A and answer B can be considered as the processing of the user outputting a fixed statement read from the storage unit 1170 instead of outputting answer A or answer B.

[0409] Furthermore, as a variation of the processing in the case where the second large-scale language model's response B to prompt word 2 is "1: Incorrect" and an explanation of the reason, the control unit 1110 can output to the user, "The calculation result of X+Y is Z. However, some opinions point out that this result is incorrect." That is, in this example, the control unit 1110 processes the data based on the fact that the second large-scale language model's response B contains "1: Incorrect," and determines that the content of the first large-scale language model's response A may be incorrect. As a result, it outputs the content of response A to the user, but appends the underlined fixed statement "However, some opinions point out that this result is incorrect." In this case, the control unit 1110 can control the output of the artificial intelligence response output device 10010 by outputting a response to the user even if the response may be incorrect, but also informing the user that some opinions point out that the result may be incorrect. This processing can be considered as processing that outputs a response containing both response A and response B after performing judgment processing using response B.

[0410] Furthermore, a pre-defined statement such as "However, some argue that the result is incorrect" is stored in the storage unit 1170, and the control unit 1110 can read and use it from the storage unit 1170. In this case, it can be considered that the processing performed by the control unit 1110 based on both answer A and answer B is a process of combining the content of answer A with the pre-defined statement read by the control unit 1110 from the storage unit 1170 to generate output to the user.

[0411] Next, regarding Figure 9B Example 2 will be used to illustrate this. Example 2 shows, for instance, an example where the control unit 1110 sends the prompt word 1, "Is X a prime number?", generated based on user input, to the first large-scale language model. In order to... Figure 9B The explanation states that "X" is replaced with a letter, but it is actually a number.

[0412] Next, an example is shown where the first large-scale language model generates an answer A such as "X is a prime number." Figure 9B In Example 2, the control unit 1110, which received the answer A, generates a prompt word 2 based on the prompt A: "Please answer whether the answer 'X is a prime number?' is appropriate. If appropriate, please answer in the following format: '0: appropriate'. If inappropriate, please answer in the following format: '1: inappropriate'. The reason is" and sends it to the second large-scale language model.

[0413] Here, regarding prompt word 2 in Example 2, new prompt words can be created by inserting prompt word 1 and answer A into the underlined parts of the articles in the table. This process can also be expressed as the processing of additional articles for prompt word 1 and answer A. Figure 9B In Example 2, prompt word 2 is inserted between the text that asks for verification of whether answer A is an appropriate answer to prompt word 1 and returns the verification result and reason in the prescribed format.

[0414] Additionally, in the generation and processing of prompt word 2 in Example 2, is it appropriate to answer the question, "Is X a prime number?" by replying "X is a prime number?"? If appropriate, please answer in the following format: "0: appropriate". If inappropriate, please answer in the following format: "1: inappropriate" "reason". In such output statements to the user, Figure 9B The underlined part in the table, "Please answer whether the prompt word "" is appropriate, "" or "" is appropriate. If appropriate, please answer in the following format: "0: Appropriate" If inappropriate, please answer in the following format: "1: Inappropriate" The part with "reason" is a fixed statement.

[0415] By inserting prompt word 1 and the content of answer A between these fixed statements, prompt word 2 can be generated. Information from these fixed statements is pre-stored in storage unit 1170, and control unit 1110 reads and uses it from storage unit 1170. In this case, the prompt word 2 generation process in Example 2 can be considered as a process of generating prompt word 2 by combining the content of answer A with the fixed statements read by control unit 1110 from storage unit 1170.

[0416] Here, in Figure 9B In the table, the column corresponding to Example 2 under "Processing based on both answer A and answer B" records the cases in Example 2 where the second large-scale language model responded to the prompt word 2 as "0: appropriate" and "1: inappropriate" with explanations.

[0417] Specifically, when the second large-scale language model's response B to prompt word 2 is "0: appropriate", the control unit 1110 outputs "X is a prime number" to the user. That is, in this example, the control unit 1110 processes the data and, based on the fact that the second large-scale language model's response B is "0: appropriate", determines that the content of the first large-scale language model's response A is appropriate. As a result, it directly outputs the content of response A to the user. This processing can be considered as directly outputting response A after performing judgment processing on response B.

[0418] If the second large-scale language model's response B to prompt word 2 contains "1: inappropriate" and includes "academic concerns" as its reason, the control unit 1110 can output to the user, "The question or its answer may be inappropriate, therefore it cannot be answered. The reason is academic concerns." That is, in this example, the control unit 1110 processes the data based on the fact that the second large-scale language model's response B contains "1: inappropriate" and includes "academic concerns" as its reason, and judges that the content of the first large-scale language model's response A is inappropriate. As a result, instead of outputting the content of response A to the user, it outputs "The question or its answer may be inappropriate, therefore it cannot be answered. The reason is academic concerns." In this case, the control unit 1110 can control the output of the artificial intelligence response output device 10010 with the response policy of not outputting potentially inappropriate responses to the user. This processing can be considered as processing that uses response B for judgment processing and outputs a response with different content instead of response A.

[0419] Additionally, in Example 2, the output statement to the user, such as "This question or its answer may be inappropriate and therefore cannot be answered. The reason is academic concerns," Figure 9B The underlined portions in the table, "The question or its answer may be inappropriate and therefore cannot be answered. The reason is..." and ".", are pre-defined statements. By inserting the answer corresponding to "the reason" contained in answer B between them, an output statement for the user can be generated. The pre-defined statements "The question or its answer may be inappropriate and therefore cannot be answered. The reason is..." and "." are pre-stored in the storage unit 1170, and the control unit 1110 can read and use them from the storage unit 1170. In this case, it can be considered that the processing performed by the control unit 1110 based on both answer A and answer B is a process of combining the content of answer B with the pre-defined statement read by the control unit 1110 from the storage unit 1170 to generate output for the user.

[0420] In the above use Figure 9A and Figure 9B In the AI ​​response output device 10010 or AI response output system of Embodiment 9 of the present invention, prompt words are sent to multiple different large-scale language models, and responses from each of the multiple large-scale language models are obtained. Processing is then performed based on these responses. Therefore, unlike directly outputting a response from a single large-scale language model to the user, it is possible to pre-confirm whether the response to be output to the user is correct or appropriate, and based on this, more optimized response processing is achieved according to a response strategy.

[0421] In addition, in use Figure 9A and Figure 9B In the AI ​​response output device 10010 or AI response output system of Embodiment 9 of the present invention, it can also be said that response A from a first large-scale language model is designated as the main response, and response B from a second large-scale language model is designated as the secondary response. The main response is, in principle, used to consider outputting to the user, while the secondary response determines whether response A can be directly output to the user or whether additional prompts or other auxiliary information can be provided to the user. In this case, the first large-scale language model can be considered the large-scale language model used for the main response. Furthermore, the second large-scale language model can be considered the large-scale language model used for the secondary response.

[0422] Here, we consider a scenario where both the first and second large-scale language models are large-scale language models on the internet, and the AI ​​response output device 10010 communicates with these large-scale language models via an API. In this case, as described in the embodiments above, when using these large-scale language models, there may be a requirement to pay API usage fees based on the number of lexical units processed. In this case, for example, by implementing the following controls, it is possible to appropriately maintain response accuracy while saving API usage fees.

[0423] Specifically, the data format of the prompt words can include the system prompt word area described in Embodiment 8 above. Parameters for limiting the number of tokens in the response can be transmitted within this system prompt word area. For example, parameters such as max_new_tokens (controlling the maximum number of tokens in the response) and / or min_new_tokens (controlling the minimum number of tokens) can be used. These can be referred to as output token number limit parameters.

[0424] In the AI ​​response output device 10010 or AI response output system of Embodiment 9 of the present invention, the control unit 1110 stores a first output lexical number limitation parameter in the system prompt word area of ​​prompt word 1 (which is used to request the main response, i.e., the first large-scale language model to give response A), and stores a second output lexical number limitation parameter in the system prompt word area of ​​prompt word 2 (which is used to request the secondary response, i.e., the second large-scale language model to give response B). Furthermore, the control unit 1110 sets the number of lexical units specified by the second output lexical number limitation parameter to a smaller number than the number of lexical units specified by the first output lexical number limitation parameter. The first output lexical number limitation parameter is stored in the system prompt word area of ​​prompt word 1 (which is used to request the main response, i.e., the first large-scale language model to give response A), and the second output lexical number limitation parameter is stored in the system prompt word area of ​​prompt word 2 (which is used to request the secondary response, i.e., the second large-scale language model to give response B).

[0425] Answer A contains the main content of the user response, and its length is also important, thus allowing for a larger number of lexical units and higher API usage fees. In contrast, answer B determines whether answer A can be directly output to the user, and its length is not as crucial. Therefore, by ensuring sufficient lexical units for answer A and conserving lexical units for answer B, it is possible to maintain appropriate answer accuracy while saving on API usage fees.

[0426] Furthermore, the comparison of the output token count limits mentioned above is based on the same type of output token count limit parameters. That is, if the output token count limit parameter is `max_new_tokens`, the comparison is made between tokens specified by `max_new_tokens`. If the output token count limit parameter is `min_new_tokens`, the comparison is made between tokens specified by `min_new_tokens`. When setting `max_new_tokens` and `min_new_tokens` in the system prompt word areas for both prompt word 1 and prompt word 2, the values ​​for `max_new_tokens` and `min_new_tokens` can be set such that the parameter value for prompt word 2 is smaller than the parameter value for prompt word 1.

[0427] Furthermore, the technology of this embodiment can provide better AI-powered response output technology. It is expected that such AI-powered response output technology will be adopted in higher-quality, more reliable infrastructure. By adopting this technology in infrastructure, it can help support economic development and individual well-being that are affordable and equitable for all. This, in turn, contributes to the UN's Sustainable Development Goals (SDGs) "9: Industries, Innovation and Infrastructure".

[0428] Furthermore, the technology of this embodiment can provide better AI-powered response output technology. It is anticipated that such AI-powered response output technology will be adopted in public transport systems to improve the usability of transport systems for vulnerable groups. By adopting this technology in public transport systems, it is possible to contribute to improved transport safety through the expansion of public transport systems and to achieve sustainable transport systems that are safe, affordable, and easily accessible to all. This, in turn, contributes to the United Nations Sustainable Development Goals (SDGs) of "Sustainable Cities and Communities."

[0429] Various embodiments have been described in detail above, but the present invention is not limited to the above embodiments and includes various modifications. For example, the above embodiments have described the entire system in detail for ease of understanding of the present invention, but are not limited to having all the described structures. In addition, a part of the structure of one embodiment can be replaced with the structure of another embodiment, and the structure of another embodiment can be added to the structure of one embodiment. Furthermore, for a part of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0430] Explanation of reference numerals in the attached figures

[0431] 10010……Artificial intelligence response output device, 10011……Display unit, 10028……Local LLM processing unit, 1107……Operation input unit, 1110……Control unit, 1132……Communication unit, 1140……Sound output unit, 1139……Microphone, 1160……Image control unit, 1170……Storage unit, 1180……Camera unit.< / audio> < / video> < / audio> < / video>

Claims

1. A response output device, characterized in that, include: The control unit obtains the responses given by the prompt words input to the large-scale language model. Display section; and Audio output section In the control state of the control unit, there is a state in which the answer from the large-scale language model is output via the display unit or the sound output unit.

2. The answer output device as described in claim 1, characterized in that: Including the storage section, The storage unit contains a database that stores multiple fixed statements that form the basis of the answers. In the control state of the control unit, there is a state in which the answer generated based on the fixed sentences stored in the database is not output, but the answer is not output from the large-scale language model.

3. The answer output device as described in claim 2, characterized in that: Including the operation input section or microphone, The control unit performs the following controls: When a user inputs a request for an answer from the large-scale language model via the operation input unit or the microphone, preparation begins to output an answer generated based on the fixed phrases stored in the database, and the answer generated based on the fixed phrases stored in the database begins to be output via the display unit or the sound output unit. Before completing the output of the answer generated based on the fixed phrases stored in the database, a prompt word is sent to the large-scale language model to begin obtaining the answer from the large-scale language model. After the answer is generated based on the fixed phrases stored in the database, the obtained answer from the large-scale language model is then output via the display unit or the sound output unit.

4. The answer output device as described in claim 2, characterized in that: The display unit can display multiple different AI assistants. The database in the storage unit stores fixed-form statement data that can generate different answers for each of the multiple different AI assistants.

5. The answer output device as described in claim 2, characterized in that: Including the Ministry of Communications, The control unit can send prompt words to the large-scale language model on the server on the network via the communication unit, and obtain a response to the prompt words from the large-scale language model on the server on the network. In the control state of the control unit, the following exists: In the first state, when it is possible to connect to the network via the communication unit, the answer obtained from the large-scale language model in the server on the network is output via the display unit or the sound output unit. and In the second state, when the communication unit cannot connect to the network, the answer generated based on the fixed phrases stored in the database is output via the display unit or the sound output unit.

6. The answer output device as described in claim 2, characterized in that: Including the Ministry of Communications, The control unit can send prompt words to the large-scale language model on the server on the network via the communication unit, and obtain a response to the prompt words from the large-scale language model on the server on the network. In the control state of the control unit, the following exists: In the first state, if a response from a large-scale language model on a server connected via the communication unit can be obtained within a specified time, the response obtained from the large-scale language model on the server on the network is output via the display unit or the sound output unit. and In the second state, if a response from a large-scale language model on a server connected via the communication unit cannot be obtained within the specified time, a response generated based on fixed phrases stored in the database is output via the display unit or the sound output unit.

7. The answer output device as described in claim 2, characterized in that: Including the Ministry of Communications, The control unit can send prompt words to the large-scale language model on the server on the network via the communication unit, and obtain a response to the prompt words from the large-scale language model on the server on the network. In the control state of the control unit, the following exists: In the first state, if the usage of the large-scale language model in the server on the network connected via the communication unit has not reached the prescribed limit, the answer obtained from the large-scale language model in the server on the network is output via the display unit or the sound output unit. and In the second state, when the usage of a large-scale language model in a server on the network connected via the communication unit has reached a predetermined limit, a response generated based on a fixed phrase stored in the database is output via the display unit or the sound output unit.

8. The answer output device as described in claim 1, characterized in that: Includes a local large-scale language model processing unit capable of performing large-scale language model processing within the response output device. In the control state of the control unit, there is a state in which the answer obtained from the large-scale language model of the local large-scale language model processing unit is output via the display unit or the sound output unit.

9. The answer output device as claimed in claim 1, characterized in that, include: Ministry of Communications; and Local large-scale language model processing unit, In the control state of the control unit, the following exists: In the first state, a prompt word is sent to a large-scale language model on a server on the network via the communication unit, and a response to the prompt word is obtained from the large-scale language model on the server on the network. The response is then output via the display unit or the sound output unit. and In the second state, the answer obtained from the large-scale language model of the local large-scale language model processing unit is output via the display unit or the sound output unit.

10. The answer output device as claimed in claim 1, characterized in that, include: Ministry of Communications; and Local large-scale language model processing unit, In the control state of the control unit, the following exists: In the first state, when a connection to the network is possible via the communication unit, the answer obtained from a large-scale language model on the server on the network is output via the display unit or the sound output unit. and In the second state, when the communication unit cannot connect to the network, the answer obtained from the large-scale language model of the local large-scale language model processing unit is output via the display unit or the sound output unit.

11. The answer output device as claimed in claim 1, characterized in that, include: Ministry of Communications; and Local large-scale language model processing unit, In the control state of the control unit, the following exists: In the first state, if a response from a large-scale language model on a server connected via the communication unit can be obtained within a specified time, the response obtained from the large-scale language model on the server on the network is output via the display unit or the sound output unit. and In the second state, if a response from a large-scale language model in a server on the network connected via the communication unit cannot be obtained within the specified time, the response obtained from the large-scale language model in the local large-scale language model processing unit is output via the display unit or the sound output unit.

12. The answer output device as claimed in claim 1, characterized in that, include: Ministry of Communications; and Local large-scale language model processing unit, In the control state of the control unit, the following exists: In the first state, if the usage of the large-scale language model in the server on the network connected via the communication unit has not reached the prescribed limit, the answer obtained from the large-scale language model in the server on the network is output via the display unit or the sound output unit. and In the second state, when the usage of the large-scale language model in the server on the network connected via the communication unit has reached a predetermined limit, the answer obtained from the large-scale language model of the local large-scale language model processing unit is output via the display unit or the sound output unit.

13. The answer output device as claimed in claim 1, characterized in that: Including the Ministry of Communications, The control unit can send prompt words to a first large-scale language model on a server on the network via the communication unit, and obtain a response to the prompt words from the first large-scale language model on the server on the network. Furthermore, it can send prompt words to a second large-scale language model on a server on the network via the communication unit, and obtain a response to the prompt words from the second large-scale language model on the server on the network.

14. The answer output device as claimed in claim 13, characterized in that: In the control state of the control unit, the following exists: In the first state, the response obtained from the first large-scale language model in the server on the network is output via the display unit or the sound output unit. and In the second state, the response obtained from the second large-scale language model in the server on the network is output via the display unit or the sound output unit.

15. The answer output device as claimed in claim 13, characterized in that: In the control state of the control unit, the following exists: In the first state, if the usage of the first large-scale language model in the server on the network connected via the communication unit has not reached the prescribed upper limit, the answer obtained from the first large-scale language model in the server on the network is output via the display unit or the sound output unit. and In the second state, when the usage of the first large-scale language model in the server on the network connected via the communication unit has reached a predetermined limit, the answer obtained from the second large-scale language model in the server on the network is output via the display unit or the sound output unit.

16. The answer output device as claimed in claim 1, characterized in that, include: Ministry of Communications; and Operate the input section or microphone. The control unit can send prompt words to a large-scale language model on a server on the network via the communication unit, and obtain a response to the prompt words from the large-scale language model on the server on the network. In the control state of the control unit, the following exists: In the first state, when it is possible to connect to the network via the communication unit, the answer obtained from the large-scale language model in the server on the network is output via the display unit or the sound output unit. and In the second state, if the user cannot connect to the network via the communication unit, even if the user makes an input requesting a response from a large-scale language model via the operation input unit or the microphone, no response to the input will be output via the display unit or the sound output unit.

17. The answer output device as claimed in claim 1, characterized in that, include: Ministry of Communications; and Operate the input section or microphone. The control unit can send prompt words to a large-scale language model on a server on the network via the communication unit, and obtain a response to the prompt words from the large-scale language model on the server on the network. In the control state of the control unit, the following exists: In the first state, if the usage of the large-scale language model in the server on the network connected via the communication unit has not reached the prescribed limit, the answer obtained from the large-scale language model in the server on the network is output via the display unit or the sound output unit. and In the second state, when the usage of the large-scale language model in the server on the network connected via the communication unit has reached a predetermined limit, even if the user makes an input requesting a response from the large-scale language model via the operation input unit or the microphone, no response to the input will be output via the display unit or the sound output unit.

18. The answer output device as claimed in claim 2, characterized in that: The display unit can show the AI ​​assistant. The response output device includes a communication unit. The control unit can send prompt words to a large-scale language model on a server on the network via the communication unit, and obtain a response to the prompt words from the large-scale language model on the server on the network. In the display control state of the AI ​​assistant on the display unit implemented by the control unit, there exists: The first display control state is in which the communication unit can connect to the network; and The second display control state indicates that connection to the network via the communication unit is not possible. The display state of the AI ​​assistant in the second display control state is different from that in the first display control state. It is displayed as if the AI ​​assistant is in a sleep state.

19. The answer output device as claimed in claim 18, characterized in that: Including the operation input section or microphone, In the second display control state, the start time of the display state when the AI ​​assistant is in a sleep state satisfies the following: After the user can no longer connect to the network via the communication unit, and before the user makes input via the operation input unit or the microphone that requires a response from a large-scale language model.

20. The answer output device as claimed in claim 2, characterized in that: The display unit can show the AI ​​assistant. The response output device includes an operation input unit or a microphone. In the display control state of the AI ​​assistant on the display unit implemented by the control unit, there exists: In the first display control state, the response from the large-scale language model can be output via the display unit or the sound output unit. and In the second display control state, even if the user inputs a request for a response from the large-scale language model via the operation input unit or the microphone, no response to that input will be output via the display unit or the sound output unit. The display state of the AI ​​assistant in the second display control state is different from the display state of the AI ​​assistant in the first display control state.

21. The answer output device as claimed in claim 20, characterized in that: In the second display control state, the display state of the AI ​​assistant is displayed in a state different from that of the AI ​​assistant in the first display control state, that is, the AI ​​assistant is displayed as if it is in a sleep state.

22. The answer output device as claimed in claim 20, characterized in that: In the second display control state, the start time point at which the display state of the AI ​​assistant is displayed in a state different from that in the first display control state satisfies the following: This state occurs after the user inputs a request for a response from the large-scale language model via the operation input unit or the microphone, but before the user inputs a request for a response from the large-scale language model via the operation input unit or the microphone.

23. The answer output device as claimed in claim 20, characterized in that: The operation input unit or the microphone is at least one of the operation input units. The display unit displays a predetermined mark in the second display control state. The control unit performs the following controls: This explains why, when the user selects the specified flag via the operation input unit, the display state of the AI ​​assistant on the display unit becomes different from the display state of the AI ​​assistant in the first display control state.

24. The answer output device as claimed in claim 2, characterized in that: The display unit can show the AI ​​assistant. In the display control state of the AI ​​assistant on the display unit implemented by the control unit, there exists: In the first display control state, the response from the large-scale language model can be output via the display unit or the sound output unit. and In the second display control state, the response that can be output from the response output device is a lower-performance response output compared to when an response is output using the response from the large-scale language model. The display state of the AI ​​assistant in the second display control state is different from the display state of the AI ​​assistant in the first display control state.

25. The answer output device as claimed in claim 24, characterized in that: In the second display control state, the AI ​​assistant's display state is displayed in a state different from that in the first display control state, which is a state in which the AI ​​assistant is drowsy.

26. The answer output device as claimed in claim 24, characterized in that: Including the operation input section or microphone, In the second display control state, the start time point at which the display state of the AI ​​assistant is displayed in a state different from that in the first display control state satisfies the following: This occurs after the response output device outputs a response that is of lower performance compared to when the response is output using the response from the large-scale language model, and before the user inputs a request for a response from the large-scale language model via the operation input unit or the microphone.

27. The answer output device as claimed in claim 24, characterized in that: Including the operation input section, The display unit displays a predetermined mark in the second display control state. The control unit performs the following controls: This explains why, when the user selects the specified flag via the operation input unit, the display state of the AI ​​assistant on the display unit becomes different from the display state of the AI ​​assistant in the first display control state.

28. A response output system, characterized in that, include: Display section; Audio output section; The control unit generates prompt words based on user input and sends the prompt words to a large-scale language model; and The large-scale language model generates an answer to the prompt word and sends the answer to the control unit. The control state of the control unit includes the following states: Control is performed to output the response generated based on the large-scale language model via the display unit or the sound output unit.

29. The answer output system as described in claim 28, characterized in that: The following states exist: If inappropriate content is contained in the prompt word or the response generated by the large-scale language model, the response generated by the large-scale language model is filtered, and the filtered response and a flag indicating that the filtering has been performed are sent to the control unit from the large-scale language model.

30. The answer output system as described in claim 29, characterized in that: The control unit that receives the flag performs control, executes processing corresponding to the flag, and outputs the result to the user via the display unit or the sound output unit based on the processing result.

31. The answer output system as described in claim 30, characterized in that: The large-scale language model is stored on a server connected to the Internet. The control unit is located in a different device than the server. The large-scale language model and the control unit send and receive information via the Internet.

32. The answer output system as described in claim 30, characterized in that: The control unit performs control, and outputs the character's response to the user's input via the display unit or the sound output unit. The output of the response given by the character is generated based on the flag received by the control unit.

33. The answer output system as described in claim 28, characterized in that: The control unit transforms the prompt words generated based on user input to generate a second prompt word, and sends the second prompt word to the large-scale language model.

34. The answer output system as described in claim 33, characterized in that: The transformation processing performed by the control unit is a process of adding auxiliary natural language information to the natural language information of the prompt words generated based on the user input.

35. The answer output system as described in claim 33, characterized in that: The transformation processing performed by the control unit is to add auxiliary English natural language information to the non-English natural language information of the prompt words generated based on the user input.

36. The answer output system as described in claim 33, characterized in that: The transformation process performed by the control unit is to transform non-English natural language information of prompt words generated based on the user input into English natural language information.

37. The answer output system as described in claim 33, characterized in that: The transformation process performed by the control unit is to add prompt words composed of natural language or code to a second data area that is different from the first data area in the format of the prompt words. The first data area is used to store the natural language information of the prompt words generated based on the user input.

38. The answer output system as described in claim 37, characterized in that: The second data area is a data area for sending and receiving data in a pre-agreed format for the processing to be performed.

39. The answer output system as described in claim 37, characterized in that: Non-English prompts are stored in the first data area, and English prompts are stored in the second data area.

40. A response output system, characterized in that, include: Display section; Audio output section; Control Department; The first large-scale language model; and The second large-scale language model, The control state of the control unit includes the following states: The system performs control, generates a first prompt word based on user input, sends the first prompt word to the first large-scale language model, obtains a first answer generated by the first large-scale language model based on the first prompt word, generates a second prompt word based on the first answer, sends the second prompt word to the second large-scale language model, obtains a second answer generated by the second large-scale language model based on the second prompt word, processes the first answer and the second answer, and outputs the output generated based on the processing of the first answer and the second answer via the display unit or the sound output unit.

41. The answer output system as described in claim 40, characterized in that: Including the storage section, The second prompt word generation process involves combining the content of the first answer with the fixed phrase stored in the storage unit to generate the second prompt word.

42. The answer output system as described in claim 40, characterized in that: In the processing based on both the first and second answers, the output to the user is switched according to the second answer.

43. The answer output system as described in claim 42, characterized in that: Including the storage section, In the process of generating the user's output based on both the first and second answers, the output to the user is generated using fixed statements stored in the storage unit, without including the content of the first or second answer.

44. The answer output system as described in claim 42, characterized in that: Including the storage section, In the process of generating the user's output based on both the first and second answers, the content of the first answer is combined with the fixed statement stored in the storage unit to generate the output to the user.

45. The answer output system as described in claim 42, characterized in that: Including the storage section, In the process of generating the user's output based on both the first and second answers, the content of the second answer is combined with the fixed statement stored in the storage unit to generate the user's output.

46. ​​The answer output system as described in claim 40, characterized in that: The control unit sends first control information to the first large-scale language model and second control information to the second large-scale language model. The first control information is used to limit the number of lexical units in the response to the first prompt word output by the first large-scale language model, and the second control information is used to limit the number of lexical units in the response to the second prompt word output by the second large-scale language model. The number of lemmas shown in the parameters of the second control information is less than the number of lemmas shown in the parameters of the first control information.

47. A response output device, characterized in that, include: Display section; Audio output section; and The control unit generates prompts based on user input and sends these prompts to a large-scale language model. The control state of the control unit includes the following states: The system controls the acquisition of the response generated by the large-scale language model based on the prompt words, and outputs the response generated based on the prompt words via the display unit or the sound output unit.

48. The answer output device as claimed in claim 47, characterized in that: If the prompt word or the answer generated by the large-scale language model contains inappropriate content, the control unit obtains the filtered answer and a flag indicating that the filtering has been performed from the large-scale language model.

49. The answer output device as claimed in claim 48, characterized in that: The control unit that receives the flag performs control, executes processing corresponding to the flag, and outputs the result to the user via the display unit or the sound output unit based on the processing result.

50. The answer output device as claimed in claim 49, characterized in that: The response output device is connected to a server that stores the large-scale language model via the Internet. The control unit of the answer output device sends and receives information with the large-scale language model via the Internet.

51. The answer output device as described in claim 49, characterized in that: The control unit performs control, and outputs the character's response to the user's input via the display unit or the sound output unit. The output of the response given by the character is generated based on the flag received by the control unit.

52. The answer output device as claimed in claim 47, characterized in that: The control unit transforms the prompt words generated based on user input to generate a second prompt word, and sends the second prompt word to the large-scale language model.

53. The answer output device as described in claim 52, characterized in that: The transformation processing performed by the control unit is a process of adding auxiliary natural language information to the natural language information of the prompt words generated based on the user input.

54. The answer output device as described in claim 52, characterized in that: The transformation processing performed by the control unit is to add auxiliary English natural language information to the non-English natural language information of the prompt words generated based on the user input.

55. The answer output device as described in claim 52, characterized in that: The transformation process performed by the control unit is to transform non-English natural language information of prompt words generated based on the user input into English natural language information.

56. The answer output device as described in claim 52, characterized in that: The transformation process performed by the control unit is to add prompt words composed of natural language or code to a second data area that is different from the first data area in the format of the prompt words. The first data area is used to store the natural language information of the prompt words generated based on the user input.

57. The answer output device as claimed in claim 56, characterized in that: The second data area is a data area for sending and receiving data in a pre-agreed format for the processing to be performed.

58. The answer output device as described in claim 56, characterized in that: Non-English prompts are stored in the first data area, and English prompts are stored in the second data area.

59. A response output device, capable of sending and receiving data with a first large-scale language model and with a second large-scale language model, characterized in that, include: Display section; Audio output section; and Control Department The control state of the control unit includes the following states: A first prompt word is generated based on user input, and the first prompt word is sent to the first large-scale language model to obtain a first answer generated by the first large-scale language model based on the first prompt word. A second prompt word is generated based on the first answer, and the second prompt word is sent to the second large-scale language model to obtain a second answer generated by the second large-scale language model based on the second prompt word. Processing is performed based on both the first answer and the second answer, and the output generated based on the processing of the first answer and the second answer is output via the display unit or the sound output unit.

60. The answer output device as claimed in claim 59, characterized in that: Including the storage section, The second prompt word generation process involves combining the content of the first answer with the fixed phrase stored in the storage unit to generate the second prompt word.

61. The answer output device as claimed in claim 59, characterized in that: In the processing based on both the first and second answers, the output to the user is switched according to the second answer.

62. The answer output device as described in claim 59, characterized in that: Including the storage section, In the process of generating the user's output based on both the first and second answers, the output to the user is generated using fixed statements stored in the storage unit, without including the content of the first or second answer.

63. The answer output device as described in claim 59, characterized in that: Including the storage section, In the process of generating the user's output based on both the first and second answers, the content of the first answer is combined with the fixed statement stored in the storage unit to generate the output to the user.

64. The answer output device as described in claim 59, characterized in that: Including the storage section, In the process of generating the user's output based on both the first and second answers, the content of the second answer is combined with the fixed statement stored in the storage unit to generate the user's output.

65. The answer output device as claimed in claim 59, characterized in that: The control unit sends first control information to the first large-scale language model and second control information to the second large-scale language model. The first control information is used to limit the number of lexical units in the response to the first prompt word output by the first large-scale language model, and the second control information is used to limit the number of lexical units in the response to the second prompt word output by the second large-scale language model. The number of lemmas shown in the parameters of the second control information is less than the number of lemmas shown in the parameters of the first control information.

Images