Information processing device, information processing system, information processing method, program

Abstract

The purpose is to support the registration process of registering information that can be read from a file as searchable information in order to output answer data for questions. [Means for Solving the Problem] An information processing device that receives input of question data from a user via a terminal device, searches a file stored in a storage unit, and outputs answer data, comprises: a receiving unit that receives a request to register a file to be stored in the storage unit; a prompt generating unit that generates a prompt for outputting content that can be read from an object contained in the file for which the registration request was received; a transmitting unit that sends the prompt generated by the prompt generating unit and the object to a large-scale language model; and a registration unit that registers the result output by the large-scale language model in association with the file for which the registration request was received to the storage unit.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing system, an information processing method, and a program.

Background Art

[0002] In recent years, in response systems that output answer data for questions input in text or voice, various devices have been made to improve the accuracy of answers. As an example of this, there is known a technique for generating screen information for searching for answers including words included in a question and editing at least one or more searched answers on a terminal device of a system administrator.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In the conventional technique, when registering information to be searched for outputting answer data for a question, if a user attempts to register an answer based on information that can be read from a file or the like, the user has to check the content of the file and register the readable information as text.

[0004] The disclosed technique has been made in view of the above circumstances, and aims to assist in the registration work of registering information that can be read from a file as information to be searched for outputting answer data for a question.

Means for Solving the Problems

[0005] The disclosed technology is an information processing device that, when it receives input of question data from a user via a terminal device, searches a file stored in a storage unit and outputs answer data, and comprises: a receiving unit that receives a request to register a file to be stored in the storage unit; a prompt generating unit that generates a prompt for outputting content that can be read from an object contained in the file for which the registration request was received; a transmitting unit that sends the prompt generated by the prompt generating unit and the object to a large-scale language model; and a registration unit that registers the result output by the large-scale language model in association with the file for which the registration request was received to the storage unit. [Effects of the Invention]

[0006] This system can assist in the registration process of registering information that can be read from a file as searchable information to output answer data for questions. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram showing an example of the system configuration of a chat system. [Figure 2] This figure shows an example of the hardware configuration of an information processing device. [Figure 3] This figure shows an example of the hardware configuration of a terminal device. [Figure 4] This diagram illustrates the functional configuration of each component in the chat system. [Figure 5] This figure shows an example of a conversation history storage unit in the first memory device. [Figure 6] This figure shows an example of the document storage section of the second memory device. [Figure 7] This shows an example of the chunk storage section of the second memory device. [Figure 8] This figure shows an example of the user information storage section of the second memory device. [Figure 9] This is the first sequence diagram illustrating the operation of the chat system according to the first embodiment. [Figure 10] It is a diagram showing an example of an in-company information registration screen of the first embodiment. [Figure 11] It is a diagram showing an example of in-company information of the first embodiment. [Figure 12] It is a diagram for explaining the analysis result of in-company information by an analysis server. [Figure 13] It is a diagram showing an example of a prompt of the first embodiment. [Figure 14] It is a diagram showing an example of the result of verbalizing an object by a large language model in the first embodiment. [Figure 15] It is a diagram showing an example of verbalized in-company information. [Figure 16] It is a second sequence diagram for explaining the operation of the chat system of the first embodiment. [Figure 17] It is a diagram showing an example of display when question data is input to a terminal device in the first embodiment. [Figure 18] It is a diagram showing an example of in-company information of the second embodiment. [Figure 19] It is a diagram showing an example of a prompt of the second embodiment. [Figure 20] It is a diagram showing an example of the result of verbalizing an object by a large language model in the second embodiment. [Figure 21] It is a diagram showing an example of display of a terminal device of the second embodiment. [Figure 22] It is a sequence diagram for explaining the operation of the chat system of the third embodiment. [Figure 23] It is a diagram showing an example of a prompt of the third embodiment. [[ID=4I]] [Figure 24] It is a diagram showing an example of in-company information of the fourth embodiment. [Figure 25] It is a diagram showing an example of an in-company information registration screen of the fourth embodiment.

Embodiments for Carrying Out the Invention

[0008] <oo00089>(First Embodiment) Hereinafter, this embodiment will be described with reference to the drawings. FIG. 1 is a diagram showing an example of the system configuration of a chat system.

[0009] The chat system 100 of this embodiment includes an information processing device 200, a first storage device 300, a second storage device 400, a server device 500, an analysis server 550, a terminal device 600, and a terminal device 700, and is an example of an information processing system in which each is connected via a communication network 1. Also, the chat system 100 of this embodiment may be an information processing device having a computer configuration or a system including a plurality of computers. The chat system 100 may be, for example, an interactive agent that interacts with a user via the terminal device 600. Further, the chat system 100 of this embodiment is a program or system that responds to inquiries from users or the like using the terminal device 600, and is, for example, a service that automatically responds using AI (Artificial Intelligence) or knowledge including registered information and knowledge, and is also called an AI agent. Preferably, the chat system 100 supports the execution of a predetermined task such as a help desk or a business negotiation by means of an interaction in which the chat system automatically responds to a voice or an input message from a user. As a use case of the chat system 100, it is also used, for example, as a non-human AI avatar or the like in a web conference, a website, a smartphone application, or a virtual space of a metaverse (hereinafter referred to as a metaverse space).

[0010] In the chat system 100, the information processing device 200, the first storage device 300, the second storage device 400, the server device 500, and the analysis server 550 provide a chatbot service to the user of the terminal device 600.

[0011] When a question is input from the terminal device 600, the information processing device 200 outputs an answer to the question to the terminal device 600.

[0012] The first storage device 300 stores conversation history information, such as questions input from the terminal device 600 and answers output to the terminal device 600. The conversation history information may be managed separately for each tenant that implements the chat system 100. In this embodiment, a tenant may be an organization such as a company, or the organization to which a user of the chat system 100 belongs.

[0013] The second storage device 400 stores user-specific information that is not publicly available. A user of the chat system 100 may be, for example, a tenant, in which case the user-specific information would be tenant-specific information. Alternatively, a user of the chat system 100 may be an individual user, in which case the user-specific information would be user-specific information.

[0014] In the following description, user-specific information will be referred to as internal company information, and internal company information will be used as an example of user-specific information. In this embodiment, user-specific information is a file stored in a storage unit containing one or more objects, and is retrieved in order to output answer data when question data is received from the user. Details of the objects will be described later.

[0015] Internal information refers to data managed by an organization consisting of multiple members, and may include data stored in a database managed by that organization. The organization may include, for example, a for-profit organization such as a company, or a non-profit organization such as a hospital, school, or cram school. As an example, internal information managed by a hospital may include patient information such as medical records. In this case, the chat system 100 can generate response data including a diagnosis based on the patient information. As another example, internal information managed by a school or cram school may include information regarding the learning status or academic performance of users such as students. In this case, the chat system 100 can generate responses including appropriate practice problems or video lessons based on the learning status or academic performance information.

[0016] The second storage device 400 may store internal company information for each tenant. Furthermore, the second storage device 400 may store information obtained by dividing the internal company information into predetermined units, with each divided unit stored separately. In the following explanation, each divided unit of internal company information will be referred to as a chunk. Additionally, the second storage device 400 may store user information related to users belonging to each tenant, for each tenant. Furthermore, the files in this embodiment only need to be stored in the memory unit in order to output the answer to the question to the generation AI described later, and are not limited to user-specific information (company information). Also, some files may not contain any objects.

[0017] The server device 500 implements artificial intelligence (AI). The artificial intelligence generates text data in response to input prompts, such as ChatGPT, and outputs it to the information processing device 200. In this embodiment, the prompts are text data that represent instructions or questions for the artificial intelligence implemented by the server device 500. Details of the large-scale language model will be described later.

[0018] The analysis server 550 receives internal company information and performs analysis of it. Specifically, if the internal company information includes information other than text data, such as objects, tables, and images, the analysis server 550 detects the non-text data and extracts the detected results as an image. In the following explanation, information other than text data included in the internal company information may be referred to as objects included in the internal company information. Objects may include graphic data represented by line drawings, tabular data, and image data such as photographs.

[0019] Furthermore, if the internal company information contains an object, the analysis server 550 of this embodiment extracts the image data of the object and the object's caption (heading) as images. In other words, the object's caption (heading) is information about the object. The analysis server 550 then outputs the image data showing the object and the image data showing the caption to the information processing device 200. Terminal device 600 is primarily used by users of the chat system 100. Terminal device 700 is primarily used by administrators who manage the chat system 100. Terminal devices 600 and 700 may be, for example, smartphones or tablet devices.

[0020] In the chat system 100 of this embodiment, when a user inputs a question in the terminal device 600, the terminal device 600 transmits question data indicating the question to the information processing device 200.

[0021] When the information processing device 200 receives input question data, it retrieves conversation history information from the first storage device 300 for a certain period prior to the input of the question to the terminal device 600. The information processing device 200 then outputs a prompt to the server device 500 requesting the generation of a query (search key) to search for chunks stored in the second storage device 400 using the input question and the retrieved conversation history information.

[0022] The server device 500 inputs a prompt to the large-scale language model requesting query generation and sends the output data output from the large-scale language model to the information processing device 200. The information processing device 200 uses the output data received from the server device 500 as a query to search the second storage device 400 and obtain chunks of the search results.

[0023] The information processing device 200 then outputs a prompt to the server device 500 requesting it to generate an answer to a question input from the terminal device 600, using the chunks obtained as search results as the information source.

[0024] The server device 500 inputs a prompt to the large-scale language model requesting the generation of a response, and transmits the output data generated by the large-scale language model to the information processing device 200. The information processing device 200 transmits the output data received from the server device 500 as response data to the terminal device 600.

[0025] The terminal device 600 displays the response data received from the information processing device 200 as the answer to the entered question.

[0026] In this embodiment, by doing so, it is possible to generate answers to questions from a large-scale language model based on user-specific information that is not publicly available. Therefore, according to this embodiment, it is possible to provide responses based on user-specific information.

[0027] Furthermore, in this embodiment, the information source used to generate the answer is internal company information obtained by searching the second storage device 400 based on conversation history information before the question was entered. Therefore, according to this embodiment, the internal company information to be used as the information source can be narrowed down and sent to the server device 500, thereby improving the accuracy of the answer while satisfying the token limitations in the large-scale language model.

[0028] Furthermore, in the chat system 100 of this embodiment, when the information processing device 200 receives a request from the terminal device 700 to store company information including objects in the second storage device 400, it stores the company information including objects in the document storage unit 410.

[0029] Furthermore, the information processing device 200, using the analysis server 550, acquires objects contained in the company information as image data and sends a prompt to the server device 500 requesting the verbalization of the image (objects contained in the company information) represented by the acquired image data. The information processing device 200 then replaces the text data received from the server device 500 with the objects contained in the company information, divides it into chunks, and stores each chunk in the second storage device 400. In other words, a chunk in this embodiment is a set of text data obtained by dividing the company information into verbalized content that can be read from the objects contained in the file.

[0030] In the chat system 100 of this embodiment, the internal company information is replaced with linguistic text data and stored in the second storage device 400. Therefore, in this embodiment, internal company information including objects can also be used as a source of information in generating answers using a large-scale language model. In other words, according to this embodiment, it is possible to support the registration of answers based on information read from a file. Furthermore, according to this embodiment, it becomes possible to generate answers to questions using objects contained in a file as a source of information.

[0031] In the example in Figure 1, the information processing device 200, the first storage device 300, and the second storage device 400 are shown as separate devices, but this is not limited to this. The first storage device 300 and the second storage device 400 may be included in the information processing device 200. Also, in the example in Figure 1, the information processing device 200 and the analysis server 550 are shown as separate devices, but this is not limited to this. The analysis server 550 may be included in the information processing device 200. In other words, the information processing device 200 may also function as the analysis server 550.

[0032] Furthermore, in Figure 1, the information processing device 200 may be implemented by one or more information processing devices, or it may be implemented by multiple information processing devices.

[0033] Furthermore, the chat system 100 only needs to include the information processing device 200 and the terminal device 600; other devices do not need to be included in the chat system 100.

[0034] Next, an example of the hardware configuration of each device in the chat system 100 of this embodiment will be described.

[0035] In this embodiment, the information processing device 200, the first storage device 300, the second storage device 400, the server device 500, and the analysis server 550 are all general-purpose computers. Therefore, Figure 2 describes the hardware configuration of the information processing device 200 as an example.

[0036] In this embodiment, terminal devices 600 and 700 are smartphones, and Figure 3 illustrates the hardware configuration of terminal device 600 as an example.

[0037] Figure 2 shows an example of the hardware configuration of an information processing device. The information processing device 200 shown in Figure 2 is built by a computer (information processing device) and includes a CPU 201, ROM 202, RAM 203, HD 204, HDD (Hard Disk Drive) controller 205, display 206, external device connection I / F (Interface) 208, network I / F 209, bus line 210, keyboard 211, pointing device 212, DVD-RW (Digital Versatile Disk Rewritable) drive 214, and media I / F 216.

[0038] Of these, the CPU 201 controls the operation of the entire information processing unit 200. The ROM 202 stores programs used to drive the CPU 201, such as IPL. The RAM 203 is used as the work area for the CPU 501. The HD 204 stores various data such as programs. The HDD controller 205 controls the reading or writing of various data to the HD 204 according to the control of the CPU 501. The display 206 displays various information such as cursors, menus, windows, characters, or images. The external device connection I / F 208 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printers. The network I / F 209 is an interface for data communication using a communication network. The bus line 210 is an address bus, data bus, etc., for electrically connecting each component such as the CPU 201 shown in Figure 2.

[0039] The keyboard 211 is a type of input means equipped with multiple keys for inputting characters, numbers, and various instructions. The pointing device 212 is a type of input means for selecting and executing various instructions, selecting processing targets, and moving the cursor. The DVD-RW drive 214 controls the reading or writing of various data to the DVD-RW 213, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may also be DVD-R, etc. The media I / F 216 controls the reading or writing (storage) of data to the recording medium 215, such as flash memory.

[0040] Figure 3 shows an example of the hardware configuration of a terminal device. The terminal device 600 in this embodiment includes a CPU 601, ROM 602, RAM 603, EEPROM 604, CMOS sensor 605, image sensor I / F 606, acceleration / direction sensor 607, media I / F 609, and GPS receiver 611.

[0041] Of these, the CPU 601 is an arithmetic processing unit that controls the operation of the entire terminal device 600. The ROM 602 stores programs used to drive the CPU 601, such as the CPU 601 and the IPL. The RAM 603 is used as the work area for the CPU 601. The EEPROM 604 reads or writes various data, such as smartphone programs, according to the control of the CPU 601. The ROM 602, RAM 603, and EEPROM 604 are examples of the storage devices of the terminal device 600.

[0042] The CMOS (Complementary Metal Oxide Semiconductor) sensor 605 is a type of built-in imaging device that captures an image of a subject (mainly a self-portrait) and obtains image data according to the control of the CPU 601. Note that an imaging device such as a CCD (Charge Coupled Device) sensor may also be used instead of a CMOS sensor.

[0043] The image sensor interface 606 is a circuit that controls the driving of the CMOS sensor 605. The acceleration / direction sensor 607 is a variety of sensors such as an electronic magnetic compass, gyrocompass, and acceleration sensor that detect the Earth's magnetic field. The media interface 609 controls the reading or writing (storage) of data to the recording medium 608, such as flash memory. The GPS receiver 611 receives GPS signals from GPS satellites.

[0044] Furthermore, the terminal device 600 includes a long-range communication circuit 612, an antenna 612a for the long-range communication circuit 612, a CMOS sensor 613, an image sensor interface 614, a microphone 615, a speaker 616, an audio input / output interface 617, a display 618, an external device connection interface 619, a short-range communication circuit 620, an antenna 620a for the short-range communication circuit 620, and a touch panel 621.

[0045] Of these, the long-distance communication circuit 612 is a circuit that communicates with other devices via a communication network. The CMOS sensor 613 is a type of built-in imaging means that captures an image of a subject and obtains image data according to the control of the CPU 601. The image sensor interface 614 is a circuit that controls the driving of the CMOS sensor 613. The microphone 615 is a built-in circuit that converts sound into electrical signals. The speaker 616 is a built-in circuit that converts electrical signals into physical vibrations to produce sounds such as music and speech. The sound input / output interface 617 is a circuit that processes the input and output of sound signals between the microphone 615 and the speaker 616 according to the control of the CPU 601.

[0046] The display 618 is a type of display means, such as a liquid crystal or organic EL (electroluminescence), that displays images of the subject and various icons. The external device connection I / F 619 is an interface for connecting various external devices. The short-range communication circuit 620 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 621 is a type of input means that allows the user to operate the terminal device 600 by pressing the display 618. The display 618 is an example of a display unit of the terminal device 600.

[0047] Note that the hardware configuration shown in Figure 3 is just one example of a terminal device 600, and the hardware configuration of the terminal device 600 is not limited to this. The hardware configuration of the terminal device 600 may also be the configuration shown in Figure 2.

[0048] Next, with reference to Figure 4, the functions of each device in the chat system 100 of this embodiment will be described. Figure 4 is a diagram illustrating the functional configuration of each device in the chat system.

[0049] First, the functions of the first storage device 300 will be described. The first storage device 300 has a conversation history storage unit 310, a search unit 330, and a storage unit 340.

[0050] The conversation history storage unit 310 may be implemented by a storage device of the first storage device 300, and the search unit 330 and storage unit 340 are implemented by the CPU of the first storage device 300 reading and executing a program stored in the storage device.

[0051] The conversation history storage unit 310 stores conversation history information that associates questions input from the terminal device 600 with answers output to the terminal device 600. The search unit 330 searches the conversation history storage unit 310 in response to a search request from the information processing device 200 and transmits the search results to the information processing device 200. When the information processing device 200 receives question data and outputs response data, the storage unit 340 stores the conversation history information, including the question data and the output response data, in the conversation history storage unit 310.

[0052] The first storage device 300 may also have a memory unit that stores a set of question-answer information in which questions and answers are associated. In that case, when the first storage device 300 receives a question input from the terminal device 600, it may search the question-answer information set and output the answer. The question-answer information may be stored for each tenant that implements the chat system 100.

[0053] Next, the functions of the second storage device 400 will be described. The second storage device 400 includes a document storage unit 410, a chunk storage unit 415, a user information storage unit 420, a search unit 430, and a storage unit 440.

[0054] In the second storage device 400, the document storage unit 410, the chunk storage unit 415, and the user information storage unit 420 may be implemented by the storage devices of the second storage device 400, and are provided for each tenant. The search unit 430 and the storage unit 440 of the second storage device 400 are implemented by the CPU of the second storage device 400 reading and executing programs stored in the storage devices.

[0055] The document storage unit 410 stores internal company information for each tenant. The chunk storage unit 415 stores chunks of internal company information, which are groups of internal company information that have been divided. The chunk storage unit 415 stores chunks for each piece of internal company information. The user information storage unit 420 stores user information for each tenant, relating to users belonging to that tenant.

[0056] The search unit 330 searches the conversation history storage unit 310 in response to a search request from the information processing device 200 and transmits the search results to the information processing device 200. When an operation to upload company information is performed by the administrator of the chat system 100, the storage unit 440 stores the new company information in the document storage unit 410. Also, when the storage unit 440 receives chunks of company information from the information processing device 200, it stores the chunks in the chunk storage unit 415. Details of each storage unit of the first storage device 300 and the second storage device 400 will be described later.

[0057] Next, the functions of the server device 500 will be described. The server device 500 has a large-scale language model storage unit 510, an input unit 520, and an output unit 530. The large-scale language model storage unit 510 stores a large-scale language model 511 that realizes generative AI.

[0058] The large-scale language model 511 of this embodiment is a computer language model that is generated by performing a training process using a vast amount of unlabeled text as training data and consists of an artificial neural network with many parameters. The large-scale language model 511 can capture much of the syntax and meaning of human language by being sufficiently trained with context-learning techniques such as next sentence prediction, which understands the context by determining whether sentence 1 and sentence 2 are consecutive, and a masked language model, which understands the context by masking words in a sentence and predicting the masked words from the words before and after them.

[0059] The input unit 520 inputs prompts sent from the information processing device 200 to the large-scale language model 511. The output unit 530 outputs (sends) the output data output from the large-scale language model 511 to the information processing device 200. In other words, the output unit 530 sends the output response data output from the large-scale language model 511 to the information processing device 200.

[0060] Next, the functions of terminal devices 600 and 700 will be described. Each of the terminal devices 600 and 700 includes a communication control unit 630, 730, an input receiving unit 640, 740, and a display control unit 650, 750.

[0061] Communication control units 630 and 730 control communication between terminal devices 600 and 700, respectively, and the information processing device 200. Input receiving units 640 and 740 receive input to terminal devices 600 and 700. Display control units 650 and 750 control the display on terminal devices 600 and 700.

[0062] Before describing the functions of the information processing device 200, the storage units of the first storage device 300 and the second storage device 400 will be described with reference to Figures 5 to 8.

[0063] Figure 5 shows an example of a conversation history storage unit in the first storage device. The conversation history information stored in the conversation history storage unit 310 shown in Figure 5 includes information items such as conversation ID, date and time, tenant ID, question, answer, and information source of the answer, with the values ​​of each item being associated with the information.

[0064] The value of the "Conversation ID" field is identification information used to identify the conversation. The value of the "Date and Time" field indicates the date and time when the question was entered into terminal device 600 or the date and time when the answer was output to terminal device 600. The value of the "Tenant ID" field is identification information that identifies the tenant to which the user of terminal device 600 belongs.

[0065] The value of the item "Question" indicates the data (question data) entered into the terminal device 600 as the question text. The value of the item "Answer" indicates the data (answer data) output to the terminal device 600 as the answer text.

[0066] In the following description, the data output to the terminal device 600 as an answer will be referred to as output answer data. Therefore, the conversation history information in this embodiment includes a conversation ID that identifies the conversation history information, the tenant ID of the tenant to which the user of the terminal device 600 belongs, the date and time when the question data was entered or the date and time when the output answer data was output, the question data, and the output answer data.

[0067] The value of the item "Response Source" indicates the source of the output response data. Specifically, for example, the value of the item "Response Source" is information that identifies the internal company information used as the source of the output response data. Note that the information identifying the internal company information may also be a link or similar that indicates the location where the relevant internal company information is stored.

[0068] Figure 6 shows an example of a document storage unit of the second storage device. In the document storage unit 410 of this embodiment, company information is stored for each tenant and may be managed separately for each tenant.

[0069] Furthermore, the document storage unit 410 may be divided into multiple storage areas depending on the type of company information to be stored. In other words, the document storage unit 410 may have multiple folders corresponding to the type of company information.

[0070] In the document storage unit 410, the file name of the internal company information to be stored, the folder name in which the internal company information is stored, the update date and time of the internal company information, etc., may be stored in association with the internal company information. In the example in Figure 6, it can be seen that the internal information file named "Sales Report" was stored in the "Transaction History" folder at 9:00 AM on June 24, 2024. Also, in the example in Figure 5, it can be seen that the internal information file named "Regarding Paid Leave" was stored in the "End-of-Work Regulations A" folder at 9:00 AM on June 18, 2024.

[0071] In this embodiment, folders for storing internal company information are created according to the type of internal company information, but this is not limited to this. Folders created in the document storage unit 410 may, for example, be created for each business activity within the tenant, or for each department within the tenant. Folders in the document storage unit 410 can be arbitrarily configured by the administrator of the tenant implementing the chat system 100.

[0072] Figure 7 shows an example of the chunk storage section of the second storage device. In Figure 7, the file name of the internal company information, the update date and time of the internal company information, the chunk ID that identifies the chunk, and the chunk itself are associated with each other.

[0073] Furthermore, in the example shown in Figure 7, one page of internal information from the internal document "Sales Report" is treated as a chunk, and the page number is used as the chunk ID to identify the chunk.

[0074] Furthermore, a chunk is not limited to a single page of internal company information; for example, it could be a paragraph within an internal document, or even a chapter.

[0075] Figure 8 shows an example of a user information storage unit in the second storage device. The user information stored in the user information storage unit 420 of this embodiment may be managed on a per-tenant basis.

[0076] The user information in this embodiment includes the following information items: Tenant ID, User ID, Username, Email Address, etc. The value of the "Tenant ID" item is identification information that identifies the tenant to which the user identified by the User ID belongs. The value of the "User ID" item is identification information that identifies the user of terminal device 600. The "Username" item indicates the name of the user identified by the User ID, and the value of the "Email Address" item indicates the email address of the user identified by the User ID.

[0077] In this embodiment, the user information storage unit 420 is stored in the second storage device 400. However, the second storage device 400 does not necessarily have to contain the user information storage unit 420; it is sufficient that it contains a document storage unit 410 and a chunk storage unit 415.

[0078] Now, let's return to Figure 4 and explain the functions of the information processing device 200.

[0079] The information processing device 200 of this embodiment includes an input receiving unit 251, a search request unit 252, a conversation history acquisition unit 253, an information acquisition unit 254, a prompt generation unit 255, a prompt output unit 256, an analysis instruction unit 237, an analysis result holding unit 258, an embedding unit 259, a division unit 260, and an information output unit 261. Each of these units is realized by the CPU 201 of the information processing device 200 reading and executing a program stored in the HD 204 or the like.

[0080] The input receiving unit 251 receives various inputs to the information processing device 200. Specifically, the input receiving unit 251 receives inquiry data from the terminal device 600. The input receiving unit 251 also receives instructions from the terminal device 700 to store company information in the second storage device 400.

[0081] The search request unit 252 makes search requests to the first storage device 300 and the second storage device 400. Specifically, the search request unit 252 sends a search request to the first storage device 300 for the conversation history storage device 310, using the question data as the search key. The search request unit 252 also sends a search request to the second storage device 400 for the chunk storage device 415, based on a query generated by the server device 500.

[0082] The conversation history acquisition unit 253 acquires conversation history information, which is the search result of the conversation history storage unit 310 in response to a search request from the search request unit 252, from the first storage device 300.

[0083] The information acquisition unit 254 acquires chunks, which are the search results of the chunk storage unit 415 in response to the search request from the search request unit 252, from the second storage device 400.

[0084] The prompt generation unit 255 generates a prompt to be output to the server device 500. For example, when information to be included in the prompt is obtained, the prompt generation unit 255 generates a prompt in a predetermined format that includes the obtained information.

[0085] Specifically, the prompt generation unit 255 generates a prompt requesting the creation of a search query to search the chunk storage unit 415 of the second storage device 400. The prompt generation unit 255 also generates a prompt requesting the generation of an answer to the question. Furthermore, the prompt generation unit 255 generates a prompt requesting the verbalization of objects obtained through the analysis of internal company information by the analysis server 550.

[0086] The prompt output unit 256 outputs the prompt generated by the prompt generation unit 255 to the server device 500.

[0087] The analysis instruction unit 237 requests the analysis server 550 to analyze the internal company information that is to be stored in the second storage device 400. The analysis result holding unit 258 holds the analysis results obtained from the analysis server 550. The analysis results held here may be image data showing images of objects included in the internal company information and image data showing images of object headings.

[0088] The embedding unit 259 embeds text data, which is a representation of the object's image using the large-scale language model 511, at the location where the object was placed in the company information. The splitting unit 260 divides the company information, which now has the text data represented by the language embedded at the location where the object was placed, into multiple chunks.

[0089] The information output unit 261 outputs various types of information to the outside of the information processing device 200. Specifically, for example, the information output unit 261 outputs answer data to a question to the terminal device 600. The information output unit 261 also outputs conversation history information to the first storage device 300. Furthermore, the information output unit 261 outputs multiple chunks of company information, in which verbalized data is embedded at the location where an object was placed, to the second storage device 400.

[0090] Next, the operation of the chat system 100 in this embodiment will be described. First, referring to Figure 9, the operation of the chat system 100 when it receives a command to store company information from the terminal device 700 will be described.

[0091] Figure 9 is a first sequence diagram illustrating the operation of the chat system according to the first embodiment.

[0092] In the chat system 100, the terminal device 700 displays a registration screen for registering company information via the display control unit 750 (step S901). Next, the terminal device 700 selects a folder to store the company information via the input reception unit 740 (step S902). Subsequently, the terminal device 700 accepts the selection of company information to be stored in the folder via the input reception unit 740 (step S903). Next, the terminal device 700 transmits (uploads) the selected company information to the information processing device 200 via the communication control unit 730 (step S904).

[0093] When the information processing device 200 receives company information via the input receiving unit 251, the information output unit 261 transmits the received company information to the second storage device 400 (step S905), and sends an acknowledgment notification to the terminal device 700 indicating that the company information has been received (step S906). In other words, the input receiving unit 251 is an example of a receiving unit that receives a request to register a file to be stored in the storage unit. The second storage device 400 stores the company information in a selected folder in the document storage unit 410 via the storage unit 440 (step S907).

[0094] Next, the information processing device 200 performs a process to divide the company information stored in the document storage unit 410 of the second storage device 400 into chunks. The processes from step S908 onward may be performed at a timing independent of the processes from steps S901 to S907. In other words, the process of dividing the company information into chunks may be executed asynchronously with the process of storing the company information in the document storage unit 410.

[0095] Furthermore, Figure 9 shows the operation of the chat system 100 when the internal company information stored in the document storage unit 410 includes objects. However, if the document storage unit 410 does not contain objects, the information processing device 200 may proceed to step S917, which will be described later, after step S907.

[0096] The information processing device 200 transmits an analysis instruction to the document storage unit 410 via the analysis instruction unit 237 (step S908). The analysis server 550 receives this analysis instruction and retrieves the company information stored in the document storage unit 410 from the second storage device 400 (steps S909, 910).

[0097] When the analysis server 550 acquires internal company information, it performs analysis and transmits the analysis results to the information processing device 200 (step S911). At this time, the analysis server 550 may also transmit the internal company information acquired from the second storage device 400 to the information processing device 200 along with the analysis results.

[0098] The information processing device 200 acquires the analysis results from the analysis server 550 via the information acquisition unit 254, and then stores the analysis results in the analysis result storage unit 258 (step S912). The analysis results stored in the analysis result storage unit 258 are images of objects and captions extracted from internal company information. Next, the information processing device 200 executes the processes from step S913 to step S915 as many times as there are objects included in the company information.

[0099] When the information processing device 200 has the analysis results stored, the prompt generation unit 255 generates a prompt that includes an image of the object and an image of its caption, and requests the verbalization of the image of the object (step S913). In other words, the prompt generation unit 255 generates a prompt to output the content that can be read from the object contained in the file that received the registration request. Next, the information processing device 200 sends the generated prompt to the server device 500 using the prompt output unit 256 (step S914). Details of the prompt sent to the server device 500 will be described later.

[0100] The information processing device 200, using the information acquisition unit 254, acquires text data from the server device 500, which is the result of the large-scale language model 511 verbalizing objects (step S915).

[0101] Next, the information processing device 200 uses the embedding unit 259 to embed the text data acquired in step S915 into the internal company information that was analyzed, at the location where the object was placed (step S916). With this internal company information, internal company information that includes objects, which are information other than text data, becomes internal company information consisting only of text data.

[0102] Next, the information processing device 200 divides the company information, which has been converted into text data, into chunks using the division unit 260 (step S917). Subsequently, the information processing device 200 transmits the divided chunks to the second storage device 400 using the information output unit 261 (step S918). In other words, the information output unit 261 in this embodiment functions as a registration unit that registers text data (the result output by the large-scale language model 511) in the storage unit, in which the objects have been verbalized by the large-scale language model 511.

[0103] The second storage device 400 stores the chunks received from the information processing device 200 in the chunk storage unit 415 via the storage unit 440, along with information identifying the company information containing the chunks (step S919). Therefore, all chunks stored in the chunk storage unit 415 are text data.

[0104] Next, the information processing device 200 sends a completion notification to the terminal device 700 via the information output unit 261, indicating that the registration of company information has been completed (step S920).

[0105] Next, with reference to Figures 10 to 15, the operation shown in Figure 9 will be explained in detail. Figure 10 is a diagram showing an example of a registration screen for internal company information in the first embodiment. Screen 101 shown in Figure 10 is an example of a registration screen displayed on the terminal device 700 in step S901 of Figure 9.

[0106] Screen 101 includes display areas 101A and 101B, and operation buttons 101C. Display area 101A displays a list of folders created in the document storage unit 410. In other words, display area 101A displays a list of folder names of folders created within the document storage unit 410.

[0107] In this embodiment, the folder selected from the list of folders displayed in the display area 101A becomes the storage location for the internal company information to be uploaded.

[0108] Display area 101B displays a message prompting the user to select the company information to be uploaded, and operation buttons 101D for selecting the company information to be uploaded.

[0109] Operation button 101C is an operation button for uploading company information selected in the display area 101B.

[0110] In the example shown in Figure 10, the folder name "Transaction History" is selected in display area 101A, so the storage location for the uploaded internal company information will be the Transaction History folder.

[0111] In this embodiment, when the operation button 101D is selected in the display area 101B, a list of company information managed by the terminal device 700 is displayed. After the company information is specified, when the operation button 101C is operated, the specified company information is transmitted to the second storage device 400 via the information processing device 200. The specified company information is then stored in the "Transaction History" folder created in the document storage unit 410 of the second storage device 400.

[0112] Thus, in this embodiment, when storing company information in the second storage device 400, it is possible to select a folder to which the company information will be stored.

[0113] In this embodiment, if a predetermined operation is performed on screen 101, a new folder may be created in the document storage unit 410 of the second storage device 400. In other words, the folders in the document storage unit 410 may be edited arbitrarily by the administrator of the chat system 100 or the like.

[0114] Figure 11 shows an example of internal company information in the first embodiment. The following description will explain the case where the internal company information 111 shown in Figure 11 is uploaded to the chat system 100.

[0115] The internal company information 111 shown in Figure 11 can be explained as, for example, page 5 of a file named "Sales Report.pdf".

[0116] The internal company information 111 shown in Figure 11 includes text data 111A, a graph heading 111B, and a bar graph 111C. Here, the heading 111B and the bar graph 111C are data other than text data. In other words, the internal company information 111 includes two objects. In this embodiment, when the analysis server 550 receives an instruction from the information processing device 200 to analyze the company information 111, it performs the analysis of the company information 111 and transmits the analysis results to the information processing device 200.

[0117] Figure 12 illustrates the results of the analysis of internal company information by the analysis server. In this embodiment, when the analysis server 550 receives an analysis instruction for internal company information 111 containing an object, it recognizes the rectangle containing the heading 111B and the rectangle containing the bar graph 111C in the internal company information 111, and extracts them as images. In other words, the analysis server 550 recognizes the rectangle that the object encloses and extracts it as an image.

[0118] Furthermore, the analysis server 550 extracts the coordinates of four points for each rectangle containing the heading 111B and the rectangle containing the bar graph 111C. These coordinates may be those with the top-left corner of the screen displaying the company information 111 as the origin.

[0119] The analysis server 550 then transmits to the information processing device 200, as analysis results, image data showing the image of heading 111B, image data showing the image of bar graph 111C, and the coordinates of the four points in the rectangle containing each of these.

[0120] Note that the method for extracting object images on the analysis server 550 is not limited to the example described above. The analysis results do not need to include the coordinates of the four points of the rectangle. The analysis results only need to include image data showing the image of the object contained in the internal company information 111.

[0121] In this embodiment, when the information processing device 200 obtains analysis results from the analysis server 550, the prompt generation unit 255 generates a prompt requesting the verbalization of the object and sends it to the server device 500. The prompts generated by the prompt generation unit 255 are described below.

[0122] Figure 13 shows an example of a prompt in the first embodiment. Prompt 131 shown in Figure 13 is an example of a prompt generated in step S913 of Figure 9.

[0123] The prompt 131 includes text data 131A, an image 131B of the heading 111B obtained as an analysis result by the analysis server 550, and an image 131C of the bar graph 111C.

[0124] Furthermore, text data 131A includes messages that request the large-scale language model 511 to convert the images contained in the prompt into searchable text, and messages that request that everything that can be read from the images be written into text. In other words, text data 131A includes messages that cause the content that can be read from the objects contained in the file to be translated into language and output. Note that "writing into text" means expressing it in text data.

[0125] In the example in Figure 13, text data 131A requests that all information that can be gleaned from images 131B and 131C be converted into text data.

[0126] In this embodiment, the information processing device 200 transmits the prompt 131 to the server device 500. The server device 500 inputs the prompt 131 to the large-scale language model 511, and transmits the text data output by the large-scale language model 511 as a response to the prompt 131 to the information processing device 200. The text data output from the large-scale language model 511 as a response to the prompt 131 is text data that shows the content of images 131B and 131C translated into language. In the example shown in Figure 13, prompt 131 includes, but is not limited to, an image which is an example of an object, and a message for outputting the content that can be read from the object. The prompt generated in step S913 of Figure 9 may consist only of text data 131A and may not include any objects. In this case, the data management device 200 may send the object along with the prompt generated in step S913 to the server device 500 via the information output unit 261. In other words, the information output unit 261 of this embodiment also functions as a transmission unit that sends the prompt generated by the prompt generation unit 255 and the objects contained in the file to the large-scale language model 511.

[0127] Figure 14 shows an example of the result of object languageization by a large-scale language model in the first embodiment. The text data 141 shown in Figure 14 is an example of the result of the large-scale language model 511 languageization of the bar graph 111C. The text data 141 is received by the information processing device 200 from the server device 500 in step S915 of Figure 9.

[0128] Text data 141 includes text data 141A and text data 141B. Text data 141A is text data that verbalizes what can be read from the image of heading 111B. Text data 141B is text data that verbalizes what can be read from the image of bar graph 111C.

[0129] In this embodiment, by inputting a prompt to the large-scale language model 511 that includes an image of an object contained in internal company information and requests the object image to be translated into language, text data in which the object image has been translated into language can be obtained.

[0130] When the information processing device 200 of this embodiment acquires the text data 141, it replaces the heading 111B and bar graph 111C in the original company information 111 with the text data 141.

[0131] In other words, the information processing device 200, using the embedding unit 259, removes the heading 111B from the company information 111 and embeds the text data 141A in the position where the heading 111B was located. The embedding unit 259 also removes the bar graph 111C from the company information 111 and embeds the text data 141B in the position where the bar graph 111C was located.

[0132] In this embodiment, the processing by the embedded unit 259 replaces the internal company information, including objects, with verbalized internal company information. In the following description, the process of replacing objects included in the internal company information with text data obtained from the large-scale language model 511 may be referred to as verbalizing the internal company information.

[0133] Figure 15 shows an example of verbalized internal company information. The internal company information 111-1 shown in Figure 15 represents the case where the internal company information 111 shown in Figure 11 is verbalized in step S916 of Figure 9. In other words, internal company information 111-1 is internal company information that has been verbalized from internal company information 111.

[0134] In internal information 111-1, text data 141A is placed where heading 111B was placed in internal information 111, and text data 141B is placed where bar graph 111C was placed.

[0135] In this embodiment, when internal company information includes objects, the objects are replaced with text data to verbalize the internal company information. Then, in this embodiment, the verbalized internal company information is divided into chunks, and each chunk is stored in the chunk storage unit 415 of the second storage device 400.

[0136] In this embodiment, internal company information, including objects, can be used as a source of information in the generation of response data by the large-scale language model 511, thereby improving the accuracy of responses to questions.

[0137] Next, referring to Figure 16, the operation of the chat system 100 when question data is input from the terminal device 600 to the chat system 100 will be explained.

[0138] In the chat system 100 of this embodiment, when question data is input from the terminal device 600, answer data is output for the input question data, using the chunks stored in the chunk storage unit 415 as the information source. In the following description, as shown in Figure 16. Figure 16 is a second sequence diagram illustrating the operation of the chat system according to the first embodiment.

[0139] In the chat system 100, the terminal device 600 displays the home screen of the chat system 100 (step S1601). Subsequently, the terminal device 600 receives a selection of a folder containing the question data and related company information via the input reception unit 640 (step S1602).

[0140] Next, the terminal device 600 receives the input of question data via the input reception unit 640 (step S1603). Subsequently, the terminal device 600 transmits the input question data to the information processing device 200 via the communication control unit 630 (step S1604). The question data transmitted here may include the tenant ID of the tenant to which the user of the terminal device 600 belongs.

[0141] When the information processing device 200 receives the input of question data via the input receiving unit 251, the search request unit 252 sends a search request including the tenant ID to the first storage device 300 (step S1605).

[0142] When the first storage device 300 receives this search request, the search unit 330 searches the conversation history storage unit 310 (step S1606).

[0143] The processing of the search unit 330 will now be explained. The search unit 330 identifies conversation history information that includes the tenant ID included in the search request from the conversation history information stored in the conversation history storage unit 310, and obtains conversation history information related to the question and content indicated by the question data as a search result from the identified conversation history information.

[0144] Specifically, the search unit 330 identifies conversation history information including the tenant ID included in the search request, and from the identified conversation history information, it obtains conversation history information that continued up until immediately before the question data was entered in step S901.

[0145] The conversation history information that continued until just before the question data was entered in step S1603 is conversation history information in which the time the question data was entered and the time the output response data was output immediately before this question data was entered are within a certain period of time. In this embodiment, if the next question data is entered within a certain period of time after the output response data has been output, the series of questions and answers are assumed to be responses concerning a common inquiry.

[0146] In this embodiment, by acquiring conversation history information in this way, it is possible to obtain conversation history information related to the question indicated by the question data and its content.

[0147] When the search unit 330 of the first storage device 300 searches the conversation history storage unit 310, it transmits the search result to the information processing device 200, and the information processing device 200 acquires the conversation history information using the conversation history acquisition unit 253 (step S1607).

[0148] Here, when conversation history information is obtained from the first storage device 300 as a search result, the information processing device 200 performs the following steps S1608 to S1610.

[0149] In addition, conversation history information is acquired when, before the question data entered in step S1906 is entered, question data related to the question indicated by the question data entered in step S1603 is entered, and corresponding output answer data is output. In this case, conversation history information including this question data and output answer data is stored in the conversation history storage unit 310.

[0150] Furthermore, if conversation history information is not obtained from the first storage device 300 as a search result, the information processing device 200 proceeds to step S1611. The case in which conversation history information is not obtained is when the question data entered in step S1603 indicates the first question in the inquiry.

[0151] The information processing device 200 generates a prompt in step S1608 using the prompt generation unit 255 to request the generation of a query for searching the document storage unit 410. At this time, the prompt generation unit 255 includes the question data entered in step S1603 and the conversation history information acquired by the conversation history acquisition unit 253, and generates a prompt requesting the generation of a query based on the question data and the conversation history information.

[0152] Next, the information processing device 200 sends a prompt to the server device 500 requesting the generation of a query via the prompt output unit 256 (step S1609).

[0153] The server device 500 inputs a prompt to the large-scale language model 511 via the input unit 520, and sends the output data output from the large-scale language model 511 as a query to the information processing device 200 via the output unit 530 (step S1610).

[0154] The information processing device 200, via the search request unit 252, sends a search request to the chunk storage unit 415 of the document storage unit 410 of the second storage device 400 for chunks that can serve as information sources when generating output answer data for question data (step S1611).

[0155] Specifically, if the search request unit 252 has obtained a query in step S1610, it sends a search request based on the query to the second storage device 400. If the search request unit 252 has not obtained a query, that is, if it has not obtained conversation history information in step S1607, it sends a search request including the question data entered in step S1603 to the second storage device 400.

[0156] When the second storage device 400 receives a search request, the search unit 430 searches the chunk storage unit 415 of the second storage device 400 (step S1612), and transmits the search result chunks to the information processing device 200 (step S1613).

[0157] When the information processing device 200 acquires a chunk of search results using the information acquisition unit 254, the prompt generation unit 255 generates a prompt requesting the generation of an answer to the question data using the chunk of search results as the information source (step S1614), and sends it to the server device 500 (step S1615).

[0158] When the server device 500 receives this prompt, the input unit 520 inputs the prompt to the large-scale language model 511, and the output unit 530 transmits the output data output from the large-scale language model 511 as response output data to the information processing device 200 (step S1616).

[0159] When the information processing device 200 acquires output response data, the information output unit 261 outputs the acquired output response data to the terminal device 600 (step S1617). The terminal device 600 displays the output response data on the display 618 using the display control unit 650 (step S1618). At this time, the display control unit 650 displays the output response data on the display 618 in association with the question data.

[0160] Furthermore, the information processing device 200, via its information output unit 261, sends a request to the first storage device 300 to save conversation history information, which includes the question data received in step S1603 and the output response data output to the terminal device 600 in step S1616 (step S1619). Here, the request to save conversation history information includes the tenant ID of the tenant to which the user of the terminal device 600 belongs.

[0161] In the first storage device 300, the storage unit 340 stores new conversation history information in the conversation history storage unit 310 (step S1620). Specifically, the storage unit 340 assigns a new conversation ID and date and time to the question data, output answer data, and tenant ID acquired from the information processing device 200, and stores them in the conversation history storage unit 310 as conversation history information. In this embodiment, the conversation history storage unit 310 is updated in this way.

[0162] In the following explanation, the operation of the chat system 100 described in Figure 16 may be referred to as the question-answering process. The question-answering sequence refers to the process from step S1601 to step S1620 in Figure 16 of the chat system 100.

[0163] The following describes an example of how the home screen displayed on the terminal device 600 of this embodiment looks when question data is entered, with reference to Figure 17. Figure 17 is a diagram showing an example of how the terminal device looks when question data is entered in the first embodiment.

[0164] The screen 171 shown in Figure 17 is an example of a screen displayed on the terminal device 600 in step S1618 of Figure 16.

[0165] Screen 171 includes a display area 101A, an input field 171a, operation buttons 171b, and a display area 171c. Display area 101A displays a list of folders located in the document storage unit 410 of the second storage device 400.

[0166] Input field 171a is an input field for the user of the terminal device 600 to enter a question. Operation button 171b is an operation button for sending question data, which represents the question entered in input field 171a, to the information processing device 200.

[0167] Display area 171c displays question data indicating the question entered by the user of terminal device 600 in input field 171a, and answer data to the question generated by the large-scale language model 511. In other words, display area 171c displays the content of the conversation between the user of terminal device 600 and the chat system 100.

[0168] The screen 171 shown in Figure 17 indicates the state in the display area 101A where a folder has been selected, a question has been entered in the input field 171a, and the operation button 71b has been operated, after which the terminal device 600 has received answer data indicating the answer to the question.

[0169] In other words, in this embodiment, when a user of the terminal device 600 enters a question, they can select a folder that is likely to contain company information related to their question, and thereby obtain an answer using the company information stored in the selected folder as the source of information.

[0170] Furthermore, in this embodiment, by allowing the user of the terminal device 600 to select a folder, the search target of the search unit 430 in step S1612 of Figure 6 can be limited to only the chunks of company information stored in the selected folder, thereby reducing the load on the search process.

[0171] In the example shown in Figure 17, display area 171c includes display areas 171d and 171e. Display area 171d displays question data indicating a question entered by the user of terminal device 600. Display area 171d also displays answer data indicating the answer to the question.

[0172] Furthermore, display area 171e includes display areas 171f and 171g. Display area 171f displays answer data indicating the answer to the question, and display area 171g displays information that identifies the internal company information used as the source of the answer. The information that identifies the internal company information may be the name of the internal company information and a link indicating the storage location of the internal company information. In that case, the information indicating the storage location of the internal company information is information indicating the storage location of the internal company information in the document storage unit 410.

[0173] Furthermore, the internal company information that serves as the source of the answer is the information stored in the folder selected from the list of folders displayed in display area 101A.

[0174] In the example in Figure 17, the question displayed in display area 171d is, "What is the sales amount to Company C?". The answer displayed in display area 171f is, "Approximately 12.5 billion yen." The information that identifies the company's internal information, displayed in display area 171g, is a link indicating the location of "Sales Report.pdf#page5".

[0175] In this embodiment, when a link displayed in the display area 171g is selected in the terminal device 600, the information processing device 200 may use the information acquisition unit 254 to acquire the selected company information, and the information output unit 261 may use it to display the acquired company information on the terminal device 600. In this embodiment, "Sales Report.pdf#page5" is the internal company information 111 shown in Figure 11. Therefore, when a link displayed in the display area 171g on the terminal device 600 is selected, the information processing device 200 uses the information acquisition unit 254 to acquire the internal company information 111 from the document storage unit 410 of the second storage device 400 and displays it on the screen 171. The internal company information 111 displayed here includes the objects, namely the heading 111B and the bar graph 111C.

[0176] Therefore, it can be seen that the response data displayed in display area 171g is information read from an object included in internal company information 111.

[0177] Thus, in this embodiment, it is possible to generate answers to questions using internal company information, including objects, as a source of information. Therefore, according to this embodiment, information including objects in user-specific information can also be used as a source of information for generating answers to questions using the large-scale language model 511, thereby improving the accuracy of the answers to questions.

[0178] (Second Embodiment) The first embodiment will be described below with reference to the drawings. The second embodiment differs from the first embodiment in that, when the internal company information includes an object and there is supplementary information such as a description of the object, the prompt requesting the object to be verbalized also includes the supplementary information. In the following description of the second embodiment, the differences from the first embodiment will be explained, and components having the same functional configuration as in the first embodiment will be given the same reference numerals as those used in the description of the first embodiment, and their explanations will be omitted.

[0179] Figure 18 shows an example of internal company information in the second embodiment. The internal company information 181 shown in Figure 18 is an example of internal company information uploaded to the information processing device 200 in step S904 of Figure 9.

[0180] Internal information 181 includes text data 111A, a graph heading 111B, a bar graph 111C-1 which is one of the objects, and text data 111D which explains what the bar graph 111C-1 represents. In other words, text data 111D is supplementary information to the bar graph 111C-1.

[0181] In this embodiment, the prompt generation unit 255 of the information processing device 200 includes the text data 111D in a prompt requesting the large-scale language model 511 to verbalize an object.

[0182] Figure 19 shows an example of a prompt in the second embodiment. Prompt 191 shown in Figure 19 is an example of a prompt generated in step S913 of Figure 9.

[0183] Prompt 191 includes text data 131A, image 131B of the heading 111B obtained as an analysis result by the analysis server 550, image 131C-1 of the bar graph 111C-1, and text data 191A.

[0184] Text data 191A is text data that requests reference to text data 111D contained in internal company information 181. In other words, text data 191A is text data that requests reference to supplementary information that explains what bar graph 111C-1 represents when verbalizing the object.

[0185] In this embodiment, by including supplementary information about the object and text data requesting a reference to the supplementary information in the prompt requesting the large-scale language model 511 to verbalize the object in more detail, the large-scale language model 511 can be made to verbalize the object in more detail. Therefore, according to this embodiment, the accuracy of the answer to the question can be improved.

[0186] Figure 20 shows an example of the result of object languageization by a large-scale language model in the second embodiment. The text data 201 shown in Figure 20 is an example of the result of the large-scale language model 511 languageization of the bar graph 111C-1. The text data 201 is received by the information processing device 200 from the server device 500 in step S915 of Figure 9.

[0187] Text data 201 includes text data 201A and text data 201B. Text data 201A includes text data that verbalizes what can be read from the image of heading 111B, and part of text data 111D, which is supplementary information for bar graph 111C-1. More specifically, text data 201A includes text data that is part of text data 111D, which reads, "Sales performance by customer for the generation AI business in fiscal year 2024."

[0188] Text data 201B includes text data that verbalizes what can be read from the image of bar graph 111C-1, and a portion of text data 111D, which is supplementary information to bar graph 111C-1. More specifically, text data 201B includes the text data "Company C has sales of approximately 7 billion yen due to special factors," which is a portion of text data 111D.

[0189] Thus, in this embodiment, by inputting a prompt to the large-scale language model 511 requesting the verbalization of an object based on supplementary information, it is possible to obtain text data in which the object is verbalized in detail based on the supplementary information.

[0190] Next, with reference to Figure 21, an example of the display of the terminal device 600 in this embodiment will be described. Figure 21 is a diagram showing an example of the display of the terminal device in the second embodiment.

[0191] The screen 171A shown in Figure 21 is an example of a screen displayed on the terminal device 600 in step S1618 of Figure 16.

[0192] Screen 171A includes a display area 101A, an input field 171a, operation buttons 171b, and a display area 171c.

[0193] In this embodiment, the display area 171c displays question data indicating the question entered in the input field 171a, and answer data for the question generated by the large-scale language model 511.

[0194] In the example shown in Figure 21, display area 171c includes display areas 171h and 171i. Display area 171h displays question data indicating a question entered by the user of terminal device 600. Display area 171i displays answer data indicating the answer to the question.

[0195] Furthermore, display area 171i includes display areas 171j and 171k. Display area 171j displays answer data indicating the answer to the question, and display area 171k displays information identifying the internal company information used as the source of the answer, and a link indicating the location where the internal company information is stored.

[0196] In the example in Figure 21, the question displayed in display area 171h is, "What amount of sales to Company C was due to special factors?" The answer displayed in display area 171j is, "Of the sales to Company C, the amount due to special factors is approximately 7 billion yen." The information identifying the internal company, displayed in display area 171k, is shown as the file name of internal company information 111, "Sales Report.pdf#page5," and a link indicating the storage location of internal company information 181.

[0197] In this embodiment, when a link displayed in the display area 171k is selected, the corresponding company information may be displayed on screen 171A.

[0198] In this embodiment, "Sales Report.pdf#page5" is the internal company information 181 shown in Figure 18. Therefore, when the link displayed in the display area 171g is selected, the internal company information 181 shown in Figure 18 is displayed on screen 171A. This internal company information 181 includes the object bar graph 111C-1 and supplementary information for the bar graph 111C-1, which indicates that Company C has sales of approximately 7 billion yen due to special factors.

[0199] Therefore, it can be seen that the response data displayed in display area 171k was generated from information read from objects included in internal company information 181 and supplementary information about those objects.

[0200] In this embodiment, by including supplementary information about the object in the prompt requesting the object's verbalization, the content represented by the object can be verbalized in detail. Therefore, in this embodiment, the accuracy of responses using internal company information, including the object, as the source of information can be improved.

[0201] (Third embodiment) A third embodiment will be described below with reference to the drawings. The third embodiment differs from the first embodiment in that, when appropriate answer data is not output for question data entered by the user of the terminal device 600, the content of the question indicated by the question data is included in a prompt requesting verbalization. In the following description of the third embodiment, the differences from the first embodiment will be explained, and components having the same functional configuration as in the first embodiment will be given the same reference numerals as those used in the description of the first embodiment, and their explanation will be omitted.

[0202] Figure 22 is a sequence diagram illustrating the operation of the chat system according to the third embodiment. The process shown in Figure 22 may be executed in the question answering process shown in Figure 16 when appropriate answer data cannot be obtained from the large-scale language model 511 in response to the question data input from the terminal device 600.

[0203] If appropriate response data cannot be obtained, for example, the response data for the question data is "No response has been registered." In other words, if appropriate response data cannot be obtained, the information processing device 200 may be unable to obtain a chunk as a search result from the chunk storage unit 415 in the second storage device 400.

[0204] In this embodiment, the information processing device 200 may analyze the contents of the response data acquired from the server device 500, and if the contents of the response data indicate that a chunk serving as the information source for the response is not stored, it may execute the processing from step S2201 onwards in Figure 22.

[0205] Furthermore, in this embodiment, for example, the user of the terminal device 600 that input the question data may evaluate whether the content of the answer data is appropriate or not. Also, in this embodiment, if the user of the terminal device 600 evaluates the answer content as inappropriate, the processing from step S2201 onwards in Figure 22 may be executed.

[0206] Furthermore, the processing from step S2201 onwards in Figure 22 may be executed following the question answering process shown in Figure 16 if appropriate answer data could not be obtained.

[0207] In the chat system 100 of this embodiment, the information processing device 200 sends an analysis instruction to the analysis server 550 of the company information via the analysis instruction unit 257 (step S2201). At this time, the analysis instruction unit 257 may issue an analysis instruction to all of the company information stored in the document storage unit 410. Alternatively, the analysis instruction unit 257 may issue an analysis instruction to the company information stored in the folder selected in step S1602 of Figure 16 from among the company information stored in the document storage unit 410.

[0208] The process from step S2202 to step S2205 in Figure 22 is the same as the process from step S909 to step S912 in Figure 9, so the explanation is omitted.

[0209] Next, the information processing device 200 executes the processes from step S2206 to step S22018 for each object included in the company information.

[0210] When the information processing device 200 has the analysis results stored, the prompt generation unit 255 generates a prompt that includes an image of the object, an image of its caption, and question data for when appropriate answer data cannot be obtained, and requests the verbalization of the image of the object (step S2206). Details of the prompt generated in step S2206 will be described later.

[0211] Next, the information processing device 200 sends the generated prompt to the server device 500 via the information output unit 261 (step S2207).

[0212] The process from step S2207 to step S2213 in Figure 22 is the same as the process from step S914 to step S920 in Figure 9, so the explanation is omitted.

[0213] Next, with reference to Figure 23, the prompt generated in step S2206 of Figure 22 will be described. Figure 23 shows an example of a prompt in a third embodiment.

[0214] Prompt 231 includes text data 131A, 231A, image 131B of the heading 111B obtained as an analysis result by the analysis server 550, and image 131C of the bar graph 111C.

[0215] Text data 231A represents the question data for which appropriate response data could not be obtained. Note that text data 231A may include all question data for which appropriate response data could not be obtained.

[0216] In this embodiment, by including the content of questions for which appropriate answers were not previously obtained in the prompt requesting the verbalization of objects contained in internal company information, the accuracy of verbalization by the large-scale language model 511 can be improved. Therefore, according to this embodiment, when a question for which appropriate answers were not previously obtained is entered again into the chat system 100, the likelihood of being able to provide an appropriate answer can be increased.

[0217] (Fourth embodiment) A fourth embodiment will be described below with reference to the drawings. The fourth embodiment differs from the first embodiment in that the prompt requesting the verbalization of internal company information includes information indicating how to read the objects contained in the internal company information. In the following fourth embodiment, the differences from the first embodiment will be explained, and components having the same functional configuration as in the first embodiment will be given the same reference numerals as those used in the description of the first embodiment, and their explanation will be omitted.

[0218] Some objects included in internal company information are created according to predetermined rules. When such objects are to be represented in language using the large-scale language model 511, it is necessary to train the large-scale language model 511 with these predetermined rules.

[0219] Therefore, in this embodiment, when an object included in internal company information is created according to a predetermined rule, the prompt requesting the verbalization of the internal company information includes information indicating the predetermined rule. An example of information indicating the predetermined rule is information indicating how to read the object.

[0220] Figure 24 shows an example of internal company information in the fourth embodiment. The internal company information 112 shown in Figure 24 includes a work process chart 112A and a heading 112B for the work process chart 112A. The work process chart 112A and the heading 112B are data other than text data and are objects included in the internal company information 112. Furthermore, the work process chart 112A is an object created based on predetermined rules.

[0221] In this embodiment, when uploading internal company information containing such objects, the administrator of the terminal device 700 uploading the internal company information is required to input a predetermined rule for reading the object. The registration screen for internal company information in this embodiment is described below.

[0222] Figure 25 shows an example of a registration screen for internal company information in the fourth embodiment. Screen 241 shown in Figure 25 is an example of a registration screen displayed on the terminal device 700 in step S903 of Figure 9.

[0223] Screen 241 includes display areas 101A and 101B, operation button 101C, display area 101E, and input field 101F.

[0224] Display area 101E displays a message prompting the user to specify how to read the objects included in the internal company information to be uploaded. Input field 101F is an input field for entering how to read the objects.

[0225] In this embodiment, when the company information to be uploaded is selected on screen 241, and the method for reading the objects included in the company information is entered in input field 101F, and then the operation button 101C is selected, the terminal device 700 transmits the selected company information along with the information indicating the method for reading the objects entered in input field 101F. The information indicating the method for reading the objects is in text data format.

[0226] When the information processing device 200 receives internal company information and information indicating how to read an object, it includes an image of the object, which is the result of analyzing the internal company information obtained from the analysis server 550, and information indicating how to read the object, in a prompt requesting the verbalization of the internal company information.

[0227] Specifically, in this embodiment, the image of the work process chart 112A, the image of the heading 112B, and the text data entered in the input field 101F are included in the prompt requesting the verbalization of the company information 112.

[0228] In this embodiment, by including an image of an object and information indicating how to read the object in the prompt input to the large-scale language model 511, the large-scale language model 511 can be trained to read objects. Therefore, according to this embodiment, even internal company information containing objects created according to special rules can be verbalized and used as a source of information for answering questions.

[0229] Each of the embodiments described above can be applied, for example, to a chatbot system for a product support department in the manufacturing industry.

[0230] By applying this embodiment to such a system, the chat system for the product support department can use diagrams and charts such as performance graphs, structural diagrams, and flowcharts contained in product manuals, troubleshooting guides, and technical specifications as information sources when outputting answers to customer questions. Therefore, the accuracy of answers to questions can be improved.

[0231] Furthermore, each of the embodiments described above can be applied, for example, to a chat system that receives inquiries regarding product structure. By applying this embodiment to such a system, when a question such as "Please tell me about the structure of the cooling system of the new product X. How is it different from the previous model?" is entered, the system can output an answer based on information that verbalizes the structural diagrams of the old and new cooling systems included in the technical specifications, etc. Therefore, according to this embodiment, the accuracy of the answers to questions can be improved.

[0232] Furthermore, each of the embodiments described above can be applied, for example, to a chat system that receives inquiries regarding troubleshooting. By applying this embodiment to such a system, when a question such as, "My new product X won't power on, please tell me the possible causes and how to deal with them," is entered, an answer can be output based on information that verbalizes the power system diagnostic flowchart, etc. Therefore, according to this embodiment, the accuracy of the answers to questions can be improved.

[0233] In this way, by applying this embodiment to various chat systems, it becomes possible to generate comprehensive answers from text data and chart data. Furthermore, according to this embodiment, it is possible to provide easy-to-understand answers to user questions based on relevant charts and graphs. Moreover, according to this embodiment, by utilizing the quantitative data and visual information contained in the charts and graphs, it is possible to output more accurate and specific answers.

[0234] Furthermore, according to this embodiment, by automatically analyzing and translating objects (chart data) into language, the manual work required to create explanatory texts for charts and graphs can be reduced.

[0235] Furthermore, in this embodiment, if an appropriate answer is not obtained, a prompt including the content of the question at that time is input into the large-scale language model 511 to verbalize internal company information, thereby improving the accuracy of answers over time and reducing the need for human support. In addition, in this embodiment, when new product information or technical documents are added, the added information, including figures and tables, is verbalized and used as a source of information, so it can be immediately reflected in the answer.

[0236] Furthermore, each function of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" in this specification includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each of the functions described above.

[0237] Furthermore, the apparatus described in each embodiment represents only one of several computing environments for carrying out the embodiments disclosed herein.

[0238] In one embodiment, the information processing device 200 includes a plurality of computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network or shared memory, and perform the processing disclosed herein. Similarly, the information processing device 200 may include a plurality of computing devices configured to communicate with each other.

[0239] Furthermore, the information processing device 200 can be configured to share the disclosed processing steps in various combinations. For example, a process performed by a predetermined unit may be performed by the information processing device 200. Similarly, the functions of a predetermined unit may be performed by the information processing device 200. Also, each element of the information processing device 200 may be combined into a single server device or divided among multiple devices.

[0240] The information processing device 200 may be any device equipped with communication capabilities. For example, the information processing device 200 may be an output device such as a PJ (Projector), IWB (Interactive White Board: an electronic whiteboard with the ability to communicate with each other), or digital signage; a HUD (Head Up Display) device; industrial machinery; imaging devices; sound collection devices; medical equipment; networked home appliances; automobiles (Connected Cars); notebook PCs (Personal Computers); mobile phones; smartphones; tablet terminals; game consoles; PDAs (Personal Digital Assistants); digital cameras; wearable PCs; or desktop PCs.

[0241] Examples of the present invention are as follows: <1> In an information processing device that receives question data input from a user via a terminal device, searches a file stored in the memory unit, and outputs answer data, A receiving unit that receives a request to register a file to be stored in the aforementioned storage unit, A prompt generation unit generates a prompt to output content that can be read from the object contained in the file that received the registration request, A transmission unit that sends the prompt generated by the prompt generation unit and the object to a large-scale language model, A registration unit registers the results output by the large-scale language model to the storage unit, associating them with the file that received the registration request. An information processing device having <2> An input receiving unit that accepts the input of question data, A prompt output unit that, upon receiving the input of the aforementioned question data, outputs a prompt to the large-scale language model requesting the generation of answer data for the aforementioned question data based on the aforementioned file, The system includes an information output unit that outputs to a terminal device the response data output from the large-scale language model based on the prompt output from the prompt output unit, and information indicating the storage location of the file used as the source of the response data. The aforementioned information output unit is When the information indicating the storage location displayed on the terminal device is selected, and the file used as the source of the response data contains an object that is data other than text data, the file containing the object is displayed. <1> The information processing device described. <3> The prompt generation unit, The system includes a prompt generation unit that generates a prompt requesting the verbalization of an image, which is an object extracted from a file that has received a registration request to the storage unit, The prompt output unit is, The large-scale language model is output the prompt requesting the language translation of the image of the object. The aforementioned information output unit is The text data, which is a representation of the object obtained from the large-scale language model, is output to the storage unit. <2> The information processing device described. <4> An embedding unit that embeds the object into text data representing the object in language at the location where the object is located in the file for which a registration request to the storage unit has been received, The file, after the text data representing the object has been embedded, is divided into predetermined units of information, and includes a division unit. The aforementioned information output unit is The information divided into the predetermined units is output to the storage unit. <3> The information processing device described. <5> The prompt generation unit, The prompt requesting the verbalization of the image of the object includes an image of the object's caption extracted from the file that received the registration request to the storage unit. <3> or <4> The information processing device described. <6> The prompt generation unit, The prompt requesting the verbalization of the image of the object includes supplementary information about the object extracted from the file that received the registration request to the storage unit. <3> ~ <5> An information processing device as described in any one of the following items. <7> The prompt generation unit, Depending on the content of the response data output from the large-scale language model to the question data, the prompt requesting the verbalization of the image of the object includes the content of the question indicated by the question data. <3> ~ <6> An information processing device as described in any one of the following items. <8> The prompt generation unit, When a request to register a file to the storage unit is received along with information specifying how to read the objects contained in the file, The prompt requesting the verbalization of the image of the object includes the image of the object and information specifying how to read the object. <3> ~ <7> An information processing device as described in any one of the following items. <9> An information processing system including an information processing device and a terminal device that communicates with the information processing device, The aforementioned information processing device is In an information processing device that receives question data input from a user via the aforementioned terminal device, searches a file stored in the storage unit and outputs answer data, A receiving unit that receives a request to register a file to be stored in the aforementioned storage unit, A prompt generation unit generates a prompt to output content that can be read from the object contained in the file that received the registration request, A transmission unit that sends the prompt generated by the prompt generation unit and the object to a large-scale language model, An information processing system comprising: a registration unit that registers the results output by the large-scale language model in association with the file that received the registration request and stores them in the storage unit. <10> An information processing method by an information processing device that, upon receiving input of question data from a user via a terminal device, searches a file stored in a memory unit and outputs answer data, The aforementioned information processing device The storage unit receives a request to register a file to be stored in the aforementioned storage unit. A prompt is generated to output the content that can be read from the object contained in the file that received the registration request. The generated prompt and object are sent to the large-scale language model. An information processing method that registers the results output by the large-scale language model in the storage unit, associating them with the file for which the registration request was received. <11> When a user inputs question data via a terminal device, the information processing device searches a file stored in the memory unit and outputs the answer data. The storage unit receives a request to register a file to be stored in the aforementioned storage unit. A prompt is generated to output the content that can be read from the object contained in the file that received the registration request. The generated prompt and object are sent to the large-scale language model. A program that performs a process to register the results output by the large-scale language model in the storage unit, associating them with the file that received the registration request.

[0242] Although the present invention has been described above based on various embodiments, the present invention is not limited to the requirements shown in the above embodiments. These points can be modified as long as they do not impair the spirit of the present invention, and can be appropriately determined according to their application. [Explanation of Symbols]

[0243] 100 Chat Systems 200 Information Processing Devices 255 Prompt generation unit 257 Analysis instruction section 258 Analysis result holding section 259 Embedded part 260 Division 261 Information Output Unit 300 First memory device 310 Conversation history memory unit 400 Second storage device 410 Document Storage Unit 415 Chunk Memory Unit 420 User Information Storage Unit 500 Server Devices 511 Large-scale language models 600, 700 terminal devices [Prior art documents] [Patent Documents]

[0244] [Patent Document 1] Japanese Patent Publication No. 2024-056039

Claims

1. In an information processing device that receives question data input from a user via a terminal device, searches a file stored in the memory unit, and outputs answer data, A receiving unit that receives a request to register a file to be stored in the aforementioned storage unit, A prompt generation unit generates a prompt to output content that can be read from the object contained in the file that received the registration request, A transmission unit that sends the prompt generated by the prompt generation unit and the object to a large-scale language model, A registration unit registers the results output by the large-scale language model to the storage unit, associating them with the file that received the registration request. An information processing device having

2. An input receiving unit that receives the aforementioned question data, A prompt output unit that, upon receiving the input of the aforementioned question data, outputs a prompt to the large-scale language model requesting the generation of answer data for the aforementioned question data based on the aforementioned file, The system includes an information output unit that outputs to a terminal device the response data output from the large-scale language model based on the prompt output from the prompt output unit, and information indicating the storage location of the file used as the source of the response data. The aforementioned information output unit is The information processing apparatus according to claim 1, wherein when the information indicating the storage location displayed on the terminal device is selected, and the file used as the source of the response data contains the object which is data other than text data, the apparatus displays the file containing the object.

3. The prompt generation unit, A prompt containing an image which is an object extracted from a file that has received a registration request to the storage unit, generates the prompt which requests the language translation of the image of the object, The prompt output unit is, The large-scale language model is output the prompt requesting the language translation of the image of the object. The aforementioned information output unit is The information processing apparatus according to claim 2, which outputs text data that is a language representation of the object obtained from the large-scale language model to the storage unit.

4. An embedding unit that embeds the object into text data representing the object in language at the location where the object is located in the file for which a registration request to the storage unit has been received, The file, after the text data representing the object has been embedded, is divided into predetermined units of information, and includes a division unit. The aforementioned information output unit is The information processing apparatus according to claim 3, which outputs the information divided into the predetermined units to the storage unit.

5. The prompt generation unit, The information processing apparatus according to claim 3, wherein the prompt requesting the verbalization of the image of the object includes an image of the caption of the object extracted from a file that has received a registration request to the storage unit.

6. The prompt generation unit, The information processing apparatus according to claim 3, wherein the prompt requesting the verbalization of the image of the object includes supplementary information relating to the object extracted from a file that has received a registration request to the storage unit.

7. The prompt generation unit, The information processing apparatus according to claim 3, wherein the prompt requesting the verbalization of the image of the object includes the content of the question indicated by the question data, in accordance with the content of the response data output from the large-scale language model to the question data.

8. The prompt generation unit, When a request to register a file to the storage unit is received along with information specifying how to read the objects contained in the file, The information processing apparatus according to claim 3, wherein the prompt requesting the verbalization of an image of the object includes an image of the object and information specifying a method for reading the object.

9. An information processing system including an information processing device and a terminal device that communicates with the information processing device, The aforementioned information processing device is In an information processing device that receives question data input from a user via the aforementioned terminal device, searches a file stored in the storage unit and outputs answer data, A receiving unit that receives a request to register a file to be stored in the aforementioned storage unit, A prompt generation unit generates a prompt to output content that can be read from the object contained in the file that received the registration request, A transmission unit that sends the prompt generated by the prompt generation unit and the object to a large-scale language model, An information processing system comprising: a registration unit that registers the results output by the large-scale language model in association with the file that received the registration request and stores them in the storage unit.

10. An information processing method by an information processing device that, upon receiving input of question data from a user via a terminal device, searches a file stored in a memory unit and outputs answer data, The aforementioned information processing device The storage unit receives a request to register a file to be stored in the aforementioned storage unit. A prompt is generated to output the content that can be read from the object contained in the file that received the registration request. The generated prompt and object are sent to the large-scale language model. An information processing method that registers the results output by the large-scale language model in the storage unit, associating them with the file for which the registration request was received.

11. When a user inputs question data via a terminal device, the information processing device searches a file stored in the memory unit and outputs the answer data. The storage unit receives a request to register a file to be stored in the aforementioned storage unit. A prompt is generated to output the content that can be read from the object contained in the file that received the registration request. The generated prompt and object are sent to the large-scale language model. A program that performs a process to register the results output by the large-scale language model in the storage unit, associating them with the file that received the registration request.

Images