Information processing systems, information processing methods, and programs

Abstract

This technology allows for the classification of chats according to their content, without administrators having to view sensitive information or private user information that may be included in the chat messages. [Solution] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor, wherein the processor is configured to execute a program so that the following steps are performed: an acquisition step, in which target information is acquired, the target information includes dialogue information exchanged by the user with any other party in a chat and identification information that identifies the dialogue information; a identification step, in which a classification label corresponding to the content of the dialogue information is identified based on the dialogue information and pre-set reference information; and a display control step, in which the identified classification label and the identification information are displayed in association without displaying the dialogue information.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses a dialogue-type intention information extraction program, apparatus, and method capable of extracting user intention information from a dialogue with a chat-form user.

[0003] This dialogue-type intention information extraction program is a program implemented on a computer, which causes the computer to function as text input means for taking in a free response sentence from a user in a dialogue form, intention information extraction means for extracting intention information indicating the intention of the user included in the user's response sentence using a comparison expression database, dialogue control means for controlling the progress of the dialogue with the user by referring to the classification in the comparison expression database to which the extracted intention information belongs and a dialogue scenario database, and log analysis means for analyzing and aggregating the dialogue history.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the prior art, when the chat content includes sensitive information or the user's private information, it is necessary to restrict the administrator's access. On the other hand, there is a problem that the administrator cannot analyze the chat content due to the restricted access.

[0006] In view of the above circumstances, the present invention provides a technology that allows chats to be classified according to their content without the administrator being able to view sensitive information or private user information that may be included in the chat content. [Means for solving the problem]

[0007] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor, wherein the processor is configured to execute a program so that the following steps are performed: an acquisition step, which acquires target information, the target information including dialogue information exchanged by a user with any other party in a chat and identification information that identifies the dialogue information; a identification step, which identifies a classification label corresponding to the content of the dialogue information based on the dialogue information and pre-set reference information; and a display control step, which displays the identified classification label and the identification information in association without displaying the dialogue information.

[0008] According to this disclosure, chats can be classified according to their content without administrators having to view sensitive information or private user information that may be included in the chat content. [Brief explanation of the drawing]

[0009] [Figure 1] This is a diagram showing the configuration of the information processing system 1 according to this embodiment. [Figure 2] This is a block diagram showing the hardware configuration of the information processing device 2. [Figure 3] This is a block diagram showing the hardware configuration of user terminal 3. [Figure 4] This is a functional block diagram showing the functions of the information processing system 1 according to this embodiment. [Figure 5] This flowchart shows an overview of the processes performed by Information Processing System 1. [Figure 6] This is an activity diagram showing specific examples of processes performed by Information Processing System 1. [Figure 7]This is an activity diagram showing specific examples of processes performed by Information Processing System 1. [Figure 8] This is an overview diagram showing an example of the label list settings screen 4. [Figure 9] This is an overview diagram showing an example of chat classification screen 5a, which displays visual information for obtaining target information IFo, as part of chat classification screen 5. [Figure 10] This is an overview diagram showing an example of chat classification screen 5b, which displays visual information of the state in which the target information IFo has been acquired, as part of chat classification screen 5. [Figure 11] This is a schematic diagram showing an example of chat classification screen 5c, which displays visual information showing the association between the classification label LC and the identification information IFi, as part of chat classification screen 5. [Figure 12] This is a schematic diagram showing an example of the analysis results screen 6. [Modes for carrying out the invention]

[0010] Embodiments of this disclosure will be described below with reference to the drawings. The various features shown in the embodiments below are interchangeable.

[0011] Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0012] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as reference information) is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a trained model that has been pre-trained to learn the correlation between input and output, or a generative AI such as a large-scale language model or visual language model that can output a desired result by inputting a prompt.

[0013] Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.

[0014] Furthermore, a circuit in a broad sense is a circuit realized by combining at least a suitable combination of circuits, circuits, processors, and memory. The processor may be a general-purpose processor or a dedicated circuit. In other words, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0015] 1. Hardware Configuration In this section, the hardware configuration will be described.

[0016] 1.1 Information Processing System 1 FIG. 1 is a configuration diagram showing an information processing system 1 according to the present embodiment. The information processing system 1 includes an information processing apparatus 2 and a user terminal 3, which are connected through a general-purpose or dedicated communication network 11. Here, the system exemplified by the information processing system 1 consists of one or more devices or components. Therefore, even the information processing apparatus 2 alone or the user terminal 3 alone is included in the system exemplified by the information processing system 1. Hereinafter, each component included in the information processing system 1 will be further described. <000009 (corrected to 0000094)>

[0017] 1.2 Information Processing Apparatus 2 FIG. 2 is a block diagram showing the hardware configuration of the information processing apparatus 2. The information processing apparatus 2 is preferably configured by a server. The information processing apparatus 2 has a communication unit 21, a storage unit 22, and a control unit 23, and these components are electrically connected via a communication bus 20 inside the information processing apparatus 2. Each component will be further described.

[0018] The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, Bluetooth (registered trademark) communication, etc. as needed. That is, it is more preferable to implement it as a collection of these plural communication means. That is, the information processing apparatus 2 may communicate various information from the outside via the communication unit 21 and the communication network 11.

[0019] Note: There was a potential error in the original text where the tag <000009> was used in ID=9 and ID=10, which was likely a typo. I assumed it should be in ID=10 for consistency. If this is not correct, please adjust the translation accordingly.The storage unit 22 stores various types of information as defined above. This can be done, for example, as a storage device such as a solid-state drive (SSD) that stores various programs related to the information processing device 2 executed by the control unit 23, or as a memory such as random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to program calculations. The storage unit 22 stores various programs and variables related to the information processing device 2 executed by the control unit 23.

[0020] The control unit 23 performs processing and control of the overall operation related to the information processing device 2. The control unit 23 is a general-purpose processor such as a Central Processing Unit (CPU) (not shown), or a dedicated processor specialized for a specific process. A dedicated processor is, for example, a GPU (graphics processing unit), an FPGA (field-programmable gate array), or an ASIC (application-specific integrated circuit). The GPU may have GPU memory such as VRAM (an example of a storage unit 22), and examples of GPU memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of memory known in the art. In an example where the GPU is configured as part of another processor, such as a host processor, the GPU memory can be accessed by components other than the GPU.

[0021] The control unit 23 realizes various functions related to the information processing device 2 by reading a predetermined program stored in the storage unit 22. That is, information processing by software stored in the storage unit 22 is concretely realized by the control unit 23, which is an example of hardware, and can be executed as each functional unit included in the control unit 23. These will be described in more detail in the next section. Note that the control unit 23 is not limited to being a single unit, and may be implemented with multiple control units 23 for each function, or a combination thereof. That is, the information processing system 1 includes at least one control unit 23 as a processor. The control unit 23, which is a processor, is configured to execute a program so that each of the steps described later is performed.

[0022] 1.3 User Terminal 3 Figure 3 is a block diagram showing the hardware configuration of the user terminal 3. The user terminal 3 can be operated by a user and can access the information processing device 2 via a smartphone, tablet, computer, or other telecommunication line, regardless of its form. Specifically, the user terminal 3 comprises a communication unit 31, a storage unit 32, a control unit 33, a display unit 34, and an input unit 35, and these components are electrically connected within the user terminal 3 via a communication bus 30. The explanation of the communication unit 31, storage unit 32, and control unit 33 is the same as the explanation of each part of the information processing device 2 and is therefore omitted.

[0023] The display unit 34 may be included in the casing of the user terminal 3, for example, or it may be an external component. The display unit 34 displays a graphical user interface (GUI) screen that can be operated by the user. This is preferably done by using different display devices such as a CRT display, liquid crystal display, organic EL display, and plasma display, depending on the type of user terminal 3.

[0024] The input unit 35 may be included in the casing of the user terminal 3 or it may be an external component. For example, the input unit 35 may be integrated with the display unit 34 and implemented as a touch panel. If it is a touch panel, the user can input tap operations, swipe operations, etc. Of course, a switch button, mouse, QWERTY keyboard, etc. may be used instead of a touch panel. In other words, the input unit 35 receives operation input made by the user. This input is transmitted as a command signal to the control unit 33 via the communication bus 30, and the control unit 33 can perform predetermined controls and calculations as needed.

[0025] 2. Functional Configuration This section describes the functional configuration of this embodiment. As mentioned above, information processing by software stored in the memory unit 22 is specifically realized by the control unit 23, which is an example of hardware, and each functional unit included in the control unit 23 can be executed.

[0026] Figure 4 is a functional block diagram showing the functions of the information processing system 1 according to this embodiment. Specifically, an information processing device 2, which is an example of the information processing system 1, includes an acquisition unit 231, a reception unit 232, a specification unit 233, a generation unit 234, a modification unit 235, a display control unit 236, and a calculation unit 237.

[0027] The acquisition unit 231 is configured to acquire various types of information as an acquisition step. For example, the acquisition unit 231 acquires data, information input by the user, target information IFo (including dialogue information IFd, identification information IFi, etc.) from the storage unit 22 or from the user terminal 3 or other external devices via the communication network 11. In one embodiment, the various types of information acquired by the acquisition unit 231 are described as being stored in the storage unit 22.

[0028] The reception unit 232 is configured to receive various types of information as a reception step. For example, the reception unit 232 receives data, user input information, change information IFc, etc., from the storage unit 22, or from the input unit 35 of the user terminal 3 or other external devices via the communication network 11. In one embodiment, the various types of information received by the reception unit 232 are described as being stored in the storage unit 22.

[0029] The identification unit 233 is configured to identify various types of information related to the information processing device 2 as an identification step. Details will be described later.

[0030] The generation unit 234 is configured to generate various types of information related to the information processing device 2 as a generation step. Details will be described later.

[0031] The modification unit 235 is configured to perform various changes to information related to the information processing device 2 as a modification step. Details will be described later.

[0032] The display control unit 236 is configured to perform various display processes as a display control step. For example, the display control unit 236 controls the display unit 34 of the user terminal 3 to display visually recognizable information such as images including screens, still images, or moving images, icons, and messages. The display control unit 236 may also generate only rendering information for displaying visually recognizable information on the display unit 34 of the user terminal 3.

[0033] The arithmetic unit 237 is configured to perform various information processing calculations related to the information processing device 2.

[0034] 3. Operation of Information Processing System 1 This section describes the information processing method of the aforementioned information processing system 1 with reference to the diagram. The order of processing can be rearranged as appropriate, multiple processes may be executed simultaneously, and some processes may be omitted. Furthermore, the results of the processes described below may be output in a manner recognizable by the user using the display unit 34 of the user terminal 3, etc. Note that this information processing may include arbitrary exception handling not shown. Exception handling includes interruption of the information processing and omission of individual processes. The selections or inputs made in this information processing may be based on user operation or may be performed automatically without user operation.

[0035] 3.1 Overview Figure 5 is a flowchart outlining the process performed by the information processing system 1. In this process, first, the acquisition unit 231 acquires target information IFo (step S001). In this case, the target information IFo includes dialogue information IFd exchanged by the user with a designated party via chat, and identification information IFi that identifies the dialogue information IFd. Next, the identification unit 233 identifies a classification label LC corresponding to the content of the dialogue information IFd based on the dialogue information IFd and pre-set reference information IFr (step S002). Then, the display control unit 236 displays the identified classification label LC and the identification information IFi in association without displaying the dialogue information IFd (step S003).

[0036] In summary, the information processing system 1 according to one embodiment comprises at least one processor (for example, a control unit 23). The processor is configured to execute the following steps by reading a program. The acquisition unit 231 acquires target information IFo as an acquisition step. The target information IFo includes dialogue information IFd exchanged by the user with any other party in a chat, and identification information IFi that identifies the dialogue information IFd. The identification unit 233 identifies a classification label LC according to the content of the dialogue information IFd based on the dialogue information IFd and pre-set reference information IFr as a identification step. The display control unit 236 displays the identified classification label LC and the identification information IFi in association without displaying the dialogue information IFd as a display control step. With this configuration, the dialogue information IFd can be classified according to its content without the administrator being able to view sensitive information or private user information that may be included in the content of the dialogue information IFd.

[0037] 3.2 Specific Examples Next, a specific example of the process according to one embodiment will be described with reference to the activity diagram. The specific example may be included within the scope defined in the overview described above.

[0038] Figures 6 and 7 are activity diagrams showing specific examples of processes performed by Information Processing System 1. Below, we will first outline the information processing flow for setting up Label List LL, following the activity diagram in Figure 6. As an example, we will explain the case where a user with specific privileges, such as administrator privileges, on the system of Company ABC (hereinafter also referred to as "administrative user") classifies and analyzes chats conducted by employees of Company ABC (hereinafter also referred to as "general users"). Note that the target of classification and analysis is not limited to chats; any communication tool such as email may be included.

[0039] <Information processing for configuring the label list LL> First, the administrator user accesses the chat classification and analysis application, which is executed by the information processing system 1, using a browser on the user terminal 3. In this case, the chat classification and analysis application is an application that can classify and analyze chats in the chat tool based on the administrator user's operations. The chat classification and analysis application may also be provided as a dedicated desktop or mobile application, in which case the administrator user launches the application. Alternatively, the chat classification and analysis application may be provided as a web application or in SaaS (Software as a Service) format that does not require downloading to the terminal.

[0040] When the chat classification and analysis application is accessed, the display control unit 236 controls the display unit 34 of the user terminal 3 to display the top screen (not shown) (Activity A001).

[0041] The reception unit 232 accepts user login information input (e.g., click, tap, swipe, select, etc.) through user operations via the top screen, completing the login to the chat classification and analysis application. Subsequently, when input is received to transition from the top screen to the label list setting screen 4 (see Figure 8), the display control unit 236 controls the display unit 34 of the user terminal 3 to display the label list setting screen 4 (Activity A002).

[0042] The label list settings screen 4 displays the label list LL. The label list LL includes multiple labels L as candidates for classification labels LC to classify chats. The label list LL may be created by an administrator entering multiple labels L, or it may be composed of multiple pre-configured labels L. Specifically, for example, the label list LL has labels L such as "Information Gathering / Research," "Writing Support," "Brainstorming," "Translation," "Summarization," and "Data Analysis" set as candidates for classification labels LC. Furthermore, the label list LL is configured to be modifiable based on input from the administrator via the label list settings screen 4.

[0043] The calculation unit 237 determines whether there are any changes to the label list LL based on input from the administrator via the label list setting screen 4 (activity A003). If there are changes to the label list LL, the process proceeds to activity A004. Alternatively, if there are no changes to the label list LL, the information processing for setting the label list LL ends.

[0044] When the process proceeds to activity A004, the reception unit 232 receives change information IFc (activity A004). In this case, change information IFc is information about label L for modifying label list LL. Specifically, for example, change information IFc may include information to modify or delete an existing label L, or information about a new label L to be added.

[0045] Next, the modification unit 235 modifies the label list LL by changing multiple labels L based on the received modification information IFc (activity A005). In this configuration, the label list LL can be changed to the desired labels L.

[0046] <Information processing for classifying and analyzing chats> Next, following the activity diagram in Figure 7, we will outline the information processing flow for classifying and analyzing chats. Note that we will omit explanations of the information processing similar to that used for setting up the label list LL.

[0047] When input is received to transition from the top screen (not shown) to the chat classification screen 5 (see Figures 9-11), the display control unit 236 controls the display unit 34 of the user terminal 3 to display the chat classification screen 5 (Activity A101).

[0048] The acquisition unit 231 acquires target information IFo based on user input via the chat classification screen 5 (Activity A102).

[0049] In this case, the target information IFo includes dialogue information IFd, which is a conversation between a general user and any other party via chat, and identification information IFi, which identifies the dialogue information IFd.

[0050] Any recipient can be anyone capable of communicating via chat or similar methods, such as other general users of company ABC, or the trained model M.

[0051] In this case, the pre-trained model M may be a pre-trained model that has learned the correlation between input and output in advance, or it may be a generative AI such as a large-scale language model or visual language model that can output a desired result by inputting a prompt. Furthermore, the pre-trained model M may be a machine learning model that is trained using machine learning algorithms, for example, a machine learning model built on a decision tree. Examples of machine learning models built on a decision tree include Light GBM and XGBoost, but are not limited to these. Alternatively, the prediction model may be a model generated based on machine learning algorithms such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), or other deep learning methods. The following explanation will use the case where the pre-trained model M is an arbitrary opponent as an example.

[0052] Dialogue information IFd is information exchanged between a regular user and a trained model M via chat. For example, it could be information exchanged between a regular user and the trained model M via chat to translate Japanese into English, or information exchanged between a regular user and the trained model M via chat to brainstorm.

[0053] Identification information IFi is information that can identify the dialogue information IFd, such as information about a general user or the chat title T. Information about a general user includes an email address, a fixed or randomly assigned user ID or group ID, etc. Below, we will explain using the example where a fixed user ID is set as the identification information IFi for a general user. Details about the chat title T will be explained later.

[0054] The identification unit 233 identifies a classification label LC corresponding to the content of the dialogue information IFd based on the dialogue information IFd and the pre-configured reference information IFr1 (an example of reference information IFr) (Activity A103). In this case, the classification label LC may be at least one label L identified from the label list LL. In other words, the identification unit 233 may identify at least one label L from the label list LL as the classification label LC based on the dialogue information IFd and the pre-configured reference information IFr1.

[0055] In this case, the reference information IFr1 includes the label list LL. The reference information IFr may also be rule-based information such as a database, a lookup table, a predetermined function (including a decision formula such as a regression equation constructed using statistical methods), or a trained model M. In other words, the reference information IFr1 may consist of the label list LL and the reference information IFr. To put it another way, the identification unit 233 identifies at least one label L from the label list LL as the classification label LC based on the dialogue information IFd, the label list LL, and the reference information IFr.

[0056] Specifically, for example, based on information exchanged in a chat between a general user and a trained model M to translate Japanese into English (an example of dialogue information IFd) and reference information IFr1, the label L "translate" is identified as the classification label LC from the label list LL. In this configuration, the classification label LC can be identified from a set of multiple labels L.

[0057] Furthermore, the classification label LC is preferably the label L that has the highest degree of relevance to the content of the dialogue information IFd, as identified from the label list LL. In other words, the identification unit 233 may identify the label L that has the highest degree of relevance to the content of the dialogue information IFd from the label list LL, based on the dialogue information IFd and the reference information IFr1, as the classification label LC. According to this embodiment, the classification label LC that has the highest degree of relevance to the content of the dialogue information IFd can be identified.

[0058] Next, the display control unit 236 displays the identified classification label LC and the identification information IFi in association without displaying the dialogue information IFd (Activity A104). Specifically, for example, the display control unit 236 displays "Translation" in association with the user ID of the general user, without displaying the information (an example of dialogue information IFd) that a general user has exchanged with a trained model M in chat to translate Japanese into English. With this configuration, the dialogue information IFd can be classified according to its content without the administrator being able to view sensitive information that may be included in the content of the dialogue information IFd or the general user's private information.

[0059] Furthermore, when the system receives input to transition from the chat classification screen 5 to the analysis results screen 6 (see Figure 12), the display control unit 236 controls the display unit 34 of the user terminal 3 to display the analysis results screen 6. The analysis results screen 6 displays information based on the classification label LC. In other words, the display control unit 236 displays information based on the identified classification label LC (Activity A105). The information based on the classification label LC is information that allows for the analysis of chat usage, such as aggregated information and visual information such as graphs. In this configuration, it is possible to analyze chat usage using the information based on the classification label LC.

[0060] 3.3 Details Next, the aforementioned information processing will be further explained using a separate diagram.

[0061] Figure 8 is an overview diagram showing an example of the label list setting screen 4.

[0062] As shown in Figure 8, the label list settings screen 4, which is labeled "Label List Settings," is configured to allow users to view and modify the label list LL, and has areas 401 to 403 and buttons 404 to 406. Area 401 has areas 401a to 401f to display multiple labels L, with labels La to Lf displayed in areas 401a to 401f, respectively. In other words, area 401 displays the pre-configured label list LL.

[0063] Specifically, in area 401a, the label La, "Information Gathering / Research," is drawn; in area 401b, the label Lb, "Document Creation Support," is drawn; in area 401c, the label Lc, "Brainstorming," is drawn; in area 401d, the label Ld, "Translation," is drawn; in area 401e, the label Le, "Summarization," is drawn; and in area 401f, the label Lf, "Data Analysis," is drawn.

[0064] Furthermore, areas 401a to 401f are configured so that the content of each drawn label L can be modified by input. In addition, in areas 401a to 401f, an object Ot, indicated by a trash can icon, is drawn to the right of where the label L is drawn. When object Ot is pressed, the corresponding label L is deleted. Specifically, for example, if input is made to change the label Lb drawn in area 401b from "Document Creation Support" to "Document Creation Support - Summary", and object Ot drawn to the right of label Lb is pressed, the label L "Summary" drawn in area 401e will be deleted.

[0065] Area 402 is labeled "Please enter the label you wish to add," and a button 404 labeled "Add" is drawn to the right of area 402. Area 402 and button 404 are configured so that by entering the content of the new label L to be added in area 402 and pressing button 404, the new label L can be added to the label list LL shown in area 401.

[0066] Area 403, labeled "Recommended Label," is configured to suggest a proposed label LS to be added to the label list LL. In other words, the proposed label LS is any label L other than those already present in the label list LL. Button 405 is drawn to the right of area 403 and, when pressed, adds the proposed label LS to the label list LL.

[0067] In this case, the proposed label LS is preferably identified based on previously acquired dialogue information IFd (target information IFo) and reference information IFr1. In other words, the identification unit 233 identifies the proposed label LS based on the dialogue information IFd and reference information IFr1. The identified proposed label LS is then drawn in area 403. In other words, the display control unit 236 displays the proposed label LS that should be added to the label list LL as a candidate for the classification label LC. Specifically, for example, in area 403 shown in the label list setting screen 4, a new label L called "programming" is proposed. By pressing button 405, the label L "programming" is added to the label list LL shown in area 401. With this configuration, the proposed label LS can be identified and displayed as a new label L as a candidate for the classification label LC.

[0068] Figure 9 is a schematic diagram showing an example of chat classification screen 5a, which displays visual information for acquiring target information IFo, as part of chat classification screen 5.

[0069] As shown in Figure 9, the chat classification screen 5a, which displays the name "Chat Classification," is configured to allow chats to be classified and has buttons 501 to 504 and areas 505 and 506.

[0070] Button 501, on which the text "Data Acquisition" is displayed, is configured to be pressable and, when pressed, to acquire target information IFo. In other words, when button 501 is pressed, the acquisition unit 231 acquires target information IFo based on user input via the chat classification screen 5. In this case, target information IFo includes at least dialogue information IFd and identification information IFi.

[0071] Button 502, on which the text "Execute Classification" is displayed, is configured to be pressable and, when pressed, to identify a classification label LC corresponding to the content of the dialogue information IFd. Specifically, when button 501 is pressed, the identification unit 233 identifies a classification label LC corresponding to the content of the dialogue information IFd based on the dialogue information IFd and the pre-set reference information IFr1.

[0072] Button 503, on which the text "Download" is displayed, is configured to be clickable and, when clicked, download information such as the chat classification results. Specifically, when button 503 is clicked, the calculation unit 237 executes the process of downloading information such as the chat classification results.

[0073] Button 504, on which the text "Execute Analysis" is displayed, is configured to be pressable and, when pressed, displays information such as the chat analysis results. Specifically, when button 504 is pressed, the display control unit 236 transitions to the analysis results screen 6, thereby displaying information such as the chat analysis results.

[0074] Area 505 lists the items used for classifying and analyzing chats, and Area 506 is shown below Area 505 and is configured to draw the target information IFo corresponding to the items shown in Area 505. In other words, Area 506 is configured to show the target information IFo in a tabular format for each item shown in Area 505.

[0075] Specifically, area 505 is drawn to list the following items used for classifying and analyzing chats: "No," "Title," "Number of Messages Posted," "Token Count Input / Output," "Model," "User ID," and "Classification."

[0076] "No" is an item that indicates the number sequentially assigned to each target information IFo.

[0077] "Title" is an example of identification information IFi, and is an item that shows the title T generated according to the content of each interaction information IFd.

[0078] "Number of messages posted" is an item that indicates the number of chats included in each conversation information IFd.

[0079] The "Token Count Input / Output" field indicates the number of tokens used when interacting with a pre-trained model M, which is an arbitrary chat partner, in each dialogue information IFd. It displays the number of tokens input to the pre-trained model M and the number of tokens output from the pre-trained model M separately.

[0080] The "Model" field indicates the trained model M used as the chat partner in each dialogue information IFd.

[0081] "User ID" is an example of identification information IFi, and in each conversation information IFd, it is an item that indicates the user ID of the general user who interacted in the chat.

[0082] "Classification" is an item that indicates the classification label LC identified according to the content of each dialogue information IFd.

[0083] In the chat classification screen 5a, area 506 is blank because the target information IFo has not been acquired.

[0084] Figure 10 is a schematic diagram showing an example of chat classification screen 5b, which displays visual information of the state in which target information IFo has been acquired, within chat classification screen 5. In chat classification screen 5b shown in Figure 10, target information IFo has been acquired by pressing button 501 in chat classification screen 5a, and the acquired target information IFo has been drawn in area 506. Note that the same configuration as chat classification screen 5a will not be explained.

[0085] Specifically, in area 506, five target information IFo items, IFo1 to IFo5, are shown in a table format, categorized by the items shown in area 505.

[0086] Specifically, for example, in the target information IFo1 shown in row "1" of "No," "AAA" is shown as the "Title," "3" as the "Number of Messages Posted," "1677 / 1522" as the "Token Count Input / Output," "Model" is shown as "Model X," and "A0372" as the "User ID." Note that "Classification" is not shown. Similarly, in target information IFo2 to IFo5, the content of each target information IFo is shown to correspond to the items shown in area 505.

[0087] The title "AAA" in the target information IFo1 is a title T generated based on the dialogue information IFd1, which is the dialogue information IFd included in the target information IFo1, and the pre-configured reference information IFr. In other words, the generation unit 234 generates "AAA," which is the title T corresponding to the content of the dialogue information IFd1, based on the dialogue information IFd1 and the pre-configured reference information IFr. The display control unit 236 then displays the generated title T "AAA" in association with the dialogue information IFd1. With this configuration, by using the identification information IFi as the title T, the dialogue information IFd can be identified by the title T without having to view the content of the dialogue information IFd.

[0088] As indicated by the "3" value for "Number of Messages Posted" in the target information IFo1, the dialogue information IFd1 contains multiple messages, i.e., multiple chats, as a series of conversations. In other words, the dialogue information IFd1 contains information about a series of conversations that a regular user had with "Model X" (an example of a trained model M) as an arbitrary partner.

[0089] Furthermore, as indicated by the "Model X" being shown as the "Model" in the target information IFo1, the dialogue information IFd1 is information exchanged via chat between a general user and the trained model M, which is "Model X". In this configuration, the dialogue information IFd, which is the result of a user using the trained model M, can be classified according to its content without the administrator having to view the content of the dialogue information IFd.

[0090] Figure 11 is a schematic diagram showing an example of chat classification screen 5c, which displays visual information showing the association between the classification label LC and the identification information IFi. In chat classification screen 5c shown in Figure 11, the classification label LC is shown drawn in area 506. The same configuration as chat classification screens 5a and 5b will not be explained.

[0091] Specifically, in area 506, classification labels LC1 to LC5 (examples of classification labels LC) are drawn in each field of the "Classification" item to correspond to the target information IFo1 to 5 (examples of target information IFo). In this case, the classification labels LC are identified according to the content of each dialogue information IFd when button 502 is pressed on the chat classification screen 5b, and are drawn in area 506. More specifically, the identification unit 233 identifies each classification label LC according to the content of each dialogue information IFd based on each dialogue information IFd and the pre-set reference information IFr1.

[0092] If the dialogue information IFd is a series of dialogues, the identification unit 233 identifies a classification label LC corresponding to the content of the series of dialogues based on the dialogue information IFd and the pre-set reference information IFr1. Specifically, for example, classification label LC1 identifies "translation" to correspond to target information IFo1, classification label LC2 identifies "brainstorming" to correspond to target information IFo2, classification label LC3 identifies "information gathering / research" to correspond to target information IFo3, classification label LC4 identifies "document creation support" to correspond to target information IFo4, and classification label LC5 identifies "document creation support" to correspond to target information IFo5. In this configuration, dialogue information IFd, which includes a series of dialogues, can be classified according to its content.

[0093] Furthermore, the area 506 drawn on the chat classification screen 5c displays the identified classification labels LC1 to 5 and the generated titles T in association with each other, without displaying each dialogue information IFd in the target information IFo1 to 5. In other words, the display control unit 236 displays the identified classification labels LC and the generated titles T (an example of identification information IFi) in association with each other, without displaying the dialogue information IFd. With this configuration, the dialogue information IFd can be classified according to its content by the title T and classification label LC corresponding to the content of the dialogue information IFd.

[0094] Figure 12 is a schematic diagram showing an example of the analysis results screen 6.

[0095] In the analysis results screen 6 shown in Figure 12, the analysis results of the chat based on the target information IFo are displayed when button 504 is pressed on the chat classification screen 5c. Specifically, the analysis results screen 6 has areas 601 and 602, where area 601 is labeled "Percentage of Classification Labels [Threads]" and area 602 is labeled "Percentage of Model Usage".

[0096] Area 601 has displays 601a to 601f. Display 601a shows a pie chart indicating the proportion of classification labels LC, and displays 601b to 601f show the legend for the pie chart shown in display 601a.

[0097] In this case, the pie chart showing the proportion of classification labels LC indicates the following as a result of analyzing the acquired target information IFo: The legend for display 601b, labeled "Information Gathering / Research," shows that the proportion of classification labels LC is 44%; the legend for display 601c, labeled "Document Creation Support," shows that the proportion of classification labels LC is 24%; the legend for display 601d, labeled "Translation," shows that the proportion of classification labels LC is 17%; the legend for display 601e, labeled "Summary," shows that the proportion of classification labels LC is 9%; and the legend for display 601f, labeled "Other," shows that the proportion of classification labels LC is 7%.

[0098] In this way, the analysis results screen 6 shown in Figure 12 displays the percentage of classification labels LC as information that can be used to analyze chat usage. In other words, the display control unit 236 displays information based on the identified classification labels LC. With this configuration, it is possible to analyze chat usage using information based on classification labels LC (for example, aggregated information, graphs, and other visual information).

[0099] Furthermore, area 602 has displays 602a to 602d, where display 602a shows a pie chart indicating the usage rate of the model, and displays 602b to 601d show the legend for the pie chart displayed in display 602a.

[0100] In this case, the pie chart showing the usage rate of the models indicates the following as a result of analyzing the acquired target information IFo: The legend for display 602b labeled "Model X" shows that the model usage rate is 45%, the legend for display 602c labeled "Model Y" shows that the model usage rate is 37%, and the legend for display 602d labeled "Model Z" shows that the model usage rate is 18%. In other words, the display control unit 236 displays information regarding the usage rate of the trained model M based on the target information IFo. With this configuration, it is possible to grasp the percentage of the trained model M being used by any given chat partner.

[0101] Furthermore, the information based on the identified classification label LC may include information that identifies the trained model M to be used in each chat. Specifically, for example, a specific trained model Ms may be identified based on past dialogue information IFd and reference information IFr. In this case, the specific trained model Ms is a trained model identified from multiple trained models M. In other words, the identification unit 233 may identify a specific trained model Ms based on dialogue information IFd and reference information IFr. In this case, the reference information IFr may pre-define the processing capabilities, processing quality, inference performance, inference speed, usage fees, input file formats, etc. (hereinafter also referred to as "predefined information") for each trained model M. More preferably, at least one of the dialogue information IFd and the reference information IFr may include information regarding the content of processing, speed, fees, input file formats (including file formats that were attempted to be input but failed) (hereinafter also referred to as "information corresponding to the defined information"). Specifically, for example, the identification unit 233 may, based on the dialogue information IFd and reference information IFr, which contain specified information and information corresponding to the specified information, identify at least one specific trained model Ms from among "Model X," which is suitable for processing such as document creation, summarization, and translation; "Model Y," which is suitable for processing such as image generation and graph creation; and "Model Z," which is suitable for processing that performs high-precision Japanese generation. In this embodiment, a specific trained model Ms can be identified as the optimal trained model M that a general user should use for chat.

[0102] Furthermore, it is preferable that the specific trained model Ms identified by the identification unit 233 be displayed on the analysis results screen 6. In other words, the display control unit 236 displays the specific trained model Ms as an arbitrary person with whom the general user should chat. With this configuration, the specific trained model Ms can be displayed as the optimal trained model M that the general user should use for chatting, preventing the use of a trained model M that is unnecessarily high-performance or low-performance, or selecting a trained model M that does not support the file format that the user wants to load.

[0103] [others] With respect to the information processing system 1 according to the above embodiment, the following configurations may be adopted.

[0104] In the above embodiment, the case in which the identification information IFi is a user ID fixedly assigned to a particular general user was described as an example, but it is not limited to this. For example, the identification information IFi may be information of an identifier that is randomly assigned so as not to identify a general user. With this embodiment, the dialogue information IFd can be identified by a randomly assigned identifier without identifying an individual general user. Alternatively, the identification information IFi may be a group ID that includes multiple general users. With this embodiment, even chats exchanged between multiple general users can be classified and analyzed according to the group, such as the organization or department to which those general users belong. Note that the identification information IFi may be any information that can identify the dialogue information IFd, for example, it may be only the title T, or only a randomly set user ID.

[0105] In the above embodiment, the case in which a specific trained model Ms is identified based on past dialogue information IFd and reference information IFr was described as an example, but it is not limited to this. For example, before outputting a response to a chat entered by a general user, a specific trained model Ms may be identified based on the content of the entered chat and reference information IFr2 (an example of reference information IFr). Reference information IFr2 may include information based on the identified classification label LC. In other words, the identification unit 233 may identify a specific trained model Ms based on the input information to the trained model M (an example of dialogue information IFd) and reference information IFr2. In this case, the specific trained model Ms is a trained model identified from multiple trained models M. According to this embodiment, a specific trained model Ms can be identified as the optimal trained model M that a general user should use for chat, based on the input information to the trained model M.

[0106] Furthermore, it is desirable that a specific trained model Ms be automatically selected according to the input information to the trained model M. In other words, the acquisition unit 231 acquires the input information to the trained model M. The identification unit 233 identifies a specific trained model Ms based on the acquired input information to the trained model M (an example of dialogue information IFd) and pre-set reference information IFr. The generation unit 234 generates output information based on the input information to the trained model M and the specific trained model Ms. With this configuration, the specific trained model Ms can be automatically used as the optimal trained model M for general users to use in chat.

[0107] Furthermore, an information processing method comprising each step of the information processing system 1 may be implemented. That is, this information processing method includes each step of the information processing system 1. In this embodiment, through information processing, the dialogue information IFd can be classified according to its content without the administrator viewing sensitive information or private user information that may be included in the content of the dialogue information IFd. The above embodiment may be implemented as a distributable program. That is, this program causes at least one computer to execute each step of the information processing system 1. In this embodiment, through information processing by the program, the dialogue information IFd can be classified according to its content without the administrator viewing sensitive information or private user information that may be included in the content of the dialogue information IFd.

[0108] At least one of the devices included in the information processing system 1 may be located outside of Japan. For example, the information processing device 2 may be located outside of Japan, and a user in Japan may access the information processing device 2 using their user terminal 3, or the information processing device 2 may be located in Japan, and a user outside of Japan may access the information processing device 2 using their user terminal 3.

[0109] The information processing device 2 may be on-premise or in a cloud-based configuration. In the case of a cloud-based information processing device 2, for example, the above-mentioned functions and processing may be provided in the form of SaaS or cloud computing.

[0110] In one embodiment, the acquisition unit 231, reception unit 232, identification unit 233, generation unit 234, modification unit 235, display control unit 236, and calculation unit 237 are described as functional units realized by the control unit 23 of the information processing device 2. However, at least a part of these may be implemented as functional units realized by the control unit 33 of the user terminal 3. Furthermore, the various types of information described in the above example may be stored in a distributed manner not only in the storage unit 22 of the information processing device 2 but also in other external devices. In such cases, distributed ledger management based on blockchain or the like may be implemented.

[0111] The product may be provided in any of the following embodiments.

[0112] (1) An information processing system comprising at least one processor, wherein the processor is configured to execute a program such that the following steps are performed: an acquisition step, in which target information is acquired, wherein the target information includes dialogue information exchanged by a user with any other party in a chat, and identification information that identifies the dialogue information; a identification step, in which a classification label corresponding to the content of the dialogue information is identified based on the dialogue information and pre-set reference information; and a display control step, in which the identified classification label and the identification information are displayed in association without displaying the dialogue information.

[0113] In this configuration, dialogue information can be classified according to its content without the administrator having to view sensitive information or private user information that may be included in the content of the dialogue information.

[0114] (2) In the information processing system described in (1) above, the reference information includes a label list, wherein the label list includes a plurality of labels as candidates for the classification label, and in the identification step, based on the interaction information and the reference information, at least one of the labels is identified from the label list as the classification label.

[0115] In this configuration, a classification label can be identified from a set of predefined labels.

[0116] (3) In the information processing system described in (2) above, the system, in the specific step, identifies the label that has the highest degree of relevance to the content of the dialogue information from the label list, based on the dialogue information and the reference information, as the classification label.

[0117] This approach makes it possible to identify the classification label that is most relevant to the content of the dialogue information.

[0118] (4) In an information processing system described in any one of (1) to (3) above, the identification information is information of an identifier that is randomly assigned in such a way that the user cannot be identified.

[0119] In this configuration, dialogue information can be identified by a randomly assigned identifier without identifying individual users.

[0120] (5) An information processing system according to any one of (1) to (4) above, wherein in the generation step, a title corresponding to the content of the dialogue information is generated based on the dialogue information and the reference information, and in the display control step, the identified classification label and the generated title are displayed in association without displaying the dialogue information.

[0121] In this configuration, dialogue information can be classified according to its content using titles and classification labels that correspond to the content of the dialogue information.

[0122] (6) An information processing system according to any one of (1) to (5) above, wherein the dialogue information includes information on a series of dialogues exchanged by the user with the arbitrary other party, and in the identification step, the system identifies the classification label according to the content of the series of dialogues based on the dialogue information and the reference information.

[0123] In this configuration, dialogue information, including a series of conversations, can be classified according to its content.

[0124] (7) In an information processing system described in any one of (1) to (6) above, the display control step includes displaying information based on the identified classification label.

[0125] In this configuration, information based on classification labels (e.g., aggregated information, visual information such as graphs) can be used to analyze chat usage.

[0126] (8) An information processing system described in any one of (1) to (7) above, wherein the arbitrary counterpart is a trained model, and the dialogue information is information exchanged by the user with the trained model via chat.

[0127] In this configuration, the dialogue information resulting from a user using a pre-trained model can be classified according to the content of the dialogue information without the administrator having to view the content of the dialogue information.

[0128] (9) In the information processing system described in (8) above, the system identifies a specific trained model based on the dialogue information and the reference information in the identification step, where the specific trained model is a trained model identified from a plurality of trained models, and in the display control step, displays the specific trained model as the arbitrary person with whom the user should chat.

[0129] In this configuration, a specific trained model can be identified and displayed as the optimal trained model for the user to use in chat.

[0130] (10) An information processing system according to any one of (1) to (9) above, wherein the reference information includes a label list, wherein the label list includes a plurality of labels as candidates for the classification label, in the identification step, a proposed label is identified based on the dialogue information and the reference information, wherein the proposed label is a label other than the label in the label list, and in the display control step, the proposed label is displayed as a candidate for the classification label.

[0131] In this configuration, new labels can be identified and displayed as candidates for classification labels.

[0132] (11) An information processing system as described in any one of (1) to (10) above, wherein the reference information includes a label list, where the label list includes a plurality of labels as candidates for the classification label, and further, in the reception step, change information is received, where the change information is information about the label for changing the label list, and further, in the change step, the label list is changed by changing the plurality of labels based on the change information.

[0133] In this embodiment, the label list can be changed to a desired label.

[0134] (12) An information processing method comprising each step of an information processing system described in any one of (1) to (11) above.

[0135] In this configuration, through information processing, dialogue information can be classified according to its content without the administrator having to view sensitive information or private user information that may be included in the content of the dialogue information.

[0136] (13) A program that causes at least one computer to perform each step of the information processing system described in any one of (1) to (11) above.

[0137] In this configuration, through programmatic information processing, dialogue information can be classified according to its content without the administrator having to view sensitive information or private user information that may be included in the content of the dialogue information. Of course, this is not always the case.

[0138] Finally, while various embodiments relating to this disclosure have been described, these are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0139] 1: Information Processing System 11: Communication Network 2: Information Processing Device 20: Communications bus 21: Communications Department 22: Storage section 23: Control Unit 231: Acquisition Department 232: Reception Department 233: Specific part 234 :Generation part 235: Changes 236: Display Control Unit 237: Arithmetic section 3: User terminal 30: Communications bus 31: Communications Department 32: Storage section 33: Control Unit 34:Display section 35: Input section 4: Label list settings screen 401 :Region 401a :Region 401b :Region 401c: area 401d :Region 401e :Region 401f: area 402 :Region 403 :Region 404: Button 405: Button 406: Button 5: Chat classification screen 5a: Chat classification screen 5b: Chat classification screen 5c: Chat classification screen 501: Button 502: Button 503: Button 504: Button 505 :Area 506 :Area 6:Analysis result screen 601 :Area 601a :Display 601b :Display 601c :Display 601d :Display 601e :Display 601f :Display 602 :Area 602a :Display 602b :Display 602c :Display 602d :Display IFc: Change Information IFd: Dialogue information IFd1: Dialogue Information IFi: Identification Information IFi1: Identification Information IFi2: Identification Information IFi3: Identification Information IFi4: Identification Information IFi5: Identification Information IFo: Target Information IFo1: Target Information IFo2: Target Information IFo3: Target Information IFo4: Target Information IFo5: Target Information IFr: Reference Information IFr1: Reference Information IFr2: Reference Information L: Label La: Label Lb: Label Lc: Label Ld: Label Le: Label Lf: Label LC: Classification label LC1: Classification label LC2: Classification label LC3: Classification label LC4: Classification label LC5: Classification label LL: Label List LS: Proposal Label M: Trained model Ms: Specific pre-trained model Ot: Object T: Title

Claims

1. An information processing system, The system comprises at least one processor, the processor configured to execute a program such that the following steps are performed: In the acquisition step, target information is acquired, where the target information includes dialogue information exchanged by the user with any other party via chat, and identification information that identifies the dialogue information. In a specific step, a classification label corresponding to the content of the dialogue information is identified based on the dialogue information and pre-configured reference information. In the display control step, the system displays the identified classification label and the identification information in association without displaying the dialogue information.

2. In the information processing system described in claim 1, The aforementioned reference information includes a label list, where the label list includes multiple labels as candidates for the classification label. In the specified step, the system identifies at least one of the labels from the label list as the classification label based on the dialogue information and the reference information.

3. In the information processing system described in claim 2, In the specified step, the system identifies the label that has the highest degree of relevance to the content of the dialogue information from the label list, based on the dialogue information and the reference information, as the classification label.

4. In the information processing system described in claim 1, The system is characterized by the identification information being randomly assigned identifiers that do not identify the user.

5. In the information processing system described in claim 1, Furthermore, in the generation step, a title corresponding to the content of the dialogue information is generated based on the dialogue information and the reference information. The system, in the display control step, displays the identified classification label and the generated title in association without displaying the dialogue information.

6. In the information processing system described in claim 1, The aforementioned dialogue information includes information on a series of dialogues exchanged between the user and the aforementioned arbitrary party. In the aforementioned specific step, the system identifies the classification label corresponding to the content of the series of dialogues based on the dialogue information and the reference information.

7. In the information processing system described in claim 1, The display control step involves a system that displays information based on the identified classification label.

8. In the information processing system described in claim 1, The aforementioned opponent is a trained model, The aforementioned dialogue information is information exchanged between the user and the trained model via chat, in a system.

9. In the information processing system described in claim 8, In the aforementioned specific step, a specific trained model is identified based on the dialogue information and the reference information, where the specific trained model is a trained model identified from a plurality of trained models. In the display control step, the system displays the specific trained model as the arbitrary person with whom the user should chat.

10. In the information processing system described in claim 1, The aforementioned reference information includes a label list, where the label list includes multiple labels as candidates for the classification label. In the aforementioned specific step, a proposed label is identified based on the dialogue information and the reference information, where the proposed label is a label other than the label in the label list. The system, in the display control step, displays the proposed label as a candidate for the classification label.

11. In the information processing system described in claim 1, The aforementioned reference information includes a label list, where the label list includes multiple labels as candidates for the classification label. Furthermore, in the reception step, change information is received, where the change information is information about the label for changing the label list. Furthermore, in the change step, the system modifies the label list by changing the multiple labels based on the change information.

12. Information processing method, A method comprising each step of the information processing system described in any one of claims 1 to 11.

13. It is a program, A program that causes at least one computer to perform each step of the information processing system described in any one of claims 1 to 11.

Images