Information processing system and program

Abstract

An information processing system and program are provided that can assist in creating prompts to be input into a generative AI system. [Solution] An information processing system 10 that is available from a plurality of user terminals 12 having a display means 13 and is equipped with a control means 20 and a storage means 21, the control means 20 being equipped with a user terminal communication unit 30 that generates display content, a prompt acquisition unit 31 that acquires an input prompt, and a prompt evaluation unit 32 that evaluates the prompt, the prompt evaluation unit 32 storing the evaluated prompt in the storage means 21 as a shared prompt if the evaluated prompt is highly rated, and the user terminal communication unit 30 is an information processing system 10 that generates display content including an input area 40 for the prompt, a display area 44 for AI output information, and a display area 45 for the shared prompt stored in the storage means 21.

Description

[Technical field]

[0001] The present invention relates to an information processing system and a program for evaluating prompts to be sent to a generative AI system. [Background technology]

[0002] In recent years, generative AI systems using large-scale language models (LLMs) have been used to provide services that generate sentences in a dialogue with users. In order to obtain the desired output from such generative AI systems, users must create appropriate prompts and input them to the generative AI system.

[0003] However, it is difficult for an inexperienced user to create an appropriate prompt. For this reason, a system that generates a prompt for a generative AI system is also known. For example, Patent Document 1 lists a system that generates a prompt to be input to a generative AI system using LLM. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7313757 Summary of the Invention [Problem to be solved by the invention]

[0005] Systems that automatically generate prompts for input to generative AI systems can output appropriate prompts for specific purposes, but cannot be used universally. In addition, because the field of generative AI has a short history, there are few reference documents and books, so it was necessary for users to gain experience using generative AI systems in order to be able to create appropriate prompts.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an information processing system and program that can assist in creating prompts to be input into a generative AI system. [Means for solving the problem]

[0007] In order to solve the above problem, the information processing system of the present invention is an information processing system that can be used from a plurality of user terminals having a display means, and is equipped with a control means and a storage means, wherein the control means is equipped with a user terminal communication unit that generates display content on the display means, a prompt acquisition unit that acquires an input prompt, a prompt evaluation unit that evaluates the prompt acquired by the prompt acquisition unit, a prompt sending unit that transmits the prompt acquired by the prompt acquisition unit to a generative AI system, and an AI output information acquisition unit that acquires AI output information from the generative AI system, wherein the prompt evaluation unit stores the evaluated prompt as a shared prompt in the storage means if the evaluated prompt is highly rated, and the user terminal communication unit generates display content including an input area for the prompt, a display area for the AI ​​output information, and a display area for the shared prompt stored in the storage means.

[0008] In addition, in the information processing system of the present invention, the prompt evaluation unit generates an evaluation prompt for evaluating the prompt acquired by the prompt acquisition unit, transmits the evaluation prompt from the prompt sending unit to the generation AI system, and acquires an evaluation of the prompt by the generation AI system by the AI ​​output information acquisition unit.

[0009] In the information processing system according to the present invention, the prompt evaluation unit evaluates the prompt acquired by the prompt acquisition unit based on an evaluation score calculated based on at least a plurality of evaluation axes.

[0010] In the information processing system according to the present invention, the prompt evaluation unit calculates a comprehensive evaluation score by applying a predetermined weighting to the evaluation score for each of the evaluation axes.

[0011] In the information processing system according to the present invention, the prompt evaluation unit is configured to, when the evaluated overall evaluation point is equal to or greater than a certain level, evaluate the prompt by inputting the evaluated overall evaluation point. user permission for sharing is requested from the user, and if the user gives permission, the prompt is stored in the storage means as a sharing prompt.

[0012] In addition, in the information processing system of the present invention, the prompt evaluation unit further evaluates the title of the prompt acquired by the prompt acquisition unit, a generative AI model suitable for processing the prompt, and a category of the prompt.

[0013] In order to solve the above problem, the program of the present invention is a program that can be used from multiple user terminals having display means, and causes a computer to execute a user terminal communication process for sending and receiving information between the user terminal, a prompt acquisition process for acquiring an input prompt, a prompt evaluation process for evaluating the prompt acquired in the prompt acquisition process, a prompt transmission process for transmitting the prompt acquired in the prompt acquisition process to a generative AI system, and an information acquisition process for acquiring AI output information from the generative AI system, wherein in the prompt evaluation process, if the evaluated prompt is highly rated, the prompt is stored in a storage means as a shared prompt, and in the user terminal communication process, display content including an input area for the prompt, a display area for the AI ​​output information, and a display area for the shared prompt stored in the storage means is displayed on the display means of the user terminal. Effect of the Invention

[0014] According to the information processing system and program of the present invention, input prompts can be evaluated and highly rated prompts can be displayed as shared prompts on the screen used by the generation AI, allowing users to refer to the highly rated prompts and facilitating the creation of prompts to input into the generation AI system. [Brief description of the drawings]

[0015] [Figure 1] 1 shows a configuration diagram of an information processing system according to an embodiment of the present invention. [Diagram 2] 3 shows a detailed configuration diagram of a control means. [Diagram 3] 1 is a flowchart in an information processing system. [Figure 4] 4 is a flowchart showing the process in S2 of FIG. 3 in detail. [Diagram 5] 13 is an example of a screen displayed on a display means of a user terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] An embodiment of the present invention will be described in detail with reference to the drawings. The information processing system 10 of this embodiment is a system that acquires a prompt from a user terminal 12, evaluates the prompt, transmits it to a generative AI system 15 using LLM, acquires AI output information from the generative AI system 15, and displays it on the user terminal 12, and is a system that can support the user in creating a prompt. LLM (Large Scale Language Model) is a language model constructed using a large amount of text data and deep learning technology. This allows the generative AI system 15 to perform natural language processing and generate natural conversations and sentences like a human. The generative AI system 15 can receive a prompt from a user and transmit AI output information corresponding to the prompt.

[0017] As shown in Fig. 1, an information processing system 10 can be connected to a plurality of user terminals 12 via a communication network 16. This allows a plurality of users to log in and use the information processing system 10. Each user terminal 12 has a display means 13 such as a monitor. The user terminal 12 is a computer such as a PC, or a mobile information terminal such as a smartphone. However, the user terminal 12 is not limited to these.

[0018] The information processing system 10 can be connected to a plurality of generative AI systems 15 via a communication network 16. Each generative AI system 15 has a different AI model and has a different specialty. The AI ​​models include, for example, an AI model that is good at translation, an AI model that is good at programming problems such as generating source code and resolving system errors, an AI model that is good at calculations and mathematical processing, an AI model that is good at document creation, generating spreadsheets and presentation materials, an AI model that is good at responding to text such as article creation, sentence generation, and sentence composition, an AI model that is good at data analysis and statistical processing, and further, there are AI models for specific purposes such as data processing of uploaded files, obtaining the latest information, market research, web browsing, image generation, and image processing, and there are also AI models that can be used for general purposes. The information processing system 10 can evaluate a prompt from a user, select a generative AI model appropriate for it, and transmit the prompt to one of the generative AI systems 15 to obtain AI output information. However, the information processing system 10 may be connected to only one generation AI system 15, or the information processing system 10 itself may include the generation AI system 15.

[0019] In Figure 1, the information processing system 10 is connected to multiple user terminals 12 and the generation AI system 15 via a single communication network 16, but the information processing system 10 may also be connected to the user terminals 12 and the generation AI system 15 via separate communication networks. For example, the information processing system 10 can be connected to the user terminals 12 via an in-house LAN and to the generation AI system 15 via the Internet.

[0020] The information processing system 10 can be configured with a computer such as a server. The information processing system 10 includes at least a control means 20, a storage means 21, and a communication means 22. The control means 20 can be configured with a processor such as a CPU. The storage means 21 can be configured with, for example, a RAM, a DRAM, a HDD, an SSD, or an external cloud or distributed storage. The communication means 22 can be configured with, for example, a NIC or a wireless LAN connection device.

[0021] As shown in Figure 2, the control means 20 includes a user terminal communication unit 30 that transmits and receives information between the user terminal 12, a prompt acquisition unit 31 that acquires a prompt inputted at the user terminal 12, a prompt evaluation unit 32 that evaluates the prompt acquired by the prompt acquisition unit 31, a prompt transmission unit 33 that transmits the prompt acquired by the prompt acquisition unit 31 to the generation AI system 15, and an AI output information acquisition unit 34 that acquires AI output information from the generation AI system 15.

[0022] Next, a flow for using the generation AI system 15 from the user terminal 12 using the information processing system 10 will be described. As shown in Fig. 3, when a prompt is input to the user terminal 12, the information processing system 10 acquires the prompt in the prompt acquisition unit 31 (S1). The information processing system 10 evaluates the prompt acquired by the prompt acquisition unit 31 in the prompt evaluation unit 32 (S2).

[0023] 4, the prompt evaluation unit 32 generates an evaluation prompt for the prompt acquired by the prompt acquisition unit 31 to be evaluated by the generation AI system 15 (S2-1). The prompt is evaluated based on multiple evaluation axes. The prompt evaluation further includes categorizing the prompt, titling the prompt, and selecting a generation AI model suitable for processing the prompt.

[0024] The evaluation prompt includes instructions to carefully read the entire prompt, understand what is being asked of it, and what kind of instructions and information it contains, and to understand as much as possible of the intent and background of the prompt and consider what kind of results are expected.

[0025] The evaluation prompt includes instructions for deciding the title based on an overall understanding of the prompt to be evaluated. Here, the main theme, keywords, and proper nouns of the prompt are extracted, and an easy-to-understand title is created based on them. Other instructions include the number of characters, clarity of the title, and how to handle cases where it is difficult to judge.

[0026] The evaluation prompt includes instructions to select a recommended generative AI model suitable for handling the prompt. The instructions include available generative AI models and their corresponding descriptions. The instructions instruct the user to select only one that best matches the response format or type of output desired by the prompt.

[0027] The evaluation prompt includes instructions for categorizing the prompt. The categories are pre-specified, such as "business efficiency," "data analysis," "marketing," "programming," and "idea creation." The instructions instruct the user to select the category that best suits the prompt to be evaluated as the main category, and the category that represents a supplementary element of the prompt to be evaluated as the subcategory.

[0028] The evaluation prompt includes an instruction to output a score within a certain range for the prompt to be evaluated, for example, a multi-level evaluation for multiple evaluation axes. Examples of evaluation axes for prompts include specificity, versatility, clarity, relevance, sentence structure, flexibility, creativity, and ethics. However, the evaluation axes may include other than these, or may not include some of these. Specificity evaluates whether the prompt is specific, contains the necessary information without excess or deficiency, is unambiguous, and clearly indicates the necessary details. Versatility evaluates whether there are many scenes in which the prompt can be used for the purpose of posting. Clarity evaluates whether the intention of the prompt is clear and does not cause misunderstandings. Relevance evaluates whether the prompt is suitable for the purpose of posting and matches the user's intended purpose or task. Sentence structure evaluates whether the writing and composition of the prompt includes ideas and techniques that can be used as a reference for prompt engineering. Flexibility evaluates whether the prompt allows for diverse interpretations and answers. Creativity evaluates whether the prompt is designed to elicit new perspectives and creative ideas. In the ethical dimension, the prompt is evaluated for its ethical appropriateness and suitability. The prompt instructions include an explanation of each of these dimensions and an explanation of how to assign a score.

[0029] Once the prompt evaluation unit 32 has generated the evaluation prompt, the prompt sending unit 33 transmits the evaluation prompt to the generation AI system 15 (S2-2). The AI ​​output information acquisition unit 34 acquires the AI ​​output information generated by the generation AI system 15 (S2-3). The prompt evaluation unit 32 calculates an overall evaluation score from the evaluation scores included in the acquired AI output information (S2-4). The overall evaluation score is calculated by applying a predetermined weighting to the evaluation score for each evaluation axis. As an example of weighting, a coefficient of 0.5 to 15.0 can be set for each evaluation axis for specificity. The overall evaluation score is calculated by multiplying the evaluation scores by the weighting coefficient and adding them up.

[0030] The user can set in advance whether to specify the AI ​​model to be used by the user himself / herself, or to have the information processing system 10 select an AI model suitable for the prompt. If the user has set the information processing system 10 to select an AI model (S2-5), the prompt evaluation unit 32 extracts information on the recommended AI model included in the evaluation of the prompt (S2-6) and proceeds to S3. If the user has set the information processing system 10 to specify the AI ​​model by himself / herself, the process proceeds directly to S3.

[0031] Once the prompt has been evaluated, the prompt sending unit 33 sends the prompt to the generation AI system 15 (S3). Here, if information about a recommended AI model has been extracted in S2-6, the prompt sending unit 33 sends a prompt to the generation AI system 15 of the recommended AI model, and if the user has selected an AI model himself, the prompt sending unit 33 sends a prompt to the generation AI system 15 of that AI model.

[0032] The AI ​​output information acquisition unit 34 acquires the AI ​​output information from the generation AI system 15 that sent the prompt (S4). The user terminal communication unit 30 displays the acquired AI output information on the display means 13 of the user terminal 12 (S5). As shown in FIG. 5, the display means 13 of the user terminal 12 displays a prompt input area 40, a chat display area 42 that displays a prompt display area 43 and an AI output information display area 44 in a chat format, and a shared prompt display area 45. The AI ​​output information is displayed in the AI ​​output information display area 44 in the chat display area 42.

[0033] If the overall evaluation score of the prompt evaluated by the prompt evaluation unit 32 is equal to or higher than a certain level (S6), the user terminal communication unit 30 causes the display means 13 of the user terminal 12 to display a sharing offer asking whether the prompt may be shared (S7). If the overall evaluation score of the prompt is lower than a certain level, this flow is terminated (S8).

[0034] If the user accepts the sharing offer (S9), the prompt evaluation unit 32 sets the target prompt as a sharing prompt and stores it in the storage means 21 together with information such as the user name and category (S10). At this time, for the prompt created by the user, specific information such as proper nouns may be replaced with abstract expressions. If the user does not accept the sharing offer in S9, this flow ends (S11).

[0035] The prompt set as the shared prompt is displayed in the shared prompt display area 45 shown in FIG. 5. The shared prompt is shared with all users who use the information processing system 10. For this reason, the shared prompt display area 45 displays prompts created by other users together with their user names and categories. The shared prompts that were highly rated in the prompt evaluation are displayed in the shared prompt display area 45, so that the user can refer to them when creating a prompt. The shared prompts displayed in the shared prompt display area 45 display recommended shared prompts set in the information processing system 10, but the user may specify conditions such as categories or particularly highly rated prompts to display the shared prompts. In addition, the shared prompts displayed in the shared prompt display area 45 may be switched to display only shared prompts that the user has liked or bookmarked.

[0036] The user can bookmark the shared prompt by clicking a bookmark icon 46 in the shared prompt display area 45. The user can also rate the shared prompt by clicking a like icon 47 in the shared prompt display area 45. If a user other than the user who created the shared prompt clicks the like icon 47, it is assumed that another person has rated the shared prompt (S12), and the prompt evaluation unit 32 re-evaluates the prompt by adding points to the overall evaluation score of the shared prompt (S13). If there is no evaluation by another person, this flow ends (S14).

[0037] Also, the program used in the information processing system 10 of this embodiment can be provided independently. In this case, the program executes a user terminal communication process for transmitting and receiving information to and from the user terminal 12, a prompt acquisition process for acquiring an input prompt, a prompt evaluation process for evaluating the prompt acquired in the prompt acquisition process, a prompt transmission process for transmitting the prompt acquired in the prompt acquisition process to the generation AI system 15, and an information acquisition process for acquiring AI output information from the generation AI system 15. In the prompt evaluation process, if the evaluated prompt is highly rated, the prompt is stored in the storage means as a shared prompt. In the user terminal communication process, display content including the prompt input area 40, the chat display area 42 including the AI ​​output information display area 44, and the shared prompt display area 45 stored in the storage means 21 is displayed on the display means 13 of the user terminal 12.

[0038] Although the embodiment of the present invention has been described above, the application of the present invention is not limited to the embodiment, and the present invention can be applied in various ways within the scope of its technical concept. [Explanation of symbols]

[0039] 10. Information Processing Systems 12 User terminals 13 Display means 15 Generative AI System 16 Communication Networks 20 Control Means 21 Memory means 22 Means of communication 30 User terminal communication unit 31 Prompt Acquisition Section 32 Prompt Evaluation Section 33 Prompt sending section 34 AI output information acquisition section 40 Prompt Input Area 42 Chat display area 43 Prompt display area 44 AI output information display area 45 Shared prompt display area 46 Bookmark Icon 47 Like Icon

Claims

1. An information processing system available from a plurality of user terminals having display means, A control means and a storage means are provided, The control means A user terminal communication unit for transmitting and receiving information between the user terminal and the user terminal; a prompt acquisition unit for acquiring an input prompt; a prompt evaluation unit that evaluates the prompt acquired by the prompt acquisition unit; a prompt sending unit that sends the prompt acquired by the prompt acquisition unit to a generation AI system; An AI output information acquisition unit that acquires AI output information from the generation AI system, the prompt evaluation unit stores the evaluated prompt as a shared prompt in the storage means if the evaluated prompt is highly rated; The user terminal communication unit is an information processing system that displays display content including an input area for the prompt, a display area for the AI ​​output information, and a display area for the shared prompt stored in the memory means on a display means of the user terminal.

2. The information processing system described in claim 1, wherein the prompt evaluation unit generates an evaluation prompt for evaluating the prompt acquired by the prompt acquisition unit, transmits the evaluation prompt from the prompt sending unit to the generation AI system, and acquires an evaluation of the prompt by the generation AI system in the AI ​​output information acquisition unit.

3. The information processing system according to claim 1 , wherein the prompt evaluation unit evaluates the prompt acquired by the prompt acquisition unit with an evaluation score calculated based on at least a plurality of evaluation axes.

4. The information processing system according to claim 3 , wherein the prompt evaluation unit calculates a comprehensive evaluation score by weighting the evaluation score for each of the evaluation axes in a predetermined manner.

5. The information processing system of claim 4, wherein the prompt evaluation unit requests permission to share the prompt from the user who entered the evaluated prompt if the evaluated overall evaluation point is above a certain level, and if the user grants permission, stores the prompt in the storage means as a shared prompt.

6. The information processing system according to any one of claims 3 to 5, wherein the prompt evaluation unit further evaluates the title of the prompt acquired by the prompt acquisition unit, a generative AI model suitable for processing the prompt, and a category of the prompt.

7. A program that can be used from a plurality of user terminals having display means, On the computer, A user terminal communication process for transmitting and receiving information to and from the user terminal; A prompt acquisition process that acquires an input prompt; a prompt evaluation process for evaluating the prompt acquired by the prompt acquisition process; A prompt transmission process for transmitting the prompt acquired in the prompt acquisition process to the generation AI system; An information acquisition process for acquiring AI output information from the generation AI system; In the prompt evaluation process, if the evaluated prompt is highly rated, the prompt is stored in a storage means as a shared prompt; In the user terminal communication processing, a program is provided for displaying on the display means of the user terminal display content including an input area for the prompt, a display area for the AI ​​output information, and a display area for the shared prompt stored in the memory means.

Images