Device and method

Abstract

A device according to the present invention comprises: a reception unit that receives request information related to a prompt to be input into a generative AI model; a control unit that determines processing costs related to the request information and human costs related to the request information on the basis of answer content obtained from the generative AI model in response to the prompt corresponding to the request information; and an output unit that outputs both information related to the processing costs and information related to the human costs.

Description

Apparatus and Method

[0001] The present disclosure relates to an apparatus and a method.

[0002] Patent Document 1 discloses a system that calculates a cost for each task.

[0003] Japanese Unexamined Patent Publication No. 2011-258139

[0004] In recent years, generative AI is sometimes used to process various tasks. There is a demand for knowing how effective such task processing using generative AI is compared to a case where work is performed manually.

[0005] The present disclosure has been made in view of the above circumstances, and relates to an apparatus and a method capable of clarifying the effect of task processing using generative AI.

[0006] The apparatus according to the present disclosure includes: a reception unit that receives request information related to a prompt input to a generative AI model; a control unit that determines a processing cost related to the request information and a human cost related to the request information based on response content obtained from the generative AI model by a prompt corresponding to the request information; and an output unit that outputs each of information related to the processing cost and information related to the human cost.

[0007] In such an apparatus, when a prompt corresponding to request information is input to the generative AI model, the processing cost of the generative AI model for the request information is derived, and the human cost of each task for the request information is derived. Then, information related to the processing cost and the human cost of each task is output. According to this configuration, the actual processing cost of the generative AI model and the human cost estimated by being divided for each task are derived, and information related thereto (for example, information from which a difference between these can be known) is output, so that the effect of task processing using generative AI compared to a case where work is performed manually can be appropriately output. As described above, according to the apparatus of the present disclosure, the effect of task processing using generative AI can be clarified.

[0008] According to the present disclosure, the effect of task processing using generative AI can be clarified.

[0009] Figure 1 is a diagram illustrating the overview of the contribution amount calculation system according to this embodiment. Figure 2 is a functional block diagram of the contribution amount calculation system. Figure 3 is a table showing various information related to contribution amount calculation. Figure 4 is a diagram showing prompts and output examples. Figure 5 is a flowchart of the contribution amount calculation process. Figure 6 is a diagram showing an example of the hardware configuration of RAG.

[0010] The embodiments will be described in detail below with reference to the drawings. In the description, the same elements or elements having the same function will be denoted by the same reference numeral, and redundant descriptions will be omitted.

[0011] Figure 1 is a diagram illustrating the outline of the contribution amount calculation system according to this embodiment. The contribution amount calculation system is a system that derives and outputs the effect of executing a task requested by a user using a generation AI model (LLM as an example here), specifically the cost reduction amount compared to when it is executed manually. Hereinafter, the cost reduction amount may be referred to as "business contribution amount," which indicates the business contribution of the generation AI model. As shown in Figure 1, the processes involved in calculating the business contribution amount by the contribution amount calculation system include "1. Creation of LLM cost correspondence table," "2. Creation of prompt log," and "3. Creation of human cost correspondence table." "1. Creation of LLM cost correspondence table" is a process that is performed in advance, for example, manually, and is the process of creating an LLM cost correspondence table in which the cost (e.g., dollars) per predetermined number of processing tokens (100 tokens here) is associated for each type of LLM. A processing token here refers to the smallest processing unit in LLM.

[0012] "2. Prompt Log Creation" is the process of creating a prompt log of information when a task requested by the user is executed by the LLM, and is performed by the RAG (Retrieval-Augmented Generation) and LLM included in the contribution amount calculation system. Note that RAG is not a required component and may be implemented in a configuration that does not include RAG. Specifically, in prompt log creation, an input prompt corresponding to the user's request information (for example, a prompt including a news article and instructions for summarizing it) is input to the LLM, the output from the LLM (summary result) is obtained, and a prompt log is created that associates the input prompt, the number of input tokens, the type of LLM, the output, and the number of output tokens. Based on the number of input tokens and the number of output tokens, and the information in the LLM cost correspondence table, the processing cost of the LLM related to the request information is derived.

[0013] "3. Creation of Human Resource Cost Correspondence Table" is a process that creates a human resource cost correspondence table, which is the cost of performing tasks requested by the user manually, and is performed by RAG and LLM. Note that RAG is not a mandatory component and may be implemented with a configuration that does not include RAG. Specifically, in the creation of the human resource cost correspondence table, the above-mentioned input prompt and a prompt including instructions for deriving the human resource cost for each task related to the input prompt are generated, the prompt, the human resource cost correspondence table for each task that has been prepared in advance by a human, and the business log that associates the content and time information of the tasks actually performed manually are input into LLM, the output from LLM (task classification result, human resource cost, calculation basis, confidence level) is obtained, and a human resource cost correspondence table is created that associates the task classification result, the cost per predetermined number of processing tokens (100 tokens in this case) (e.g., dollars), the confidence level, and the calculation basis (details will be described later). Furthermore, if human costs are adjusted during the processing process due to manual intervention, a human cost allocation table is created that further assigns the adjusted cost per predetermined number of processing tokens.

[0014] Once the above processing is complete, the LLM processing cost and the human cost of performing the task manually will be derived for each task requested by the user. Finally, the LLM's business contribution is calculated by subtracting the LLM processing cost from the total human cost of each task, and this business contribution is output. In this way, the contribution calculation system can appropriately estimate the LLM's business contribution.

[0015] Figure 2 is a functional block diagram of the contribution amount calculation system 1. The contribution amount calculation system 1 comprises RAG 10 (device) and LLM 30 (generating AI model). Note that LLM 30 may be provided on RAG 10 or on a separate device from RAG 10. Note that RAG is not an essential component, and the system may be implemented with a configuration that does not include RAG. LLM 30 is a model that, in response to input of a prompt containing input information, generates content according to one or a combination of the instructions, context, questions, and output format indicated by the prompt, and returns that content as response information. The prompt can also include input information, in which case LLM 30 generates response information targeting the input information. The generating AI model may be configured to include, for example, LLM 30 and a user interface (UI) for interaction with the user, and may be an interactive AI that enables text chat or voice chat with the user. Examples of such generative AI models include ChatGPT, GPT®-3.5, GPT-4V, PaLM2, and the like. In this embodiment, an example is described in which a response to the request information is provided by one LLM30, but a response to the request information may be provided by multiple generative AI models. Also, although this embodiment describes an example using LLM30, other generative AI models may be used.

[0016] RAG10 comprises, as functional components, a reception unit 11, a first derivation unit 12 (control unit), a second derivation unit 13 (control unit), a contribution amount calculation unit 14, an output unit 15, and a storage unit 16.

[0017] The reception unit 11 is configured to receive request information from the user regarding prompts to be input to the LLM 30. The request information may include a task and an instruction to calculate the contribution amount, or it may simply include only the task (for example, an instruction such as "Please summarize this news article in about 50 characters"). Thus, the calculation of the contribution amount may be performed in response to a user's request, or it may be performed simply when the user instructs the LLM to process a task. In this embodiment, the above-described request information will be explained as the "first prompt input by the user" itself. The first prompt may be input directly from the user to the LLM 30, or it may be input to the LLM 30 via the RAG 10 (reception unit 11).

[0018] The first derivation unit 12 determines the processing cost related to the request information based on the response content (first response content) obtained from the LLM 30 in response to a prompt (first prompt) corresponding to the request information. In this way, the processing cost is derived from the first response content obtained from the LLM 30 in response to the first prompt input by the user.

[0019] The first prompt defines a task based on the requested information, such as, "You are a news writer. Summarize the news article in about 50 characters. News article: ○○…"

[0020] The first derivation unit 12 creates a prompt log based on the first response content, which is the output result from the LLM 30 to which the first prompt was input. The prompt log associates the input prompt (first prompt), the number of input tokens, the type of LLM 30, the output, and the number of output tokens. The number of input tokens is the number of processing tokens for the first prompt input to the LLM 30. The type of LLM 30 is, for example, an LLM type such as Tsuzumi 7b or Azure OpenAI Service GPT 4o. The output is the result of task execution by the LLM 30 (in the above example, the summary result of the news article). The number of output tokens is the number of tokens in the output result from the LLM 30. The first derivation unit 12 stores the created prompt log in the storage unit 16.

[0021] The first derivation unit 12 may create a prompt log based on the output results from the LLM 30, as shown in Figure 3(a), which associates the input prompt (first prompt), the number of input tokens, the application task, the application business area, keywords, application examples, the type of LLM 30, the output, and the number of output tokens. The application task is information indicating the field of the task (engineering, document summarization, Japanese-English translation, etc.). The application business area is information indicating the business area related to the task (system development, project management, general affairs / human resources, etc.). Keywords are information indicating keywords related to the task.

[0022] The first derivation unit 12 derives the processing cost related to the first prompt based on the number of input tokens, the number of output tokens, and the type of LLM 30 in the prompt log, and an LLM cost correspondence table (see Figure 1) stored in the storage unit 16, which associates a predetermined cost per number of processing tokens for each type of LLM 30. Thus, the processing cost is determined based on information regarding the type of LLM 30 used to obtain the first answer content and the number of processing tokens input to the first prompt. In detail, the processing cost is information derived using an LLM cost correspondence table that associates a cost per number of processing tokens for each type of LLM 30, based on the input content of the first prompt, the number of processing tokens related to the input of the first prompt, the first answer content, and the type of LLM 30 used to obtain the first answer content. The first derivation unit 12 stores the derived processing cost related to the first prompt in the storage unit 16, for example, in association with the prompt log.

[0023] The second derivation unit 13 generates a second prompt that includes a first prompt and instructions for deriving the human cost for each task related to the first prompt. Based on the output from the LLM 30 that receives the second prompt (the second response content obtained from the LLM 30 by the second prompt), it derives the human cost for each task related to the requested information. In this way, the human cost is derived from the second response content obtained from the LLM 30 by the second prompt. The second response content may include the basis for deriving the human cost, or it may include information about the manual work time included in the second prompt. The processing of the second derivation unit 13 is divided into prompt generation processing, LLM input processing, and human cost derivation processing.

[0024] In the prompt generation process, the second derivation unit 13 generates a second prompt that includes the first prompt and instructions for deriving the human cost for each task related to the first prompt. The second derivation unit 13 may also generate a second prompt that includes instructions for presenting the basis for deriving the human cost. The second derivation unit 13 may also generate a second prompt that includes instructions for presenting the degree of confidence in the human cost. Thus, the second prompt may include the first prompt, instructions for deriving the human cost for each task, instructions for presenting the basis for the derivation, and instructions for presenting the degree of confidence. The degree of confidence is the degree of confidence in the accuracy of the human cost. For example, the degree of confidence may decrease for tasks where accurate cost calculation is difficult even manually, such as "idea generation," and increase for tasks where estimation is possible to some extent based on the amount of text and the human cost of the implementer, such as translation and summarization.

[0025] The second prompt may be defined as follows (excerpt from Figure 4(a)): "Please tell us the human cost of performing the following task. Replace the input text with approximately 100 tokens, and calculate the cost while estimating as accurately as possible the time it would take for a human to perform the task, based on the length of the input and output texts and the difficulty of the task. - Output format: {"cost": {Enter the estimated cost (in US dollars)}, "evidence": {Enter the basis for the estimate}, "task": {Classify the content of this task into one natural language processing task and describe it}, "confidence": {Enter the confidence level of the estimate on a value from 1 to 100%}} - Task: "You are a news article writer. Summarize the news article in approximately 50 characters. News article: ○○…"

[0026] In the LLM input process, the second derivation unit 13 inputs the second prompt generated in the prompt generation process into the LLM 30. The second derivation unit 13 may also input a pre-created personnel cost table for each task into the LLM 30. The personnel cost table for each task may be created manually in advance and stored in the storage unit 16. The personnel cost table for each task may, for example, associate personnel costs in predetermined processing tokens with each task field (document summarization, Japanese-English translation, engineering) as shown in Figure 3(b), or it may associate personnel costs in predetermined processing tokens with each business area related to the task (system development, project management, general affairs / human resources, etc.) as shown in Figure 3(c). Furthermore, as shown in Figures 3(b) and 3(c), if there is a history of personnel cost revisions in the personnel cost table, the revised personnel costs may also be associated.

[0027] The second derivation unit 13 may further input a work log into the LLM 30, which associates the content and time information of tasks actually performed manually. The second derivation unit 13 may further input a cost table of specialized contractors into the LLM 30. The cost table of specialized contractors may be a list of costs incurred when a certain task is outsourced to a specialized contractor, and may be obtained by searching the internet or created from past outsourcing data. The cost table of specialized contractors may be obtained by the LLM 30 during processing, or it may be stored in advance.

[0028] The second derivation unit 13 may input the second prompt to multiple different types of LLM 30.

[0029] Figure 4(b) is a table showing an example of the output result from LLM30 when the second prompt shown in Figure 4(a) is input to LLM30. As shown in Figure 4(b), the output follows the output format instructed by the second prompt in Figure 4(a). "{"cost": $25, "evidence": "Assuming the input of a news article is approximately 100 tokens, the task of summarizing it into about 50 characters while reflecting the details and important nuances of the content is of moderate difficulty. It is estimated that an experienced writer would take about 30 minutes to complete this task. Assuming the average hourly rate for a freelance writer in the United States is $50, the cost per 30 minutes would be $25.", "task": summary, "confidence": 90%}"

[0030] In the human cost derivation process, the second derivation unit 13 derives the human cost for each task related to the requested information based on the output results from the LLM 30 (see Figure 4(b) as an example) into which the second prompt and various information (human cost table, work log, specialist vendor cost table, etc.) have been input. Specifically, the second derivation unit 13 creates a human cost correspondence table (see Figure 1) based on the output results from the LLM 30 (task classification result, human cost, calculation basis, confidence level). In the human cost correspondence table, the task classification result is associated with the cost (e.g., dollars) per predetermined number of processing tokens (100 tokens in this case), the confidence level, and the calculation basis. If the human cost is modified by human intervention during the processing, a human cost correspondence table is created that further associates the modified cost per predetermined number of processing tokens. The second derivation unit 13 stores the created human cost correspondence table in the storage unit 16.

[0031] Thus, the second derivation unit 13 may derive the human cost for each task based on the output from the LLM 30, which has been further input with a pre-created human cost table for each task, a work log, a cost table for specialists, etc., in addition to the second prompt.

[0032] The second derivation unit 13 may derive the human cost for each task based on the output from multiple types of LLM 30 into which the second prompt has been input. In this way, the human cost may be derived from the second response content obtained from multiple types of LLM 30 into which the second prompt has been input. The second derivation unit 13 may derive the human cost for each task based on the average value of the output from multiple types of LLM 30, or the output from the top N LLM 30 (N is a natural number) from which the error (deviation) from the expected cost for each task is small. The "error from the expected cost" for each LLM 30 may be the average relative error (relative error = (predicted value - correct value) / correct value) calculated from a correspondence table between the cost created manually and the estimation result by the LLM 30. The human cost for each task may then be derived based on the average value of the output from the top N LLM 30.

[0033] The contribution amount calculation unit 14, by referring to the storage unit 16, calculates the business contribution amount (cost reduction amount) that indicates the business contribution of LLM 30 by subtracting the processing cost of LLM 30 from the total human cost of each task for the same request information (same first prompt), and calculating the difference. The contribution amount calculation unit 14 stores the calculated business contribution amount in the storage unit 16.

[0034] The output unit 15, by referring to the storage unit 16, identifies the processing cost derived by the first derivation unit 12 and the human cost of each task derived by the second derivation unit for the same request information (same first prompt), and outputs information related to the processing cost and human cost. For example, the output unit 15 may output the processing cost and human cost themselves for the same request information (same first prompt), or it may output a processing cost and human cost correspondence table (see Figure 1). When outputting a human cost correspondence table, the output unit 15 may also output the basis and confidence level for the derivation of the human cost. Furthermore, the output unit 15 may, by referring to the storage unit 16, identify the business contribution amount for specific request information (specific first prompt), and output the business contribution amount. In this way, the output unit 15 may output the business contribution amount (contribution level) of LLM 30 determined based on the processing cost and human cost.

[0035] Figure 5 is a flowchart showing the contribution amount calculation process performed by RAG10 of the contribution amount calculation system 1.

[0036] As shown in Figure 5, RAG10 first receives request information related to the prompt input to LLM30 (step S1). Subsequently, RAG10 derives the processing cost of LLM30 related to the request information based on the output from LLM30 to which the first prompt was input (step S2).

[0037] Next, RAG10 generates a second prompt which includes a first prompt and instructions for calculating the human cost for each task related to the first prompt. Based on the output from LLM30 to which the second prompt is input, the human cost for each task related to the requested information is derived (step S3).

[0038] Next, RAG10 subtracts the processing cost of LLM30 from the total human cost of each task to calculate the business contribution amount (cost reduction amount) that indicates the business contribution of LLM30 (step S4). Finally, RAG10 outputs information related to the processing cost and the human cost of each task, including the business contribution amount mentioned above as an example (step S5).

[0039] Next, the effects and benefits of the RAG10 according to this embodiment will be described.

[0040] The RAG10 according to this embodiment includes a receiving unit 11 that receives request information related to prompts input to the LLM30, a control unit that determines the processing cost and the human cost related to the request information based on the response content obtained from the LLM30 in response to the prompt corresponding to the request information, and an output unit that outputs information related to the processing cost and information related to the human cost, respectively.

[0041] In this RAG10, a first prompt corresponding to the request information is input to the LLM30, which derives the processing cost of the LLM30 related to the request information, as well as the human cost of each task related to the request information. Then, information related to the processing cost and the human cost of each task is output. With this configuration, the actual processing cost of the LLM30 and the estimated human cost divided by each task are derived, and information related to these (for example, information that shows the difference between them) is output, so that the effect of task processing using generation AI compared to when the work is performed manually can be appropriately output. As described above, the device according to this embodiment can clarify the effect of task processing using generation AI.

[0042] The output unit 15 may output the business contribution amount of LLM 30 determined based on processing costs and human costs. By calculating and outputting the business contribution amount, which is the difference between the human cost and processing cost of each task, the effectiveness of task processing using the generation AI can be shown more clearly.

[0043] The processing cost may be determined based on information relating to the type of LLM 30 from which response content is obtained, and the number of processing tokens input into a prompt. According to this configuration, the processing cost can be derived with high accuracy.

[0044] The prompts include a first prompt input by a user, and a second prompt including the first prompt and a human cost derivation instruction for each task relating to the first prompt, and the response content may include first response content obtained from the LLM 30 by the first prompt, and second response content obtained from the LLM 30 by the second prompt. According to this configuration, by inputting an appropriate prompt to the LLM 30, the processing cost and the human cost can be derived with high accuracy.

[0045] The processing cost may be derived from the first response content obtained from the LLM 30 by the first prompt, and the human cost may be derived from the second response content obtained from the LLM 30 by the second prompt. According to this configuration, the processing cost and the human cost can be derived with high accuracy.

[0046] The second prompt includes an instruction to present the derivation basis for the human cost, and the second response content may include the derivation basis for the human cost. According to this configuration, the derivation basis for the human cost can be shown, and the credibility of the human cost can be improved.

[0047] The processing cost may be information derived using information in which the cost per number of processing tokens is associated for each type of LLM 30, based on the input content of the first prompt, the number of processing tokens related to input of the first prompt, the first response content, and the type of LLM 30 used to obtain the first response content. According to this configuration, the processing cost can be derived with high accuracy based on the cost prepared in advance for each type of LLM 30.

[0048] The second response content may be information corresponding to information relating to manual work time included in the second prompt. According to this configuration, the derivation accuracy for the human cost can be improved.

[0049] The human cost may be derived from the second response content obtained from the plurality of types of LLM 30 to which the second prompt has been input. According to such a configuration, for example, it becomes possible to use only the output results of the top N LLMs 30 with small errors, and the derivation accuracy of the human cost can be improved.

[0050] The apparatus and method of the present disclosure have the following configurations.

[0051] [1] An apparatus, comprising: a receiving unit that receives request information related to a prompt input to a generative AI model; a control unit that determines a processing cost related to the request information and a human cost related to the request information based on response content obtained from the generative AI model by a prompt corresponding to the request information; and an output unit that outputs each of information related to the processing cost and information related to the human cost.

[0052] [2] The apparatus according to [1], wherein the output unit outputs a business contribution degree of the generative AI model determined based on the processing cost and the human cost.

[0053] [3] The apparatus according to [1] or [2], wherein the processing cost is determined based on information relating to a type of the generative AI model from which the response content is obtained and a number of processing tokens input to the prompt.

[0054] [4] The apparatus according to any one of [1] to [3], wherein the prompt includes a first prompt input by a user, and a second prompt including the first prompt and a human cost derivation instruction for each task related to the first prompt, and the response content includes first response content obtained from the generative AI model by the first prompt, and second response content obtained from the generative AI model by the second prompt.

[0055] [5] The apparatus according to [4], wherein the processing cost is derived from the first response content obtained from the generative AI model by the first prompt, and the human cost is derived from the second response content obtained from the generative AI model by the second prompt.

[0056] [6] The apparatus according to [5], wherein the second prompt includes an instruction to present the basis for deriving the human cost, and the second response content includes the basis for deriving the human cost.

[0057] [7] The apparatus according to [5] or [6], wherein the processing cost is derived using information that associates the cost per processing token for each type of generating AI model, based on the input content of the first prompt, the number of processing tokens related to the input of the first prompt, the first response content, and the type of generating AI model used to obtain the first response content.

[0058] [8] The apparatus according to any one of items [5] to [7], wherein the second response content is information corresponding to the information regarding the time required for manual work included in the second prompt.

[0059] [9] The apparatus according to any one of [5] to [8], wherein the human cost is derived from the second response content obtained from a plurality of types of generating AI models into which the second prompt is input.

[0060]

[10] A method performed by the apparatus, comprising: receiving request information relating to a prompt input to a generating AI model; determining the processing cost and the human cost relating to the request information based on the response content obtained from the generating AI model in response to the prompt corresponding to the request information; and outputting information relating to the processing cost and information relating to the human cost, respectively.

[0061] The block diagram used in the description of the above embodiment shows functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may be realized by combining the one or more devices with software.

[0062] Functions include, but are not limited to, judgment, decision, determination, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.

[0063] For example, RAG10 in one embodiment of the present disclosure may function as a computer that processes the control method of the present disclosure. Figure 6 is a diagram showing an example of the hardware configuration of RAG10 according to this embodiment. The RAG10 described above may be physically configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc. Note that RAG10 may be configured as a computer device including at least one processor such as a CPU or GPU, may be configured as a computer device including multiple processors, or may be configured as a computer device including multiple computer devices.

[0064] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of RAG10 may include one or more of the devices shown in the diagram, or it may be configured to omit some of the devices.

[0065] Each function in RAG10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which allows the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of data reading and writing in the memory 1002 and storage 1003.

[0066] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. For example, the first derivation unit 12 described above may be implemented by the processor 1001.

[0067] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, the first derivation unit 12, etc., may be implemented by a control program stored in the memory 1002 and running on the processor 1001, and other functional blocks may be implemented similarly. The above-described various processes have been explained as being executed by one processor 1001, but they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0068] The memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may also be called a register, cache, main memory, etc. The memory 1002 can store executable programs (program code), software modules, etc., for implementing a control method according to one embodiment of the present disclosure.

[0069] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.

[0070] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc. The communication device 1004 may be configured to include high-frequency switches, duplexers, filters, frequency synthesizers, etc., in order to implement at least one of frequency division duplex (FDD) and time division duplex (TDD). For example, the receiving unit 11 etc. described above may be implemented by the communication device 1004.

[0071] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0072] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.

[0073] Furthermore, RAG10 may be composed of hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, processor 1001 may be implemented using at least one of these hardware components.

[0074] Information notification is not limited to the embodiments described herein and may be carried out by other means. For example, information notification may be carried out by physical layer signaling (e.g., DCI (Downlink Control Information), UCI (Uplink Control Information)), upper layer signaling (e.g., RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, broadcast information (MIB (Master Information Block), SIB (System Information Block))), other signals, or combinations thereof. RRC signaling may also be called RRC messages, and may be, for example, RRC Connection Setup messages, RRC Connection Reconfiguration messages, etc.

[0075] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements using exemplary order and are not limited to the specific order presented.

[0076] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.

[0077] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0078] Each aspect / embodiment described in this disclosure may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0079] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Accordingly, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0080] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0081] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technologies (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0082] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0083] In addition, terms used in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings. For example, at least one of the channel and symbol may be a signal (signaling). Also, a signal may be a message. Furthermore, a component carrier (CC) may be called a carrier frequency, cell, frequency carrier, etc.

[0084] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a given value, or other corresponding information. For example, wireless resources may be indicated by an index.

[0085] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure. Various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, and therefore, the various names assigned to these various channels and information elements are not restrictive in any way.

[0086] In this disclosure, terms such as "Mobile Station (MS)," "user terminal," "User Equipment (UE)," and "terminal" may be used interchangeably.

[0087] A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other appropriate term.

[0088] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, or inquiring (e.g., searching in a table, database, or other data structure), or ascertaining. “Determining” may also include receiving (e.g., receiving information), transmitting (e.g., sending information), inputting, outputting, or accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0089] The terms “connected,” “coupled,” or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables, and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain, and optical (both visible and invisible) domain.

[0090] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0091] Any reference to elements using designations such as “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.

[0092] Where the terms “include,” “including,” and their variations are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to be exclusive OR.

[0093] In this disclosure, if articles are added by translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0094] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."

[0095] 10...RAG (device), 11...reception unit, 12...first output unit (control unit), 13...second output unit (control unit), 15...output unit, 30...LLM (generated AI model).

Claims

1. A device comprising: a receiving unit that receives request information relating to prompts input to a generating AI model; a control unit that determines the processing cost and the human cost relating to the request information based on the response content obtained from the generating AI model in response to the prompt corresponding to the request information; and an output unit that outputs information relating to the processing cost and information relating to the human cost, respectively.

2. The apparatus according to claim 1, wherein the output unit outputs the business contribution of the generated AI model determined based on the processing cost and the human cost.

3. The apparatus according to claim 1, wherein the processing cost is determined based on information regarding the type of generating AI model used to obtain the response content, and the number of processing tokens entered into the prompt.

4. The apparatus according to claim 1, wherein the prompt includes a first prompt entered by the user and a second prompt including instructions for deriving the human cost for each task relating to the first prompt and the first prompt, and the response content includes a first response content obtained from an AI model generated by the first prompt and a second response content obtained from an AI model generated by the second prompt.

5. The apparatus according to claim 4, wherein the processing cost is derived from the first response content obtained from the AI ​​model generated by the first prompt, and the human cost is derived from the second response content obtained from the AI ​​model generated by the second prompt.

6. The apparatus according to claim 5, wherein the second prompt includes an instruction to present the basis for deriving the human cost, and the second response content includes the basis for deriving the human cost.

7. The apparatus according to claim 5, wherein the processing cost is derived using information that associates the cost per processing token for each type of generating AI model, based on the input content of the first prompt, the number of processing tokens related to the input of the first prompt, the first response content, and the type of generating AI model used to obtain the first response content.

8. The apparatus according to claim 5, wherein the second response content is information corresponding to the information regarding the manual work time included in the second prompt.

9. The apparatus according to claim 5, wherein the human cost is derived from the second response content obtained from a plurality of generating AI models into which the second prompt is input.

10. A method performed by the device, comprising: receiving request information relating to a prompt input to a generating AI model; determining the processing cost and the human cost relating to the request information based on the response content obtained from the generating AI model in response to the prompt corresponding to the request information; and outputting information relating to the processing cost and information relating to the human cost, respectively.

Images