Information processing device and information processing method

Abstract

This information processing device comprises: an answer acquisition unit that, with a given large language model (LLM) composed of a neural network including at least a plurality of intermediate layers as a reference LLM, acquires a first output prompt including a first answer sentence, which is output from a functional LLM by inputting a first input prompt including a question sentence to the functional LLM, which is the reference LLM in which a functional vector having the number of dimensions matching the intermediate layers of the reference LLM is inserted into the intermediate layers; and a functional vector generation unit that adjusts the functional vector so as to minimize the error between the first answer sentence and a given reference answer sentence.

Description

Information Processing Apparatus and Information Processing Method

[0001] The present invention relates to an information processing apparatus and an information processing method.

[0002] Large Language Models (LLMs) have become widespread, and techniques for causing an LLM to answer questions while impersonating an arbitrary personality and personal image are being studied. Here, the virtual personality and personal image realized by an LLM is referred to as a persona. By causing an LLM to implement an arbitrary persona, for example, Non-Player Characters (NPCs) in a virtual space can be configured. Further, a technique is known in which an action to be performed by a character in a game, which is an NPC, is estimated by inputting a user's conversation into a machine learning model (see, for example, Patent Document 1).

[0003] Japanese Patent No. 7525207

[0004] In order to cause an LLM to implement an arbitrary persona, it has been necessary to input a command sentence (prompt) describing the function. Since prompts are given in natural language, the output of the LLM may be unstable. Further, as prompts become longer, detailed instruction content may be omitted. In addition, in order to set the characteristics of a persona in detail, it is necessary to input a long prompt, and as the prompt becomes longer, the amount of processing increases.

[0005] Accordingly, an object of the present disclosure is to cause an LLM to implement an arbitrary function such as a persona without relying on input of a command made of natural language.

[0006] To solve the above problems, an information processing device relating to one aspect of this disclosure includes an answer acquisition unit that acquires a first output prompt including a first answer sentence, which is an answer sentence output from a function LLM, by inputting a first input prompt including a question sentence into a function LLM, which is a function LLM in which a function vector having a number of dimensions consistent with the intermediate layers of the function LLM is inserted into the intermediate layers, and using a given large-scale language model (LLM) which is configured to include a neural network including at least a number of intermediate layers as a reference LLM; and a function vector generation unit that adjusts the function vector so as to minimize the error between the first answer sentence and the given reference answer sentence.

[0007] Based on the above aspects, the function vector is learned by adjusting it so that the error between the first response sentence obtained by inputting a question sentence into a function LLM that includes the function vector in its intermediate layer and a given reference response sentence is minimized. By inserting this function vector into the LLM, it becomes possible to implement arbitrary functions in the LLM without relying on natural language input.

[0008] It becomes possible to implement arbitrary functions such as personas in LLM without inputting commands in natural language.

[0009] This is a block diagram showing the functional configuration of the information processing device of this embodiment. This is a diagram showing an overview of the function vector generation process. This is a diagram showing examples of a second input prompt and a second output prompt stored in the prompt memory unit. This is a diagram schematically showing the function vector generation process. This is a diagram showing an example of the position (intermediate layer) where the function vector is inserted in the LLM. This is a diagram showing an overview of an example of the function vector generation process. This is a diagram schematically showing the limitation of the length of tokens corresponding to the answer sentence. This is a diagram schematically showing the separation of tokens corresponding to the question sentence and the adjustment of token lengths. This is a diagram schematically showing the position (stage) of the intermediate layer into which the function vector is inserted, as well as the limitation of the length of tokens in the answer sentence and the processing of tokens in the question sentence. This is a flowchart showing the processing content of the information processing method in the information processing system. This is a diagram showing the configuration of the information processing program. This is a hard block diagram of the information processing device.

[0010] Embodiments of the information processing apparatus according to the present invention will be described with reference to the drawings. Where possible, the same parts will be denoted by the same reference numerals, and redundant descriptions will be omitted.

[0011] Figure 1 is a block diagram showing the device configuration and functional configuration of an information processing system including an information processing device according to this embodiment. The information processing system 1 is a system that generates function vectors to realize a desired arbitrary function in an LLM (Large-Scale Language Model). The LLM is a model configured to include a neural network including at least a plurality of intermediate layers. In this embodiment, the LLM has a transformer structure. A functional LLM, which is an LLM having an arbitrary function, is realized by inserting a function vector having a number of dimensions consistent with the intermediate layer into a predetermined intermediate layer of the LLM. The information processing system 1 further generates a functional LLM in which the function vector is inserted into a predetermined intermediate layer.

[0012] As shown in Figure 1, the information processing system 1 may be composed of an information processing device 10. Functionally, the information processing device 10 includes a reference answer acquisition unit 11, an answer acquisition unit 12, a function vector generation unit 13, an output unit 14, and an LLM generation unit 15.

[0013] Each functional unit of the information processing device 10 is configured to be able to access storage means such as the prompt storage unit 21, the LLM storage unit 22, and the functional vector storage unit 23.

[0014] The prompt memory unit 21 is a storage means that stores input / output prompts. Specifically, the prompt memory unit 21 may store a second input prompt and a second output prompt. The second input prompt includes a command statement and a question statement for causing the reference LLM to behave as a desired persona. The reference LLM is a given large-scale language model configured to include a neural network including at least a plurality of hidden layers, and is an LLM in which the function vectors generated by the information processing device 10 of this embodiment have not been inserted. The second output prompt includes a second response statement, which is a response statement output from the reference LLM in response to the input of the second input prompt to the reference LLM. The second response statement constitutes the correct answer data in learning the function vectors.

[0015] The LLM memory unit 22 is a memory means that stores LLM. The LLM memory unit 22 may also store a reference LLM. In addition, the LLM memory unit 22 may store the functional LLM in the learning phase of the functional vector.

[0016] The function vector storage unit 23 is a storage means that stores function vectors. The function vector storage unit 23 may store function vectors in the learning phase. The function vector storage unit 23 may also store learned function vectors.

[0017] In the example shown in Figure 1, the prompt storage unit 21, the LLM storage unit 22, and the function vector storage unit 23 are configured within the information processing device 10, but they may also be configured to be provided in an external device accessible from the information processing device 10. Also, in the example shown in Figure 1, the function units 11 to 15 are configured in a single information processing device 10, but they may also be distributed across multiple devices.

[0018] Next, we will explain the overview of the function vector generation process. Figure 2 is a diagram illustrating the overview of the function vector generation process. As shown in Figure 2, the function vector FV is inserted into a predetermined intermediate layer of the LLM (reference LLM: BL) to constitute the function LLM (FL).

[0019] The reference LLM(RL) outputs a second output prompt po2 in response to the input of the second input prompt pi2. The second input prompt pi2 may be composed of natural language and includes a command statement to cause the reference LLM(RL) to express an arbitrary function and a question statement to cause the reference LLM(RL) to respond. The command statement is a prompt to cause the reference LLM(RL) to behave as a desired persona (virtual personality).

[0020] The second output prompt po2 contains a second response sentence, which is the response sentence output from the reference LLM(RL) in response to the input of the second input prompt pi2. That is, the second response sentence is the response from the reference LLM(RL) to the question sentence. The second output prompt po2 corresponds to the correct answer data (training data) in the learning of the function vector FV.

[0021] The first input prompt pi1 contains the question statement. That is, the first input prompt pi1 is the second input prompt pi2 with the instruction statement removed. In the learning phase of the function vector FV, the function LLM (FL) outputs the first output prompt po1 in response to the input of the first input prompt pi1. That is, the first output prompt po1 contains the first answer statement, which is the function LLM (FL)'s answer to the question statement. The learning of the function vector FV proceeds by updating the function vector FV using backpropagation so that the error between the first answer statement and the second answer statement is minimized.

[0022] Next, the functional units of the information processing device 10 will be described. The reference response acquisition unit 11 acquires a second output prompt po2, which includes a second response statement output from the reference LLM(RL), in response to inputting a second input prompt pi2, which includes a command statement and a question statement for causing the reference LLM(RL) to behave as a desired persona, to the reference LLM(RL).

[0023] Specifically, the standard response acquisition unit 11 may acquire a second output prompt po2 from the prompt storage unit 21. Figure 3 shows an example of a second input prompt pi2 and a second output prompt po2 stored in the prompt storage unit 21. In the example shown in Figure 3, the prompt storage unit 21 stores the second input prompt pi2 and the second output prompt po2 in association. The second input prompt pi2 includes the command statement "Behave as a person with the following persona, not as an AI. Male, in his 20s. Answer the questions concisely," and the question statement "What are your hobbies?" to cause the standard LLM (RL) to behave as a desired persona.

[0024] The second output prompt po2 may be one that has been previously acquired and stored by inputting the second input prompt pi2 into the reference LLM (RL). The reference response acquisition unit 11 acquires the second output prompt po2 which includes the second response sentence "tennis".

[0025] Furthermore, the reference response acquisition unit 11 may input a pre-stored or inputted second input prompt pi2 to the reference LLM(RL). The reference response acquisition unit 11 may then acquire a second output prompt po2 output from the reference LLM(RL) based on the input second input prompt pi2.

[0026] The answer acquisition unit 12 acquires a first output prompt po1, which contains the first answer text output from the function LLM(FL), by inputting a first input prompt pi1 containing the question text to the function LLM(FL) in the learning phase.

[0027] Specifically, the answer acquisition unit 12 inputs the first input prompt pi1, which includes the question text contained in the second input prompt pi2, to the learning phase function LLM(FL). Then, the answer acquisition unit 12 acquires the first output prompt po1, which consists of the first answer text output from the function LLM(FL).

[0028] The function vector generation unit 13 generates the function vector FV by learning to adjust (update) the function vector FV so that the error between the first answer sentence and a given reference answer sentence is minimized. That is, the function vector generation unit 13 generates the function vector FV by performing learning using the backpropagation method.

[0029] Here, with further reference to Figure 4, the process of generating the function vector FV will be explained in detail. Figure 4 is a schematic diagram showing the process of generating the function vector FV to make LLM behave as the desired persona.

[0030] The reference response acquisition unit 11 acquires the second output prompt po2 output from the reference LLM(RL) in response to the input of the second input prompt pi2. The second input prompt pi2 includes an instruction statement ps for causing the reference LLM(RL) to perform an arbitrary function and a question statement sq for causing the reference LLM(RL) to respond. The reference response acquisition unit 11 may also input the second input prompt pi2 to the reference LLM(RL) stored in the LLM storage unit 22 and acquire the second output prompt po2.

[0031] The command statement ps is a prompt to cause the standard LLM (RL) to behave as the desired persona (virtual personality), and is composed of natural language such as "Gender: Male, Age: 20s, Occupation: Company employee". The question statement sq is a natural language question to be answered by the persona expressed by the command statement ps. In Figure 4, multiple question statements sq are shown, each being a natural language question such as "What is your gender?", "What is your age?", "What are your hobbies?", and "What is your occupation?".

[0032] The second output prompt po2 includes the second response statement sa output from the reference LLM(RL) in response to the input of the second input prompt pi2. That is, the second response statement sa is the response from the reference LLM(RL) to the question statement and constitutes the reference response statement ra, which corresponds to the correct answer data (training data) in the learning of the function vector FV.

[0033] The answer acquisition unit 12 inputs the first input prompt pi1, which includes the question sentence sq contained in the second input prompt pi2, to the learning phase function LLM(FL). That is, the first input prompt pi1 is the second input prompt pi2 with the command sentence removed.

[0034] A functional LLM (FL) is an LLM in which a desired function (persona) is expressed by inserting a functional vector FV into the intermediate layer of a reference LLM (BL). The reference LLM (BL) is the same as the reference LLM (RL) and has a transformer structure. The reference LLM (BL) includes intermediate transformer layers tl1 and tl2. In the example in Figure 4, transformer layer tl1 consists of 0 to 10 layers and transformer layer tl2 consists of 11 to 31 layers, but the number of layers is not limited to this example.

[0035] The answer acquisition unit 12 acquires a first output prompt po1 consisting of a first answer sentence output from the function LLM (FL). Specifically, the answer acquisition unit 12 acquires the first output prompt po1 as a result of operations performed by the transformer layer tl1, the function vector FV inserted in the intermediate layer, and the transformer layer tl2 on the input first input prompt pi1. The answer acquisition unit 12 may also acquire the first output prompt po1 including the first answer sentence by inputting the first input prompt pi1 to the function LLM (FL) of the learning phase stored in the LLM storage unit 22.

[0036] The function vector generation unit 13 then generates the function vector FV by learning to adjust (update) the function vector FV so that the loss function LF, which represents the error between the first answer sentence and the reference answer sentence ra, is minimized.

[0037] Figure 5 schematically shows an example of the position (intermediate layer) where a functional vector is inserted in a functional LLM(FL) having a transformer structure. As shown in Figure 5, the functional LLM(FL) includes multiple transformer layers tl. Then, in response to the input of the first input prompt pi1, tokens t1 to t5 are input to the functional LLM(FL).

[0038] Token t1 is a token that indicates the beginning of a sentence. Token t2 is a token that indicates the start of user input. Token t3 is a token that indicates the user's question. Token t4 is a token that indicates the start of the assistant's output. Token t5 is a token that indicates the assistant's answer.

[0039] In the schematic example shown in Figure 5, the function vector FV is inserted into the intermediate layer, the transformer layer tlx. Regarding the token stage in which the function vector FV is inserted, it may be inserted into multiple token stages, but it is most effective to insert it into the token stage preceding the sentence (answer) that you want to generate in accordance with the desired function expressed in the function LLM (FL). Furthermore, regarding the position of the transformer layer in which the function vector FV is inserted, it may be inserted into any multiple layers, but it is most effective to insert it into the intermediate layer where the input sentence (question) is conceptualized.

[0040] Referring again to Figure 1, the output unit 14 outputs the generated function vector FV. Specifically, the output unit 14 stores the function vector FV generated by the function vector generation unit 13 in the function vector storage unit 23.

[0041] The LLM generation unit 15 generates and outputs a functional LLM(FL) in which the generated functional vector FV is inserted into the intermediate layer. This provides a functional LLM(FL) in which any desired function is realized.

[0042] Next, with reference to Figure 6, another example of the function vector (FV) generation process will be explained. In the example explained with reference to Figure 2, a function vector (FV) is generated to make the LLM express the desired function (for example, a persona, which is a virtual personality) represented by the command statement contained in the second input prompt pi2. In contrast, in the example shown in Figure 6, a function vector (FV) is generated when the sentence (answer) to be output by the LLM is predetermined. When the sentence (answer) to be output is predetermined, for example, when the answer to the question "Bread or rice?" is to be "Bread".

[0043] In the example shown in FIG. 6, the answer acquisition unit 12 acquires the first output prompt po1 including the first answer sentence output from the functional LLM (FL) by inputting the first input prompt pi1 including the question sentence. The functional LLM (FL) is a reference LLM (BL) in which a functional vector FV is inserted into an intermediate layer.

[0044] Then, the functional vector generation unit 13 adjusts (updates) the functional vector FV by the error back propagation method so that the error between the first answer sentence and the reference answer sentence included in the second output prompt po2 is minimized. In this example, the reference answer sentence included in the second output prompt po2 is a sentence that the LLM is intended to fixedly output.

[0045] The output unit 14 outputs the functional vector FV learned and generated by the functional vector generation unit 13. The functional LLM (FL) configured by inserting this functional vector FV into an intermediate layer has a function of outputting an arbitrary preset answer.

[0046] Next, with reference to FIGS. 7 to 9, the reduction of processing amount and the acceleration of processing in the processing related to generation of the functional vector FV will be described. FIG. 7 is a diagram schematically showing the concept of limiting the length of tokens corresponding to an answer sentence.

[0047] The prompts used for learning the functional vector FV (the second input prompt pi2 and the second output prompt po2) are composed of questions and answers in natural language. By performing machine learning using a plurality of large amounts of prompts, the functional vector FV for causing the LLM to accurately express a desired function can be obtained. When some of the plurality of prompts have longer answers than other prompts, the lengths of tokens corresponding to the answers of all prompts used for learning must be matched to the部分 of prompts having long answers. As a result, all prompts have long tokens corresponding to the long answers of the部分 of prompts, which leads to an increase in the processing amount for learning. To prevent this, the length of tokens corresponding to the answer sentence may be limited.

[0048] Specifically, for learning using a plurality of large number of prompts, the reference answer acquisition unit 11 acquires a plurality of second output prompts each including a second answer sentence output from a reference LLM (RL) in response to input of a plurality of second input prompts pi2. The answer acquisition unit 12 inputs a plurality of first input prompts pi1 each including a question sentence included in each of the plurality of second input prompts pi2 into a functional LLM (FL), thereby acquiring a plurality of first output prompts po1 each including one or more tokens representing a corresponding plurality of first answer sentences. Then, the answer acquisition unit 12 limits the length of tokens corresponding to the first answer sentence in each of the plurality of first output prompts po1 to a given length.

[0049] Referring to FIG. 7, among prompts pt1 to pt5 included in a prompt group PT11 for learning, the prompt pt5 has a longer answer and corresponding tokens than other prompts. In this case, in order to perform learning using the prompt group PT11, padding pd, which is a token for unifying token lengths, is added to the prompts pt1 to pt4 in order to match the token length of the prompt pt5. In order to prevent an increase in processing amount due to such an increase in token length, the answer acquisition unit 12 limits the length of tokens corresponding to the first answer sentence in each of the plurality of first output prompts po1 to a given length, and performs learning using the prompt group PT12 from which redundant portions rd have been removed. Note that the token length corresponding to the answer in the second output prompt po2 is also reduced in the same manner. This reduces the processing amount for a set of input / output prompts for learning functional vectors, and further greatly reduces the learning processing amount for a plurality of large number of input / output prompts.

[0050] Figure 8 schematically illustrates the concepts of token separation and token length adjustment corresponding to the question sentence. In order to generate a large number of function vectors (FV), the same set of prompts may be used to generate each function vector FV. On the other hand, as explained with reference to Figure 5, it is effective to insert the function vector FV immediately after the token stage corresponding to the question, i.e., immediately before the token stage that corresponds to the question, in accordance with the desired function expressed in the function LLM (FL). In this case, the calculation results of the stages prior to the insertion of the function vector FV are independent of the function vector FV.

[0051] In view of these circumstances, the answer acquisition unit 12 may apply the calculation result in the function LLM(FL) of the learning phase with respect to the token representing the question sentence included in the second input prompt pi2, in the generation of a function vector FV corresponding to one function (persona), in the generation of a plurality of function vectors FV corresponding to each of a plurality of functions (personas).

[0052] Referring to Figure 8, the answer acquisition unit 12 separates each prompt in the learning prompt group PT21 into a prompt group PTQ consisting of tokens corresponding to questions and a prompt group PTA consisting of tokens corresponding to answers. If the prompts are simply separated, the token lengths may not match between them. In such cases, the answer acquisition unit 12 adds padding pd to each prompt in the prompt group PTQ to generate a prompt group PTQ1 (in the case of right padding) or a prompt group PTQ2 (in the case of left padding) with matching token lengths. When applied to this embodiment, the prompt group PTQ2 is used for subsequent processing.

[0053] The answer acquisition unit 12 applies the calculation result based on the prompt group PTQ2 in the generation of one function vector FV to the generation of multiple other function vectors FV. This significantly reduces the amount of processing required for the question text in LLM when generating a large number of function vectors FV. Consequently, it becomes possible to reduce the overall processing load when generating a large number of function vectors FV.

[0054] Figure 9 schematically illustrates the position (stage) of the intermediate layer into which the function vector FV is inserted, as well as the concepts of limiting the length of tokens in the answer sentence and processing tokens in the question sentence. Similar to Figure 5, the transformer structure of the function LLM (FL) is shown. As shown in Figure 9, the function LLM (FL) includes a transformer layer tl, into which the function vector FV is inserted, and into the intermediate transformer layer tlx, into which the token tk corresponding to the prompt is input.

[0055] The concept of limiting the length of tokens corresponding to answer sentences, as explained with reference to Figure 7, is indicated by the symbol sl1. That is, by limiting the token "A" corresponding to an answer sentence to a predetermined length, the processing load in learning a single function vector FV using many prompts is reduced.

[0056] The concepts of separating tokens corresponding to the question and adjusting token lengths, as explained with reference to Figure 8, are shown in code sl2. That is, when generating multiple function vectors FV, ​​the calculation result for generating one function vector FV corresponding to the question token, represented by code sl2, is applied to the generation of other function vectors FV. This makes it possible to reduce the processing load when generating a large number of function vectors FV.

[0057] Furthermore, since the function vector FV is inserted into the transformer layer tlx, the calculation results in the layers prior to the transformer layer tlx, indicated by the symbol sl3, do not depend on the function vector FV. In view of these circumstances, the answer acquisition unit 12 applies the calculation results in the layers prior to the intermediate layer (symbol sl3) into which the function vector FV in the function LLM(FL) is inserted, in the generation of multiple function vectors FV corresponding to each of the multiple functions. This significantly reduces the amount of computational processing required in the layers prior to the intermediate layer into which the function vector FV is inserted when generating a large number of function vectors FV. Consequently, it becomes possible to reduce the overall processing load when generating a large number of function vectors FV.

[0058] Figure 10 is a flowchart showing the processing details of the information processing method for generating the functional vector FV in the information processing system 1 and the information processing device 10.

[0059] In step S1, the reference response acquisition unit 11 acquires a second output prompt po2 which includes a second response statement output from the reference LLM(RL) in response to inputting a second input prompt pi2 which includes a command statement and a question statement to cause the reference LLM(RL) to behave as the desired persona.

[0060] In step S2, the answer acquisition unit 12 inputs the first input prompt pi1, which includes the question text contained in the second input prompt pi2, to the learning phase function LLM(FL).

[0061] Then, in step S3, the response acquisition unit 12 acquires a first output prompt po1 consisting of a first response sentence output from the function LLM(FL).

[0062] In step S4, the function vector generation unit 13 updates the function vector FV so that the error between the first response sentence and the reference response sentence (second response sentence) is minimized.

[0063] In step S5, the function vector generation unit 13 determines whether or not to terminate the learning of the function vector FV. For example, the function vector generation unit 13 may determine whether or not to terminate the learning based on whether or not the processing in steps S1 to S4 has been completed by a predetermined number of prompts. If it is determined that the learning of the function vector FV should be terminated, the process proceeds to step S6. On the other hand, if it is not determined that the learning of the function vector FV should be terminated, the process returns to step S1.

[0064] In step S6, the output unit 14 outputs the generated function vector FV. Specifically, the output unit 14 may store the function vector FV generated by the function vector generation unit 13 in the function vector storage unit 23.

[0065] Next, with reference to Figure 11, an information processing program for causing a computer to function as the information processing device 10 of this embodiment will be described. Figure 11 is a diagram showing the configuration of the information processing program. The information processing program P1 is composed of a main module m10 that comprehensively controls information processing in the information processing device 10, a standard answer acquisition module m11, an answer acquisition module m12, a function vector generation module m13, an output module m14, and an LLM generation module m15. Each of the modules m11 to m15 realizes the respective functions for each of the functional units 11 to 15.

[0066] The information processing program P1 may be transmitted via a transmission medium such as a communication line, or it may be stored in a recording medium M1, as shown in Figure 11.

[0067] According to the information processing system 1, information processing device 10, information processing method, and information processing program P1 of this embodiment described above, the function vector FV is learned by adjusting the function vector FV so that the error between the first answer sentence obtained by inputting a question sentence into a function LLM (FL) that includes the function vector FV in the intermediate layer is minimized, and a given reference answer sentence. By inserting this function vector FV into the LLM, it becomes possible to implement any function in the LLM without relying on natural language input.

[0068] The information processing apparatus and information processing method relating to this disclosure may have the following configurations. The operation and effects of each configuration are described below.

[0069] An information processing device relating to one aspect of this disclosure includes an answer acquisition unit that acquires a first output prompt, which is an answer sentence output from a function LLM, by inputting a first input prompt, which is an answer sentence, into a function LLM, which is a function LLM in which a function vector having a number of dimensions consistent with the intermediate layers of the function LLM is inserted into the intermediate layers, with the function LLM being a reference LLM, and a first input prompt, which is an answer sentence, being output from the function LLM; and a function vector generation unit that adjusts the function vector so as to minimize the error between the first answer sentence and the given reference answer sentence.

[0070] An information processing method relating to one aspect of this disclosure is a method executed by a processor, comprising: an answer acquisition step of obtaining a first output prompt including a first answer sentence, which is an answer sentence output from a function LLM, by inputting a first input prompt including a question sentence into a function LLM, which is a function LLM in which a function vector having a number of dimensions consistent with the intermediate layers of the function LLM is inserted into the intermediate layers, and using a given large-scale language model (LLM) configured to include a neural network including at least a number of intermediate layers as a reference LLM; and a function vector generation step of adjusting the function vector so that the error between the first answer sentence and the given reference answer sentence is minimized.

[0071] Based on the above aspects, the function vector is learned by adjusting it so that the error between the first response sentence obtained by inputting a question sentence into a function LLM that includes the function vector in its intermediate layer and a given reference response sentence is minimized. By inserting this function vector into the LLM, it becomes possible to implement arbitrary functions in the LLM without relying on natural language input.

[0072] Furthermore, an information processing device relating to other aspects may further include a reference answer acquisition unit that acquires a second output prompt, which includes a second answer statement, which is an answer statement output from the reference LLM in response to inputting a second input prompt to the reference LLM, which includes a command statement and a question statement for causing the reference LLM to behave as a desired persona, and a function vector generation unit that generates a function vector corresponding to the persona using the second answer statement as the reference answer statement.

[0073] Based on the above aspects, the second response sentence obtained by inputting a second input prompt consisting of an instruction sentence and a question sentence into the reference LLM is the response sentence that should be output when a question sentence is input into a function LLM that implements an arbitrary function (persona). By learning the function vector in such a way that the error between the first response sentence and the second response sentence is minimized, it becomes possible to generate a function vector for implementing an arbitrary function in the LLM.

[0074] Furthermore, in information processing devices relating to other aspects, the reference answer acquisition unit may acquire a plurality of second output prompts, each containing a second answer sentence output from the reference LLM in response to the input of a plurality of second input prompts, and the answer acquisition unit may input a plurality of first input prompts, each containing a question sentence contained in each of the plurality of second input prompts, to the function LLM, thereby acquiring a plurality of first output prompts, each containing one or more tokens representing a corresponding plurality of first answer sentences, and limit the length of the token corresponding to the first answer sentence in each of the plurality of first output prompts to a given length.

[0075] Based on the above aspects, limiting the length of the token corresponding to the first response reduces the processing load for a single set of input / output prompts for learning function vectors in the functional LLM, and further significantly reduces the processing load for learning with multiple and large numbers of input / output prompts.

[0076] Furthermore, in information processing devices relating to other aspects, the response acquisition unit may apply the calculation result in the function LLM relating to the token representing the question sentence included in the second input prompt in the generation of multiple function vectors corresponding to each of multiple personas in the generation of multiple function vectors corresponding to each of multiple personas.

[0077] Based on the above aspects, the results of calculations related to tokens representing question sentences during the learning and generation process of a function vector corresponding to a single persona are applied to the generation of function vectors corresponding to multiple and large numbers of personas. This significantly reduces the amount of processing related to question sentences in LLM when generating a large number of function vectors. Consequently, it becomes possible to reduce the overall processing load when generating a large number of function vectors.

[0078] Furthermore, in information processing devices relating to other aspects, the response acquisition unit may apply the calculation results from a previous layer in which the function vector in the function LLM was inserted, in the generation of multiple function vectors corresponding to each of multiple personas.

[0079] Based on the above aspects, in the learning and generation process of a functional vector corresponding to a single persona, the computation results of the layers prior to the intermediate layer into which the functional vector is inserted are applied to the generation of functional vectors corresponding to multiple and large numbers of personas. This significantly reduces the computational processing load of the layers prior to the intermediate layer into which the functional vector is inserted when generating a large number of functional vectors. Consequently, it becomes possible to reduce the overall processing load in the generation of a large number of functional vectors.

[0080] Furthermore, in information processing devices relating to other aspects, the function vector generation unit may update the function vector using the backpropagation method so as to minimize the error between the first response and the reference response.

[0081] Based on the above aspects, the function vector is learned using backpropagation so that the error between the first response and the reference response is minimized. This generates a suitable function vector for realizing the desired function in the LLM.

[0082] Furthermore, the information processing device relating to other aspects may further include an LLM generation unit that generates and outputs a functional LLM in which the generated functional vector is inserted into the intermediate layer.

[0083] Based on the aspects described above, it becomes possible to obtain a functional LLM in which any desired function is implemented.

[0084] The block diagram shown in Figure 1 represents functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining software with the one or more devices described above.

[0085] Functions include, but are not limited to, judgment, decision, determination, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.

[0086] For example, the information processing device 10 in one embodiment of the present invention may function as a computer. Figure 12 shows an example of the hardware configuration of the information processing device 10 according to this embodiment. Physically, the information processing device 10 may be configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0087] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the information processing device 10 may include one or more of the devices shown in Figure 12, or it may be configured to omit some of the devices.

[0088] Each function in the information processing device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, allowing the processor 1001 to perform calculations and control communication by the communication device 1004, as well as the reading and / or writing of data in the memory 1002 and storage 1003.

[0089] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. For example, the various functional units 11 to 15 shown in Figure 1 may be implemented by the processor 1001.

[0090] Furthermore, the processor 1001 reads programs (program code), software modules, and data from the storage 1003 and / or communication device 1004 into the memory 1002, and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, each of the functional units 11 to 15 of the information processing device 10 may be stored in the memory 1002 and implemented by a control program that runs on the processor 1001. Although the above-described processes have been explained as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented on one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0091] The memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may also be called a register, cache, main memory, etc. The memory 1002 can store executable programs (program code), software modules, etc., for carrying out an information processing method according to one embodiment of the present invention.

[0092] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including memory 1002 and / or storage 1003.

[0093] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via a wired and / or wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0094] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0095] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may consist of a single bus, or different buses may be used for communication between devices.

[0096] Furthermore, the information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.

[0097] The notification of information is not limited to the embodiments described herein and may be carried out by other means. For example, the notification of information may be carried out by physical layer signaling (e.g., DCI (Downlink Control Information), UCI (Uplink Control Information)), upper layer signaling (e.g., RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, broadcast information (MIB (Master Information Block), SIB (System Information Block))), other signals, or combinations thereof. RRC signaling may also be called RRC messages, and may be, for example, RRC Connection Setup messages, RRC Connection Reconfiguration messages, etc.

[0098] Each aspect / embodiment described in this disclosure may be applied to at least one of the following systems: LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA®, GSM®, CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi®), IEEE 802.16 (WiMAX®), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, and other appropriate systems, as well as next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).

[0099] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements using exemplary order and are not limited to the specific order presented.

[0100] The specific operations described in this disclosure as being performed by a base station may, in some cases, be performed by its upper node. In a network consisting of one or more network nodes having a base station, it is clear that various operations performed for communication with a terminal can be performed by the base station and at least one other network node (for example, an MME or S-GW, but not limited to these). Although the above example illustrates the case where there is one other network node besides the base station, it may also be a combination of multiple other network nodes (for example, an MME and an S-GW).

[0101] Information can be output from a higher layer (or lower layer) to a lower layer (or higher layer). Input and output may also occur via multiple network nodes.

[0102] Input and output information may be stored in a specific location (e.g., memory) or managed in a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be sent to other devices.

[0103] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0104] Each aspect / embodiment described in this disclosure may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0105] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0106] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0107] Furthermore, software, instructions, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies such as coaxial cable, fiber optic cable, twisted pair, and digital subscriber lines (DSL) and / or wireless technologies such as infrared, radio, and microwave, these wired and / or wireless technologies are included in the definition of a transmission medium.

[0108] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0109] In addition, terms described in this disclosure and / or terms necessary for understanding this specification may be replaced with terms having the same or similar meaning.

[0110] The terms “system” and “network” as used in this disclosure are interchangeable.

[0111] Furthermore, the information, parameters, etc., described in this disclosure may be expressed as absolute values, relative values ​​from a given value, or by corresponding other information. For example, wireless resources may be indicated by an index.

[0112] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure. Various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, and therefore, the various names assigned to these various channels and information elements are not restrictive in any way.

[0113] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, or inquiring (e.g., searching in a table, database, or other data structure), or ascertaining. “Determining” may also include receiving (e.g., receiving information), transmitting (e.g., sending information), inputting, outputting, or accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0114] As used in this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based on at least."

[0115] Where the terms “first,” “second,” etc., are used in this disclosure, no reference to those elements shall generally limit the quantity or order of those elements. These terms may be used herein as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements shall not imply that only two elements may be employed therein, or that the first element must precede the second element in any way.

[0116] In the configuration of each of the above devices, "means" may be replaced with "part," "circuit," "device," etc.

[0117] To the extent that “include,” “including,” and their variations are used herein or in the claims, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used herein or in the claims is not intended to be exclusive OR.

[0118] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0119] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."

[0120] The information processing apparatus 10 and information processing method of this disclosure may have the following configurations.

[0121] [1] An information processing device comprising: an answer acquisition unit that acquires a first output prompt including a first answer sentence, which is an answer sentence output from a functional LLM, by inputting a first input prompt including a question sentence into a functional LLM, which is a given large-scale language model (LLM) configured to include a neural network including at least a number of hidden layers, and in which a functional vector having a number of dimensions consistent with the hidden layers of the reference LLM is inserted into the hidden layers; and a functional vector generation unit that adjusts the functional vector so as to minimize the error between the first answer sentence and a given reference answer sentence.

[0122] [2] The information processing apparatus according to [1], further comprising: a reference answer acquisition unit that acquires a second output prompt which includes a second answer statement which is an answer statement output from the reference LLM in response to inputting a second input prompt which includes an instruction statement and a question statement for causing the reference LLM to behave as a desired persona, wherein the function vector generation unit generates the function vector corresponding to the persona using the second answer statement as the reference answer statement.

[0123] [3] The information processing apparatus according to [2], wherein the reference answer acquisition unit acquires a plurality of second output prompts, each containing the second answer sentence output from the reference LLM in response to the input of a plurality of second input prompts; the answer acquisition unit inputs a plurality of first input prompts, each containing the question sentence contained in each of the plurality of second input prompts, to the function LLM, thereby acquiring a plurality of first output prompts, each containing one or more tokens representing a corresponding plurality of first answer sentences; and limits the length of the token corresponding to the first answer sentence in each of the plurality of first output prompts to a given length.

[0124] [4] The information processing apparatus according to [2] or [3], wherein the response acquisition unit applies the result of the calculation in the function LLM with respect to the token representing the question sentence included in the second input prompt in the generation of a plurality of function vectors corresponding to each of a plurality of personas in the generation of a plurality of function vectors corresponding to each of a plurality of personas.

[0125] [5] The information processing apparatus according to any one of [2] to [4], wherein the response acquisition unit applies the calculation result in a layer prior to the intermediate layer into which the function vector in the function LLM is inserted in the generation of a plurality of function vectors corresponding to each of a plurality of personas in the generation of a plurality of function vectors corresponding to each of a plurality of personas.

[0126] [6] The information processing apparatus according to any one of [1] to [5], wherein the function vector generation unit updates the function vector by backpropagation so that the error between the first answer sentence and the reference answer sentence is minimized.

[0127] [7] An information processing apparatus according to any one of [1] to [6], further comprising: an LLM generation unit that generates and outputs a functional LLM in which the generated functional vector is inserted into an intermediate layer.

[0128] [8] An information processing method executed by a processor, comprising: an answer acquisition step of obtaining a first output prompt including a first answer sentence, which is an answer sentence output from a function LLM, by inputting a first input prompt including a question sentence into a function LLM, which is a function LLM in which a function vector having a number of dimensions consistent with the intermediate layers of the function LLM is inserted into the intermediate layers, with a given large-scale language model (LLM) comprising a neural network including at least a number of intermediate layers as the reference LLM; and a function vector generation step of adjusting the function vector so that the error between the first answer sentence and the given reference answer sentence is minimized.

[0129] 1... Information processing system, 10... Information processing device, 11... Standard answer acquisition unit, 12... Answer acquisition unit, 13... Functional vector generation unit, 14... Output unit, 15... LLM generation unit, 21... Prompt storage unit, 22... LLM storage unit, 23... Functional vector storage unit, M1... Recording medium, m11... Standard answer acquisition module, m12... Answer acquisition module, m13... Functional vector generation module, m14... Output module, m15... LLM generation module, P1... Information processing program, FV... Functional vector, FL... Functional LLM.

Claims

1. An information processing device comprising: an answer acquisition unit that acquires a first output prompt including a first answer sentence, which is an answer sentence output from a functional LLM, by inputting a first input prompt including a question sentence into a functional LLM, which is a functional LLM in which a functional vector having a number of dimensions consistent with the intermediate layers of the functional LLM is inserted into the intermediate layers, with a given large-scale language model (LLM) comprising a neural network including at least multiple intermediate layers as the reference LLM; and a functional vector generation unit that adjusts the functional vector so as to minimize the error between the first answer sentence and the given reference answer sentence.

2. The information processing apparatus according to claim 1, further comprising: a reference answer acquisition unit that acquires a second output prompt which includes a second answer statement which is an answer statement output from the reference LLM in response to inputting a second input prompt to the reference LLM which includes a command statement and a question statement for causing the reference LLM to behave as a desired persona, wherein the function vector generation unit generates the function vector corresponding to the persona using the second answer statement as the reference answer statement.

3. The information processing apparatus according to claim 2, wherein the reference answer acquisition unit acquires a plurality of second output prompts, each containing the second answer sentence output from the reference LLM in response to the input of a plurality of second input prompts; the answer acquisition unit inputs a plurality of first input prompts, each containing the question sentence contained in each of the plurality of second input prompts, to the function LLM, thereby acquiring a plurality of first output prompts, each containing one or more tokens representing a corresponding plurality of first answer sentences; and limits the length of the token corresponding to the first answer sentence in each of the plurality of first output prompts to a given length.

4. The information processing apparatus according to claim 2, wherein the response acquisition unit applies the calculation result in the function LLM with respect to the token representing the question sentence included in the second input prompt in the generation of a plurality of function vectors corresponding to each of a plurality of personas in the generation of a plurality of function vectors corresponding to each of a plurality of personas.

5. The information processing apparatus according to claim 2, wherein the response acquisition unit applies the calculation results from a previous layer in the intermediate layer into which the function vector in the function LLM is inserted in the generation of a plurality of function vectors corresponding to each of a plurality of personas in the generation of a plurality of function vectors corresponding to each of a plurality of personas.

6. The information processing apparatus according to claim 1, wherein the function vector generation unit updates the function vector by backpropagation so that the error between the first answer sentence and the reference answer sentence is minimized.

7. An information processing apparatus according to claim 1, further comprising an LLM generation unit that generates and outputs a functional LLM in which the generated functional vector is inserted into an intermediate layer.

8. An information processing method executed by a processor, comprising: an answer acquisition step of obtaining a first output prompt including a first answer sentence, which is an answer sentence output from a function LLM, by inputting a first input prompt including a question sentence into a function LLM, which is a function LLM in which a function vector having a number of dimensions consistent with the intermediate layers of the function LLM is inserted into the intermediate layers, with a given large-scale language model (LLM) comprising a neural network including at least several intermediate layers as the reference LLM; and a function vector generation step of adjusting the function vector so that the error between the first answer sentence and the given reference answer sentence is minimized.

Images