Information processing device, information processing method, and information processing program

Abstract

To perform a learning process suitable for a given service. [Solution] The information processing device according to the present invention comprises a reception unit and a learning unit. The reception unit receives information regarding the evaluation of the output information corresponding to the predetermined input information, which is output by the first learning model, when predetermined input information is input to a first learning model that has been trained to generate an answer to a question input as output information. Based on content provision information received from content providers that provide content in a predetermined service, the learning unit trains a second learning model, which is a different learning model from the first learning model, for each content provider to learn a modification policy for the first learning model to generate desired output information when the output information output by the first learning model is modified, and the relationship with the evaluation.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] In recent years, research and development using large language models (LLMs) have been actively conducted. For example, a technique for manipulating the answer output from an LLM into a desired answer is known (see Non-Patent Document 1 below).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since the application range of the above conventional technology is limited, for example, it may not be appropriately applied in a predetermined service that provides content.

[0005] The present application has been made in view of the above, and an object thereof is to execute learning processing suitable for a predetermined service.

Means for Solving the Problems

[0006] The information processing device according to the present invention is characterized by comprising: a receiving unit that receives information regarding an evaluation of output information corresponding to the input information, which is output by a first learning model that has been trained to generate an answer to a question input as input information as output information, when predetermined input information is input to a first learning model; a modification policy for the first learning model to generate desired output information when the output information output by the first learning model is modified based on content provision information received from a content provider that provides content in a predetermined service; and a learning unit that causes a second learning model, which is a different learning model from the first learning model, to learn the relationship with the evaluation for each content provider. [Effects of the Invention]

[0007] According to one embodiment, the effect is achieved that learning processing suitable for a given service can be performed. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of a learning process performed by the information processing device according to the embodiment. [Figure 2] Figure 2 shows an example of a generation process performed by the information processing device according to the embodiment. [Figure 3] Figure 3 shows an example of a decision process performed by the information processing device according to the embodiment. [Figure 4] Figure 4 shows an example of a learning process performed by the information processing device according to the embodiment. [Figure 5] Figure 5 shows an example of the configuration of an information processing system according to the embodiment. [Figure 6] Figure 6 shows an example of a learning data storage unit according to the embodiment. [Figure 7] Figure 7 shows an example of a content provision information storage unit according to an embodiment. [Figure 8] Figure 8 shows an example of a user information storage unit according to the embodiment. [Figure 9] Figure 9 shows an example of a correction value information storage unit according to the embodiment. [Figure 10] Figure 10 shows an example of an evaluation information storage unit according to an embodiment. [Figure 11] Figure 11 is a conceptual diagram of the generation process according to the embodiment. [Figure 12] Figure 12 is a flowchart showing an example of the flow of the learning process executed by the information processing device according to the embodiment. [Figure 13] Figure 13 is a flowchart showing an example of the flow of generation processing performed by the information processing device according to the embodiment. [Figure 14] Figure 14 is a flowchart showing an example of the flow of decision processing performed by the information processing device according to the embodiment. [Figure 15] Figure 15 is a flowchart showing an example of the flow of the learning process executed by the information processing device according to the embodiment. [Figure 16] Figure 16 is a conceptual diagram of the generation process related to the modified form. [Figure 17] Figure 17 is a hardware configuration diagram showing an example of a computer that implements the functions of an information processing device. [Modes for carrying out the invention]

[0009] The following describes in detail, with reference to the drawings, the embodiments for implementing the information processing apparatus, information processing method, and information processing program according to the present application (hereinafter referred to as "embodiments"). Note that these embodiments do not limit the information processing apparatus, information processing method, and information processing program according to the present application. Furthermore, each embodiment can be appropriately combined as long as the processing content is not inconsistent. Also, the same parts are denoted by the same reference numerals in each of the following embodiments, and redundant explanations are omitted.

[0010] [1. An example of information processing performed by an information processing device] First, the premises will be explained. In the examples of FIGS. 1 to 4 described below, it is assumed that the information processing apparatus 100 provides a service (an example of a predetermined service) that provides an advertisement (an example of content) to the user U1. In this case, the user U1 receives the advertisement on a portal site that includes, for example, a search service provided by the information processing apparatus 100. Such a portal site is provided by the information processing apparatus 100.

[0011] For example, a predetermined frame having a predetermined size and arranged at a predetermined position on the portal site includes an area for receiving input information such as character information input by the user U1, and an area for displaying output information such as character information for such input information.

[0012] Also, the output information referred to here is provided by a generative AI such as a text generation AI (Artificial Intelligence) that generates text. For example, the text generation AI is an LLM trained to estimate and output the next token from the input token sequence. For example, the LLM is a Transformer-based model, an RNN (Recurrent Neural Network)-based model, or the like. For example, the LLM is a model trained to output a response sentence corresponding to the input question sentence, and is a language model that performs natural language processing such as GPT (Generative Pre-trained Transformer) and Transformer. Note that the LLM is within the information processing apparatus 100 and may be created independently by the operator who manages the information processing apparatus 100. Also, it is desirable to have an LLM that conceals information such as personal information input by learning so that the input information is not used as a new response.

[0013] For example, user U1 creates a prompt including character information serving as a question on a predetermined frame as input information. Then, the information processing apparatus 100 transmits such a prompt to the generation AI server 20 that provides output information using a generation AI. In this case, the information processing apparatus 100 provides, on a predetermined frame, output information that is an answer to the input information and includes an advertisement by cooperating with the generation AI server 20.

[0014] Note that the advertisement arranged in the output information may be arranged in any manner. For example, the advertisement may be arranged for a predetermined period during a predetermined time zone.

[0015] Based on the above premises, hereinafter, each information process executed by the information processing apparatus 100 will be described with reference to FIGS. 1 to 4.

[0016] 〔1-1. An Example of Learning Process Executed by Information Processing Apparatus〕 First, the learning process executed by the information processing apparatus 100 will be described with reference to FIG. 1. FIG. 1 is a diagram showing an example of the learning process executed by the information processing apparatus according to the embodiment. Hereinafter, first, the process in which the information processing apparatus 100 stores a learning data set will be described. Next, the process in which the information processing apparatus 100 causes the first learning model to learn the learning data set will be described.

[0017] [[ID=IS]]First, the process in which the information processing apparatus 100 stores a learning data set will be described with reference to FIG. 1. In the example of FIG. 1, the information processing apparatus 100 receives input information from the user terminal 10 used by the user U1 (step S1). For example, assume that the user U1 inputs character information as input information. In this case, the information processing apparatus 100 receives the character information as input information.

[0018] Subsequently, the information processing apparatus 100 provides the input information to the generation AI server 20 (step S2). In this case, the generation AI server 20 generates output information corresponding to the input information.

[0019] The information processing device 100 then receives output information corresponding to the input information, and information regarding feature quantities output by the intermediate layer of the LLM as an example of a predetermined layer of the LLM when the input information is received (step S3).

[0020] Next, the information processing device 100 converts the feature quantities output by the intermediate layer into sparse feature quantities (step S4). This conversion process can be achieved using a technology based on sparse autoencoders (SAE) that utilize deep learning.

[0021] In this case, the information processing device 100 stores the input information, output information, transformed sparse features, and the correct labels assigned to such sparse features in a predetermined memory unit, associating them with each other. In this way, the information processing device 100 stores the information, as a training dataset, which is the information, which associates the input information, output information, sparse features, and correct labels with each other, in a predetermined memory unit.

[0022] Here, the correct label is assigned based on the output information. The correct label may be assigned manually or automatically. The unit for assigning the correct label may be, for example, each sparse feature. Alternatively, the unit for assigning the correct label may be, for example, each node that makes up the sparse feature.

[0023] Next, using Figure 1, we will explain the process by which the information processing device 100 trains the first learning model with the training dataset. In the example in Figure 1, the information processing device 100 trains the first learning model on the relationship between the input information input to the LLM, the output information output by the LLM when the input information is input to the LLM, sparse features, and the correct labels attached to the sparse features (step S5). In other words, the information processing device 100 trains the first learning model with the training dataset stored in a predetermined memory unit. Here, the first learning model is a learning model called SAE.

[0024] Furthermore, sparse features indicate the generation strategy for LLM to generate output information corresponding to given input information. For example, sparse features represent the internal meaning of LLM as a vector. To give a more specific example, one dimension of the vector represented by the sparse features represents a sparse feature that indicates information related to the advertisement. In this case, one node constituting the sparse feature represents the advertisement, the target of the advertisement, the characteristics of the target of the advertisement, the quality of the target of the advertisement, and user feedback on the target of the advertisement. Alternatively, one node constituting the sparse feature could represent characters, symbols, words, sentences, paragraphs, etc.

[0025] As described above, the information processing device 100 can generate a learning model for generating suitable output information by having the first learning model learn the relationship between the input information input to the LLM, the output information output by the LLM when the input information is input to the LLM, sparse features, and the correct labels attached to the sparse features.

[0026] The learning process of the first learning model is not limited to the above embodiment. For example, information relating input information to output information corresponding to such input information is stored in a predetermined memory in advance. In this case, the information processing device 100 provides input information to the generation AI server 20, causing the LLM to generate output information corresponding to the input information. The information processing device 100 also converts the feature quantities output by the intermediate layer when input to the LLM into sparse feature quantities. Then, the information processing device 100 converts these sparse feature quantities into feature quantities.

[0027] Next, the information processing device 100 learns to cause the LLM to generate output information corresponding to the input information, using these feature quantities as the feature quantities output by the hidden layer. In this way, the information processing device 100 can learn to suitably sparse the feature quantities output by the hidden layer.

[0028] The information processing device 100 then identifies the correspondence between the content of the output information and each node of the sparse features. In this way, the information processing device 100 can identify which node values ​​are high or low when certain output information is output. This allows the information processing device 100 to have the LLM estimate the relationship between each node and the correct label.

[0029] [1-2. An example of generation processing performed by an information processing device] Next, the generation process performed by the information processing device 100 will be explained using Figure 2. Figure 2 is a diagram showing an example of the generation process performed by the information processing device according to the embodiment. In the following, an example will be described in which the information processing device 100 modifies sparse features based on a predetermined modification policy set in advance, and uses the modified sparse features as the features output by the hidden layer to generate output information in the LLM.

[0030] In the example in Figure 2, Company A is used as an example of a content provider that provides content in a given service. For example, suppose Company A submits advertisements for its automobile products to a business operator that manages the information processing device 100.

[0031] In the example shown in Figure 2, the information processing device 100 receives input information from the user terminal 10 (step S21). For example, suppose user U1 inputs text information indicating the content of an automobile product as input information. In this case, the information processing device 100 receives text information indicating the content of an automobile product from the user terminal 10 as input information.

[0032] Next, the information processing device 100 modifies the sparse features based on a predetermined modification policy (step S22). The sparse features referred to here are the features output by the hidden layer that have been transformed.

[0033] In the example in Figure 2, the sparse feature SFV1 consists of four nodes UN1 to UN4. For example, suppose the output information contains content promoting automobiles. In this case, node UN1 is a node that indicates a reference to company A's automobile products. Node UN2 is a node that indicates a reference to the automobile products of company A's competitors. Node UN3 is a node that indicates a pleasant feeling towards automobiles. Node UN4 is a node that indicates quality and reliability. In the example in Figure 2, node UN1 shows "0.2". Node UN2 shows "0.8". Node UN3 shows "0.6". Node UN4 shows "-0.2".

[0034] Here, the predetermined modification policy is assumed to be pre-set by company A. For example, the predetermined modification policy is to increase the proportion of references to company A's automobile products in the output information if the output information includes references to competitors' automobile products. In this case, the value of the corresponding node UN1 is increased to increase the proportion of references to company A's automobile products in the output information.

[0035] In other words, when the information processing device 100 modifies a sparse feature based on a predetermined modification policy, it modifies the value of node UN1 (step ST1). For example, the information processing device 100 modifies a sparse feature based on a correction value based on a predetermined modification policy and the sparse feature itself. As a result, the information processing device 100 modifies the sparse feature SFV1 into the sparse feature SFV2. In the sparse feature SFV2, node UN1 shows "0.6". Also, in the sparse feature SFV2, node UN2 shows "0.8". Also, in the sparse feature SFV2, node UN3 shows "0.6". Also, in the sparse feature SFV2, node UN4 shows "-0.2".

[0036] Then, the information processing device 100 uses the transformed sparse features as the output features from the intermediate layer to cause the LLM to generate output information corresponding to the input information (step S23). For example, the information processing device 100 generates output information that includes textual information containing references to competitors' automobile products and references to company A's automobile products in roughly equal proportions. The information processing device 100 also generates output information that includes advertisements for company A's automobile products.

[0037] Next, the information processing device 100 provides output information to the user terminal 10 (step S24). For example, the information processing device 100 provides the user terminal 10 with output information that includes text information containing references to competitors' automobile products and references to company A's automobile products in roughly equal proportions, as well as advertisements for company A's automobile products.

[0038] As described above, the information processing device 100 can generate suitable output information based on a predetermined modification policy by having the LLM generate output information corresponding to the input information, using the modified sparse features as the output features from the intermediate layer.

[0039] [1-3. An example of a decision process executed by an information processing device] Next, the decision process executed by the information processing device 100 will be explained using Figure 3. Figure 3 is a diagram showing an example of the decision process executed by the information processing device according to the embodiment.

[0040] In the example shown in Figure 3, the information processing device 100 estimates a change strategy based on sparse features (step S31). For example, suppose the output information output by the LLM includes references to the automotive products of competitors. In this case, the information processing device 100 estimates a change strategy to increase the proportion of references to company A's automotive products in the output information, based on the sparse features obtained by transforming the features output by the intermediate layer.

[0041] Next, the information processing device 100 determines the correction value based on the change policy and the sparse features (step S32). Note that the sparse feature SFV1 shown in Figure 3 is the same as the sparse feature SFV1 shown in Figure 2, so a detailed explanation is omitted.

[0042] In the example in Figure 3, node UN1 shows "0.2" for the sparse feature SFV1. Node UN2 shows "0.8" for the sparse feature SFV1. Node UN3 shows "0.6" for the sparse feature SFV1. Node UN4 shows "-0.2" for the sparse feature SFV1.

[0043] For example, the information processing device 100 determines a correction value to increase the value of the corresponding node UN1 in order to increase the proportion of references to the company's automobile products in the output information. For example, the information processing device 100 determines a transformation matrix as shown in the following formula (1) as the correction value based on the number of dimensions of the vector represented by the sparse feature SFV1.

[0044]

number

[0045] k is a value set based on the degree to which the sparse features are modified. Changing the value of k changes the value of the node. For example, let's assume k is 0.5.

[0046] Then, the information processing device 100 modifies the sparse feature based on the correction value (step S33). The information processing device 100 modifies the sparse feature SFV1 by formula (1) to create the sparse feature SFV2.

[0047] In this case, node UN1 shows "0.6" in the sparse feature vector SFV2. Also, node UN2 shows "0.8" in the sparse feature vector SFV2. Also, node UN3 shows "0.6" in the sparse feature vector SFV2. Also, node UN4 shows "-0.2" in the sparse feature vector SFV2.

[0048] Next, the information processing device 100 uses the transformed sparse features as the features output by the hidden layer to cause the LLM to generate output information corresponding to predetermined input information (step S34). For example, the information processing device 100 converts the transformed sparse features into features. Then, the information processing device 100 uses the transformed features as the features output by the hidden layer to cause the LLM to generate output information corresponding to the input information.

[0049] For example, the information processing device 100 generates output information that includes textual information containing references to competitors' automobile products and references to company A's automobile products in roughly equal proportions. The information processing device 100 also generates output information that includes advertisements for company A's automobile products.

[0050] The information processing device 100 then provides output information to the user terminal 10 (step S35). For example, the information processing device 100 provides the user terminal 10 with output information that includes text information containing references to competitors' automobile products and references to company A's automobile products in roughly equal proportions, as well as advertisements for company A's automobile products.

[0051] Next, the information processing device 100 sets a fee for company A based on the correction value (step S36). In this case, the information processing device 100 performs a settlement process for company A with that fee. For example, the information processing device 100 performs a settlement process for the fee for company A in cooperation with a predetermined settlement server.

[0052] To give a more specific example, the information processing device 100 performs a settlement process with a predetermined settlement server at a fee each time, or at predetermined intervals (for example, every month). As a result, the information processing device 100 performs a settlement process with company A at a fee based on the adjustment value.

[0053] As described above, the information processing device 100 can estimate a suitable modification policy for generating desired output information. This allows the information processing device 100 to generate appropriate output information based on the estimated modification policy when the output information satisfies predetermined conditions.

[0054] [1-4. An example of a learning process performed by an information processing device] Next, the learning process performed by the information processing device 100 will be described using Figure 4. Figure 4 is a diagram showing an example of the learning process performed by the information processing device according to this embodiment. Below, first, the process by which the information processing device 100 generates a second learning model will be described. Next, the process by which the information processing device 100 provides a change policy will be described.

[0055] First, using Figure 4, we will explain the process by which the information processing device 100 generates the second learning model. In the example in Figure 4, we will use company A's business P1 as an example of a content provider.

[0056] Furthermore, user U1 is provided with output information that includes textual information containing references to competitors' automotive products and references to company A's automotive products in roughly equal proportions, as well as advertisements for company A's automotive products. Additionally, the policy for modifying such output information is to increase the proportion of references to company A's automotive products in the output information if the output information includes references to competitors' automotive products.

[0057] In the example shown in Figure 4, the information processing device 100 receives information regarding the evaluation of the output information provided to user U1 from the content provider terminal 30 used by business operator P1 (step S41). For example, suppose the evaluation is on a 5-point scale. Also, suppose the evaluation of such output information is 5. In this case, the information processing device 100 receives information from the content provider terminal 30 indicating that the evaluation of such output information is 5.

[0058] Next, the information processing device 100 trains the second learning model on the relationship between the change policy when output information is generated and the evaluation (step S42). For example, the information processing device 100 trains the second learning model on the relationship between the change policy of increasing the proportion of references to company A's automobile products in the output information when the output information includes references to competitors' automobile products, and the evaluation being 5. In this way, the information processing device 100 generates a second learning model that outputs a change policy corresponding to a predetermined evaluation when information about a predetermined evaluation is input to the second learning model.

[0059] Next, using Figure 4, the process by which the information processing device 100 provides the change policy will be explained. In the example in Figure 4, the information processing device 100 receives information regarding the request for provision of the change policy from the content provider terminal 30 (step S43).

[0060] Next, the information processing device 100 inputs information about a predetermined evaluation to the second learning model to estimate a change policy corresponding to that predetermined evaluation (step S44). For example, the information processing device 100 estimates a change policy corresponding to evaluation 5 as information about a predetermined evaluation. To give a more specific example, the information processing device 100 estimates a change policy corresponding to evaluation 5 in which the output information includes only references to company A's automobile products.

[0061] The information processing device 100 then provides information regarding the change policy (step S45). For example, the information processing device 100 provides information regarding the change policy corresponding to evaluation 5. To give a more specific example, the information processing device 100 provides information regarding a change policy in which the output information includes only references to company A's automobile products, as the change policy corresponding to evaluation 5.

[0062] As described above, the information processing device 100 can perform learning processes suitable for a given service. For example, the information processing device 100 trains a second learning model on the relationship between the change policy and the evaluation received by the content provider. Subsequently, the information processing device 100 inputs information about the given evaluation into the second learning model to estimate the change policy corresponding to the given evaluation. Then, the information processing device 100 provides the estimated change policy to the content provider. In this way, the information processing device 100 can provide the content provider with information about a suitable change policy.

[0063] In the above embodiment, an example was given where the evaluation was on a 5-point scale, but the invention is not limited to this. For example, the evaluation may be on an N-point scale (where N is any number).

[0064] [2. Configuration of the Information Processing System] Next, the configuration of the information processing system 1 according to the embodiment will be described using Figure 5. Figure 5 is a diagram showing an example of the configuration of the information processing system 1 according to the embodiment. As shown in Figure 5, the information processing system 1 includes a user terminal 10, a generation AI server 20, a content provider terminal 30, a payment server 40, and an information processing device 100. The user terminal 10, the generation AI server 20, the content provider terminal 30, the payment server 40, and the information processing device 100 are connected via network N so as to be able to communicate by wired or wireless means.

[0065] The information processing system 1 shown in Figure 5 may include multiple user terminals 10, multiple generation AI servers 20, multiple content provider terminals 30, multiple payment servers 40, and multiple information processing devices 100.

[0066] User terminal 10 is an information processing device used by users to access content such as web pages and application content displayed in a browser. For example, user terminal 10 may be a desktop PC (Personal Computer), a notebook PC, a tablet device, a mobile phone, or a PDA (Personal Digital Assistant).

[0067] The generating AI server 20 is an information processing device that receives input information and provides response information corresponding to the input information as output information, and is implemented, for example, by a server device or a cloud system.

[0068] For example, a generative AI is a text generation AI that generates text. A text generation AI is, for example, a large-scale language model that has been trained to estimate and output the next token from an input sequence of tokens. For example, large-scale language models include transformer-based models and RNN-based models.

[0069] Transformer-based models include, but are not limited to, GPT (Generative Pre-trained Transformer) and BERT (Bidirectional Encoder Representations from Transformers). RNN-based models include, but are not limited to, RWKV (Receptance Weighted Key Value).

[0070] Furthermore, it is desirable that the entered information be kept confidential by learning not to use it as a new answer. In addition, the generating AI may be a language model that has been specifically trained (e.g., fine-tuned) to generate answer information.

[0071] For example, the generation AI server 20 inputs the input information received from the information processing device 100 into the generation AI, and provides the output information output from the generation AI to the information processing device 100. The generation AI server 20 may also be implemented using an API (Application Programming Interface).

[0072] The content provider terminal 30 is an information processing terminal used by the content provider. For example, the content provider terminal 30 may be a desktop PC, a notebook PC, a tablet device, a mobile phone, a PDA, etc.

[0073] For example, the content provider terminal 30 provides content provision information to the information processing device 100 regarding the content it wishes to distribute. For example, the content provision information includes information that the content provider can freely input text information about the content, as well as information about images, videos, banners, flyers, etc., that the content provider has generated for the content.

[0074] To give a more specific example, let's assume the content is an advertisement. In this case, the content provider terminal 30 inputs advertising information related to the advertisement it wishes to distribute into the information processing device 100. Here, the advertising information includes input information in which the advertiser can freely input text information about the advertisement, as well as advertising creatives such as images, videos, banners, and flyers generated by the content provider for advertising purposes.

[0075] The payment server 40 is an information processing device that performs payment processing, and is implemented, for example, by a server device or a cloud system. For example, when various payment processes are performed for an expense, the payment server 40 works in cooperation with the information processing device 100 to perform the payment processing for the content provider.

[0076] The information processing device 100 is an information processing device that can communicate with various devices via a network N, and is implemented, for example, by a server device or a cloud system. For example, the information processing device 100 is connected to other various devices via the network N in a way that allows communication.

[0077] The information processing device 100 may also be an information processing device that provides various services to the user terminal 10. For example, various services include internet access, search services, SNS (Social Networking Service), e-commerce (EC), electronic payment, online games, online banking, online trading, accommodation / ticket reservations, video / music distribution, news, maps, route search, route guidance, route information, service information, and weather forecasts. To give a more specific example, the information processing device 100 may provide the above-mentioned services by cooperating with various external servers that provide these services. Alternatively, the information processing device 100 may mediate the provision of various services to the user by cooperating with various external servers.

[0078] [3. Configuration of the Information Processing Device] Next, an example of the functional configuration of the information processing device 100 will be described using Figure 5. As shown in Figure 5, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

[0079] (Regarding Communications Unit 110) The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to the network N by wire or wireless connection and transmits and receives information with various other devices.

[0080] (Regarding memory unit 120) The memory unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. The memory unit 120 also includes a learning data storage unit 121, a first learning model 122, a content provision information storage unit 123, a user information storage unit 124, a correction value information storage unit 125, an evaluation information storage unit 126, and a second learning model 127.

[0081] (Regarding the learning data storage unit 121) The training data storage unit 121 stores various information related to the training data. Here, Figure 6 shows an example of the training data storage unit 121 according to the embodiment. In the example shown in Figure 6, the training data storage unit 121 has items such as "training dataset ID (Identifier)", "input information", "output information", "sparse features", and "ground truth label".

[0082] The "Training Dataset ID" is an identifier that identifies the training dataset. The "Input Information" is the input information associated with the "Training Dataset ID". The "Output Information" is the output information associated with the "Training Dataset ID".

[0083] "Sparse features" are information about sparse features associated with the "training dataset ID". "Ground truth labels" are information about ground truth labels associated with the "training dataset ID".

[0084] For example, in Figure 6, "P1," identified by the training dataset ID, has input information "IN1," output information "OP1," sparse features "SF1," and the correct label "LT1."

[0085] In the example shown in Figure 6, the input information is represented by an abstract code such as "IN1," but the input information may also be numerical values, strings, or a file format containing various information indicating the input information.

[0086] (Regarding the content provision information storage unit 123) The content provision information storage unit 123 stores various information related to the content. Here, Figure 7 shows an example of the content provision information storage unit 123 according to the embodiment. In the example shown in Figure 7, the content provision information storage unit 123 has items such as "Content Provider ID" and "Content Provision Information". For example, "Content Provision Information" has items such as "Content ID", "Provision Information", "Content", and "Price".

[0087] The "Content Provider ID" is an identifier that identifies the content provider. The "Content ID" is an identifier that identifies the content associated with the "Content Provider ID". The "Provided Information" is the information provided by the content provider of the content associated with the "Content ID". For example, provided information may include textual information describing the content, textual information promoting the content, and information related to images and videos.

[0088] "Content" refers to information about the content associated with the "Content ID." "Fees" refers to information about the fees paid by the content provider when providing the content associated with the "Content ID."

[0089] For example, in Figure 7, "P1," identified by the content provider ID, has a content ID of "C1," provision information of "CP1," content of "CO1," and a price of "PF1."

[0090] In the example shown in Figure 7, the provided information is represented by an abstract code such as "CP1," but the provided information may also be numerical values, strings, or file formats containing various information indicating the provided information.

[0091] (Regarding the user information storage unit 124) The user information storage unit 124 stores various information about the user. Here, Figure 8 shows an example of the user information storage unit 124 according to the embodiment. In the example shown in Figure 8, the user information storage unit 124 has items such as "User ID" and "User Information". For example, "User Information" has items such as "Attribute Information" and "Conversation History".

[0092] A "User ID" is an identifier that identifies a user. "Attribute Information" is information about the user's attributes associated with the "User ID". For example, attribute information includes demographic attributes and psychographic attributes. For example, demographic attributes are demographic attributes. To give a more specific example, demographic attributes include age, gender, occupation, place of residence, annual income, family structure, etc. For example, psychographic attributes are psychological attributes. To give a more specific example, psychographic attributes include lifestyle, values, interests, etc.

[0093] The "conversation history" is information relating to the history of input information entered by a user associated with a "user ID" and output information provided by the generating AI server 20. The conversation history may also be information relating to the history of conversations between the user and the chatbot provided by the generating AI server 20. In this case, the conversation history is stored with the input information entered by the user and the corresponding output information provided by the chatbot.

[0094] For example, in Figure 8, "U1," identified by the user ID, has attribute information as "UA1" and conversation history as "UC1." In the example shown in Figure 8, attribute information is represented by abstract codes such as "UA1," but attribute information may also be numerical values, strings, or file formats containing various information indicating attribute information.

[0095] (Regarding the correction value information storage unit 125) The correction value information storage unit 125 stores various information related to the correction value. Here, Figure 9 shows an example of the correction value information storage unit 125 according to the embodiment. In the example shown in Figure 9, the correction value information storage unit 125 has items such as "Correction Value ID", "Content Provider ID", "Content ID", "Change Policy", and "Correction Value".

[0096] The "Correction Value ID" is an identifier that identifies the correction value. The "Content Provider ID" is an identifier that identifies the content provider associated with the "Correction Value ID". The "Content ID" is an identifier that identifies the content associated with the "Correction Value ID".

[0097] "Change Policy" refers to information about the change policy associated with the "Correction Value ID". "Correction Value" refers to information about the correction value associated with the "Correction Value ID".

[0098] For example, in Figure 9, "R1," identified by the correction value ID, has a content provider ID of "P1," a content ID of "C1," a change policy of "CI1," and a correction value of "RV1."

[0099] In the example shown in Figure 9, the change policy etc. was represented by an abstract code such as "CI1," but the change policy etc. may also be a numerical value, a string, or a file format containing various information indicating the change policy etc.

[0100] (Regarding the evaluation information storage unit 126) The evaluation information storage unit 126 stores various information related to the evaluation of the output information. Here, Figure 10 shows an example of the evaluation information storage unit 126 according to the embodiment. In the example shown in Figure 10, the evaluation information storage unit 126 has items such as "Content Provider ID", "Content ID", "Output Information", and "Evaluation".

[0101] The "Content Provider ID" is an identifier that identifies the content provider. The "Content ID" is an identifier that identifies the content associated with the "Content Provider ID".

[0102] "Output information" refers to the output information corresponding to the content associated with the "Content ID". "Evaluation" refers to information regarding the evaluation of the output information corresponding to the content associated with the "Content ID". For example, the evaluation may be the content provider's evaluation of the output information.

[0103] For example, in Figure 10, "P1," identified by the content provider ID, has a content ID of "C1," output information of "COP1," and a rating of "CE1." In the example shown in Figure 10, the output information is represented by abstract codes such as "COP1," but the output information may also be numerical values, strings, or file formats containing various information indicating the output information.

[0104] (Regarding the control unit 130) The control unit 130 is a controller, and is implemented, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing various programs (an example of an information processing program) stored in the memory device inside the information processing device 100 using RAM as the working area. Alternatively, the control unit 130 is a controller and can be implemented, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0105] As shown in Figure 2, the control unit 130 includes a reception unit 131, an acquisition unit 132, a provision unit 133, a conversion unit 134, a learning unit 135, a modification unit 136, a generation unit 137, an estimation unit 138, a determination unit 139, and a setting unit 140, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in Figure 5, and other configurations are also possible as long as they perform the information processing described later. Also, the connection relationships of the various processing units in the control unit 130 are not limited to the connection relationships shown in Figure 5, and other connection relationships are also possible.

[0106] (Regarding reception desk 131) The reception unit 131 receives various types of information. Specifically, the reception unit 131 receives input information from the user terminal 10. For example, suppose user U1 inputs text information as input information. In this case, the reception unit 131 receives the text information as input information from the user terminal 10. The reception unit 131 then stores this input information in the learning data storage unit 121.

[0107] Furthermore, the reception unit 131 receives output information corresponding to the input information, and information regarding the feature quantities output by the intermediate layer of the LL (an example of a predetermined layer in the LLM) when the input information is input. The reception unit 131 then stores this output information in the learning data storage unit 121.

[0108] Furthermore, the reception unit 131 receives information regarding the evaluation of the output information from the content provider terminal 30. For example, suppose the evaluation is on a 5-point scale. Also, suppose the evaluation of the output information is 5. In this case, the reception unit 131 receives information from the content provider terminal 30 indicating that the evaluation of the output information is 5. The reception unit 131 then stores this evaluation information in the evaluation information storage unit 126. The reception unit 131 may also further store the evaluated output information in the evaluation information storage unit 126.

[0109] Furthermore, the reception unit 131 receives various requests. For example, the reception unit 131 receives information regarding requests for the provision of change policies from the content provider terminal 30.

[0110] (Regarding acquisition section 132) The acquisition unit 132 acquires various types of information. Specifically, the acquisition unit 132 acquires user information from the user terminal 10. Then, the acquisition unit 132 stores the user information in the user information storage unit 124.

[0111] (Regarding Section 133) The provisioning unit 133 provides various types of information. For example, the provisioning unit 133 provides input information to the generation AI server 20.

[0112] Furthermore, the providing unit 133 provides output information to the user terminal 10. For example, the providing unit 133 provides the user terminal 10 with output information that includes text information containing references to competitors' automobile products and references to company A's automobile products in roughly equal proportions, as well as advertisements for company A's automobile products.

[0113] Furthermore, the provisioning unit 133 provides the content provider terminal 30 with information regarding the change policy, which is output by inputting predetermined evaluation information to the second learning model 127 that has been trained by the learning unit 135.

[0114] For example, the provision unit 133 provides information regarding a change policy corresponding to evaluation 5. To give a more specific example, the provision unit 133 provides information regarding a change policy corresponding to evaluation 5 in which the output information includes only references to company A's automotive products.

[0115] (Regarding the conversion unit 134) The transformation unit 134 converts the features output by the intermediate layer into sparse features. The transformation unit 134 then stores these sparse features in the training data storage unit 121. The transformation process performed by the transformation unit 134 can be realized using SAE-based technology that utilizes deep learning.

[0116] (Regarding Learning Section 135) The learning unit 135 trains the first learning model 122 on the relationship between predetermined input information input to the LLM, output information output by the LLM when the predetermined input information is input to the LLM, sparse features obtained by transforming the features output by the hidden layer when the predetermined input information is input to the LLM, and the correct labels attached to the sparse features. In other words, the learning unit 135 trains the first learning model 122 on the learning dataset stored in the learning data storage unit 121. Here, the first learning model 122 is a learning model called SAE. Then, the learning unit 135 stores the first learning model 122 in the storage unit 120.

[0117] Furthermore, the learning unit 135 trains the second learning model 127 on the relationship between the change policy and the evaluation. For example, suppose the change policy when generating output information is to increase the proportion of references to company A's automobile products in the output information if the output information includes references to a competitor's automobile products. In this case, the learning unit 135 trains the second learning model 127 on the relationship between such a change policy and an evaluation of 5. In this way, the learning unit 135 generates a second learning model 127 that outputs a change policy corresponding to a predetermined evaluation when information about a predetermined evaluation is input to the second learning model 127. The learning unit 135 then stores the second learning model 127 in the storage unit 120.

[0118] (Regarding change 136) The modification unit 136 modifies the sparse features, which are obtained by transforming the features output by the intermediate layer when a predetermined input is input, for an LLM that has been trained to generate an answer to a question input as output information, based on a predetermined modification policy. These sparse features indicate the generation strategy for the LLM to generate output information corresponding to the predetermined input information. For example, the modification unit 136 modifies the sparse features based on a predetermined modification policy so that the generation strategy is changed.

[0119] Here, we will explain the change process using Figure 2. In the example in Figure 2, node UN1 shows "0.2". Node UN2 shows "0.8". Node UN3 shows "0.6". Node UN4 shows "-0.2".

[0120] Here, the predetermined change policy is assumed to be pre-set by company A. Furthermore, the predetermined change policy is assumed to be a reference to a competitor's automotive product. In this case, the value of the corresponding node UN1 is increased to increase the proportion of references to company A's automotive products in the output information.

[0121] In other words, when the modification unit 136 modifies a sparse feature based on a predetermined modification policy, it modifies the value of node UN1. For example, the modification unit 136 modifies a sparse feature based on a correction value based on a predetermined modification policy and the sparse feature itself. As a result, the modification unit 136 modifies the sparse feature SFV1 to the sparse feature SFV2. For example, the modification unit 136 modifies a sparse feature based on a correction value. In the example in Figure 3, the modification unit 136 modifies the sparse feature SFV1 to the sparse feature SFV2 by multiplying it by formula (1). In the sparse feature SFV2, node UN1 shows "0.6". Also, in the sparse feature SFV2, node UN2 shows "0.8". Also, in the sparse feature SFV2, node UN3 shows "0.6". Also, in the sparse feature SFV2, node UN4 shows "-0.2".

[0122] As another example, suppose the predetermined change policy is a reference to a competitor's automotive product. Also, suppose the output information includes a reference to a competitor's automotive product. In this case, the values ​​of the corresponding nodes UN1 and UN4 are increased to increase the ratio of references to Company A's automotive product and references to the quality reliability of Company A's automotive product in the output information. In such a case, the change unit 136 modifies the sparse features to increase the ratio of references to Company A's automotive product and references to the quality reliability of Company A's automotive product in the output information.

[0123] Furthermore, the predetermined change policy is to mention that Company A's automobile products are prone to breaking. In this case, if the output information includes a mention that Company A's automobile products are prone to breaking, the values ​​of the corresponding nodes UN1 and UN4 will be increased to increase the ratio of the mention of Company A's automobile products to the mention of the quality reliability of Company A's automobile products in the output information. In such a case, the change unit 136 will modify the sparse features to increase the ratio of the mention of Company A's automobile products to the mention of the quality reliability of Company A's automobile products in the output information.

[0124] Furthermore, the predetermined modification policy shall be a reference to the automotive products of a competitor. In addition, references to the automotive products of a competitor shall be included in the output information. In this case, the values ​​of the corresponding nodes UN1 and UN3 shall be increased to increase the ratio of references to Company A's automotive products and references to pleasant feelings towards Company A's automotive products in the output information. In such a case, the modification unit 136 shall modify the sparse features to increase the ratio of references to Company A's automotive products and references to pleasant feelings towards Company A's automotive products in the output information.

[0125] As another example, the modification unit 136 may modify sparse features based on content provision information as a predetermined modification policy. For example, the modification unit 136 may modify sparse features based on information about the content as content provision information. To give a more specific example, the modification unit 136 may modify sparse features based on textual information describing the advertisement, textual information promoting the advertisement, or information about images and videos as content provision information.

[0126] Furthermore, the modification unit 136 may modify the sparse features based on information about other targets different from the target of the advertisement, as content provision information. For example, if the output information includes automobile products of competitors, the modification unit 136 may modify the sparse features so that only automobile products of company A are included in the output information.

[0127] Furthermore, the modification unit 136 may modify the sparse features based on information regarding fees paid by content providers as content provision information. For example, the modification unit 136 may modify the sparse features so that the output information includes advertisements provided by the content provider that paid the highest fee among the fees paid by multiple content providers.

[0128] Furthermore, the modification unit 136 may modify the sparse features based on the user's attribute information stored in the user information storage unit 124, as per a predetermined modification policy. For example, suppose the user is a man in his 30s. In this case, the modification unit 136 may modify the sparse features based on the attribute information so that only automobile products from company A are included in the output information. Alternatively, suppose the user's interests are automobiles, specifically SUVs (Sport Utility Vehicles). In this case, the modification unit 136 may modify the sparse features based on the attribute information so that only SUV automobile products from company A are included in the output information.

[0129] Furthermore, the modification unit 136 may modify the sparse features based on the user's attribute information estimated from the history of input information entered by the user, as a predetermined modification policy. For example, suppose the user is estimated to be a male in his 30s from the history of the user's input information. In this case, the modification unit 136 may modify the sparse features based on the estimated attribute information so that only automobile products of company A are included in the output information. Alternatively, suppose the user's interests are automobiles, specifically SUVs. In this case, the modification unit 136 may modify the sparse features based on the estimated attribute information so that SUV automobile products from company A are included in the output information.

[0130] Furthermore, the modification unit 136 may, as a predetermined modification policy, modify the sparse features based on the history of input information entered by the user and input information entered into the LLM, and the output information output by the LLM when the input information was entered. For example, suppose the history of input information and output information includes a reference to company A's automobile products. In this case, the modification unit 136 may modify the sparse features based on the history of input information and output information so that only company A's automobile products are included in the output information.

[0131] (Regarding the generation unit 137) The generation unit 137 generates various types of information. Specifically, the generation unit 137 uses the modified sparse features, which have been modified by the modification unit 136, as the output features of the intermediate layer, causing the LLM to generate output information corresponding to predetermined input information. For example, the generation unit 137 generates text information as output information that includes references to competitors' automobile products and references to company A's automobile products in roughly equal proportions. The generation unit 137 also generates output information that includes advertisements for company A's automobile products.

[0132] Here, using Figure 11, a specific example of the generation process executed by the generation unit 137 will be explained. Figure 11 is a conceptual diagram of the generation process according to the embodiment. In the example in Figure 11, an example is described in which the LLM outputs output information OU21 when input information IN21 is input to the LLM. In the example in Figure 11, input information IN21 is input to the LLM (step ST21). In this case, the feature quantity FV21 is a feature quantity output by the hidden layer. Also, for example, the feature quantity FV21 is composed of three nodes. Subsequently, the feature quantity FV21 is input to the SAE (corresponding to the first learning model 122) (step ST22).

[0133] In SAE, the feature vector FV21 is input to the encoder and converted into a sparse feature vector SFV21. The sparse feature vector SFV21 consists of four nodes UN21 to UN24. Here, a predetermined modification policy is to change the value of node UN22 by a factor of 10. In this case, the sparse feature vector SFV21 is changed to a sparse feature vector SFV22. Then, the sparse feature vector SFV22 is input to the decoder and converted into a feature vector.

[0134] Furthermore, the feature vector FV21 is input into a predetermined error function error(x) (step ST23). The predetermined error function error(x) is calculated, for example, as the reconstruction error between the SAE(x) reconstructed by the SAE and the input feature vector FV21 when the feature vector FV21 is input to the SAE.

[0135] Next, the feature quantity output from the predetermined error function error(x) (step ST25) and the feature quantity converted from the sparse feature quantity SFV22 (step ST24) are added together. Then, the added feature quantity FV21 is used to cause the LLM to generate output information OU21 corresponding to the input information IN21. In this way, the generation unit 137 causes the LLM to generate output information OU21 corresponding to the input information IN21.

[0136] (Regarding Estimation Section 138) The estimation unit 138 estimates a modification strategy for the LLM to generate the desired output information, based on sparse features obtained by transforming the features output by the intermediate layer when a predetermined input information is input, and which indicate a generation strategy for the LLM to generate the output information corresponding to the predetermined input information. The estimation unit 138 then stores the estimated modification strategy in the correction value information storage unit 125.

[0137] For example, the estimation unit 138 uses the first learning model 122 to estimate the output information that the LLM will output based on sparse features obtained by transforming the features output by the hidden layer. For example, suppose the output information that the LLM will output includes references to the automobile products of competitors. In this case, the estimation unit 138 estimates a change policy to increase the proportion of references to company A's automobile products in the output information, based on the sparse features obtained by transforming the features output by the hidden layer.

[0138] Furthermore, the estimation unit 138 estimates a change policy for a predetermined evaluation by inputting information about the predetermined evaluation to the second learning model 127. For example, the estimation unit 138 estimates a change policy corresponding to evaluation 5 as information about the predetermined evaluation. To give a more specific example, the estimation unit 138 estimates a change policy corresponding to evaluation 5 in which the output information includes only references to company A's automobile products.

[0139] (Regarding decision section 139) The decision unit 139 determines correction values ​​for modifying sparse features based on the sparse features and the modification policy. The decision unit 139 then stores these correction values ​​in the correction value information storage unit 125.

[0140] In the example shown in Figure 3, the determination unit 139 determines a correction value to increase the value of the corresponding node UN1 in order to increase the proportion of references to the company's automotive products in the output information. For example, the determination unit 139 determines a transformation matrix as shown in equation (1) as the correction value. In this case, k is, for example, 0.5.

[0141] (Regarding setting section 140) The setting unit 140 sets a fee based on the correction value for the content provider. In this case, the setting unit 140 performs payment processing for the content provider at that fee. For example, the setting unit 140 works in cooperation with the payment server 40 to perform payment processing for the fee for the content provider.

[0142] To give a more specific example, the configuration unit 140 performs payment processing with the payment server 40 for the fee each time, or at predetermined intervals. As a result, the configuration unit 140 performs payment processing with the content provider for the fee based on the correction value.

[0143] [4. Processing procedures executed by the information processing device] Next, we will explain the flow of each information processing performed by the information processing device 100 using Figures 12 to 15.

[0144] [4-1. Processing Procedure (1)] First, the procedure for the learning process executed by the information processing device 100 according to the embodiment will be explained using Figure 12. Figure 12 is a flowchart showing an example of the flow of the learning process executed by the information processing device 100 according to the embodiment.

[0145] As shown in Figure 12, the acquisition unit 132 determines whether a predetermined timing has elapsed (step S101). Specifically, if the predetermined timing has not elapsed (step S101; No), the acquisition unit 132 waits until the predetermined timing has elapsed. The predetermined timing here is any arbitrary timing.

[0146] Meanwhile, when a predetermined timing has elapsed (step S101; Yes), the acquisition unit 132 acquires the input information input to the LLM, the output information output by the LLM when the input information is input to the LLM, the sparse features obtained by transforming the features output by the hidden layer when the input information is input to the LLM, and the ground truth labels attached to the sparse features (step S102). Subsequently, the learning unit 135 trains the first learning model 122 on the relationship between the input information, the output information, the sparse features, and the ground truth labels attached to the sparse features (step S103).

[0147] [4-2. Processing Procedure (2)] Next, the procedure for the generation process executed by the information processing device 100 according to the embodiment will be described using Figure 13. Figure 13 is a flowchart showing an example of the flow of the generation process executed by the information processing device 100 according to the embodiment.

[0148] As shown in Figure 13, the reception unit 131 receives input information from the user (step S201). Specifically, if the reception unit 131 has not received input information from the user (step S201; No), it waits until it receives input information from the user.

[0149] On the other hand, when the reception unit 131 receives input information from the user (step S201; Yes), the modification unit 136 modifies the sparse features based on a predetermined modification policy (step S202).

[0150] Next, the generation unit 137 uses the transformed sparse features as the features output by the intermediate layer to cause the LLM to generate output information corresponding to the input information (step S203). Then, the providing unit 133 provides the output information generated by the generation unit 137 to the user (step S204).

[0151] [4-3. Processing Procedure (3)] Next, the procedure for the decision processing performed by the information processing device 100 according to the embodiment will be described using Figure 14. Figure 14 is a flowchart showing an example of the flow of the decision processing performed by the information processing device according to the embodiment.

[0152] As shown in Figure 14, the estimation unit 138 determines whether a predetermined timing has elapsed (step S301). Specifically, if the predetermined timing has not elapsed (step S301; No), the estimation unit 138 waits until the predetermined timing has elapsed. The predetermined timing here is any arbitrary timing.

[0153] On the other hand, if a predetermined time has elapsed (step S301; Yes), the estimation unit 138 estimates the change policy based on the sparse features (step S302). Subsequently, the determination unit 139 determines the correction value based on the change policy and the sparse features (step S303).

[0154] Then, the modification unit 136 modifies the sparse features based on the correction value determined by the determination unit 139 (step S304). Subsequently, the generation unit 137 uses the modified sparse features as the transformed features output by the intermediate layer to cause the LLM to generate output information corresponding to predetermined input information (step S305).

[0155] Then, the provision unit 133 provides the output information generated by the generation unit 137 to the content provider (step S306). Subsequently, the setting unit 140 sets the fee based on the correction value (step S307).

[0156] [4-4. Processing Procedure (4)] Next, the procedure for the learning process executed by the information processing device 100 according to the embodiment will be described using Figure 15. Figure 15 is a flowchart showing an example of the flow of the learning process executed by the information processing device according to the embodiment.

[0157] As shown in Figure 15, the reception unit 131 receives information regarding the evaluation of the output information from the content provider (step S401). Specifically, if the reception unit 131 has not received information regarding the evaluation of the output information from the content provider (step S401; No), it waits until it receives such information from the content provider.

[0158] On the other hand, when the reception unit 131 receives information from the content provider regarding the evaluation of the output information (step S401; Yes), the learning unit 135 trains the second learning model 127 on the relationship between the change policy and the evaluation for each content provider (step S402).

[0159] Next, the estimation unit 138 inputs information about a predetermined evaluation to the second learning model 127 to estimate a change policy corresponding to the predetermined evaluation (step S403). Then, the provision unit 133 provides the content provider with the information about the change policy estimated by the estimation unit 138 (step S404).

[0160] [5. Variations] The information processing device 100 described above may be implemented in various other forms besides those described above. Therefore, other embodiments of the information processing device 100 will be described below.

[0161] [5-1. Contents] In the above embodiment, advertising content was used as an example of content, but the invention is not limited to this. For example, the content may be any kind of content.

[0162] [5-2. Designated Services] In the above embodiment, an advertising service was used as an example of a predetermined service, but the invention is not limited to this. For example, the predetermined service may be any service. For example, the predetermined service may be a service that recommends content, or a service that provides a guardrail function or security function against fraudulent emails, etc.

[0163] [5-3. Content Providers] In the above embodiment, a content provider was used as an example, but instead of a content provider, for example, an administrator of a predetermined service provided by the information processing device 100, who manages the content provided by the content provider, may provide and manage the content.

[0164] [5-4. Input Information] In the above embodiment, an example was given where the input information is textual information, but the embodiment is not limited to this. For example, the input information may be an image or video entered by the user. Alternatively, the input information may be audio information spoken by the user.

[0165] [5-5. Output Information] In the above embodiment, an example was given where the output information is text information, but the embodiment is not limited to this. For example, the output information may be audio information or the like that is output to the user.

[0166] [5-6. Training Dataset] The above embodiment shows an example in which the information processing device 100 generates a training dataset, but it is not limited to this. For example, the information processing device 100 may acquire a training dataset generated by another external server and store it in a predetermined storage unit.

[0167] [5-7. Sparse Features] In the above embodiment, a sparse feature consisting of four nodes was used as an example, but the embodiment is not limited to this. For example, a sparse feature may consist of any number of nodes. That is, the number of dimensions of the vector represented by the sparse feature may be any number.

[0168] Furthermore, a sparse feature may be composed of multiple sparse features. In this case, the modification unit 136 modifies each of the multiple sparse features that make up the sparse feature based on a predetermined modification policy.

[0169] Furthermore, a sparse feature may be composed of multiple sparse features with different vector dimensions. For example, a sparse feature may be composed of a first sparse feature having the first dimension of the vector and a second sparse feature having the second dimension of the vector. For example, the first dimension of the vector represented by the first sparse feature may be smaller than the second dimension of the vector represented by the second sparse feature. In this case, the first sparse feature is composed of nodes that represent a higher level of abstraction than the level of abstraction of the meaning represented by one of the nodes that make up the second sparse feature.

[0170] Here, using Figure 16, we will explain the generation process that the generation unit 137 performs when a sparse feature is composed of multiple sparse features with different vector dimensions. Figure 16 is a conceptual diagram of the generation process related to a modified example. In the example in Figure 16, the direction in which the learning process is executed is direction DI31. Feature quantity FV31 is a feature output by the hidden layer. For example, feature quantity FV31 is composed of three nodes.

[0171] In this case, it is assumed that multiple SAEs are pre-generated in order to generate multiple sparse features. In the example in Figure 16, SAE1 and SAE2 are pre-generated. For example, SAE1 converts feature FV31 to the first sparse feature SFV31. SAE2 converts feature FV31 to the second sparse feature SFV32. Subsequently, SAE1 converts the first sparse feature SFV31 to feature FV32. SAE2 converts the second sparse feature SFV32 to feature FV33.

[0172] The generation unit 137 then calculates the average value of feature quantity FV32 and feature quantity FV33. In this case, the generation unit 137 calculates feature quantity FV34, which represents the average value of feature quantity FV32 and feature quantity FV33, as the average value. Subsequently, the generation unit 137 uses feature quantity FV34 as a feature quantity output by the hidden layer to cause the LLM to generate output information corresponding to predetermined input information.

[0173] In this way, even if there are multiple sparse features with different vector dimensions, the generation unit 137 converts each of the multiple sparse features into a feature, calculates the average feature, and uses the calculated feature as the feature output by the hidden layer to cause the LLM to generate output information corresponding to the predetermined input information. As a result, the generation unit 137 can generate a first learning model with high expressive power. Furthermore, by using the first learning model, the generation unit 137 can generate suitable output information.

[0174] Next, we will describe the modification process performed by the modification unit 136 when a sparse feature is composed of multiple sparse features with different vector dimensions. For example, suppose a sparse feature is composed of a first sparse feature having the first dimension of the vector and a second sparse feature having the second dimension of the vector. In this case, assume that a transformation matrix corresponding to the first dimension of the vector represented by the first sparse feature is calculated in advance as the first correction value. Also, assume that a transformation matrix corresponding to the second dimension of the vector represented by the second sparse feature is calculated in advance as the second correction value. At this time, the modification unit 136 modifies the first sparse feature by multiplying it by the first correction value. The modification unit 136 also modifies the second sparse feature by multiplying it by the second correction value.

[0175] In this case, the generation unit 137 calculates the average value of the first feature obtained by transforming the first sparse feature modified by the modification unit 136, and the second feature obtained by transforming the modified second sparse feature. Then, the generation unit 137 uses the average value of the first feature and the second feature as the feature output by the hidden layer, and causes the LLM to generate output information corresponding to the predetermined input information.

[0176] [5-8. Evaluation] In the above embodiment, an example was given in which the receiving unit 131 receives information regarding the evaluation of output information output by LLM that corresponds to predetermined input information, but the embodiment is not limited to this. For example, the receiving unit 131 may receive information regarding the evaluation of each predetermined piece of information included in the output information as an evaluation of the output information.

[0177] For example, suppose the output information is text information. In this case, the receiving unit 131 may receive information regarding the evaluation of each word contained in the text information. The receiving unit 131 may also receive information regarding the evaluation of each sentence contained in the text information. The receiving unit 131 may also receive information regarding the evaluation of each clause of a sentence contained in the text information. The receiving unit 131 may also receive information regarding the evaluation of each paragraph contained in the text information. In this way, the receiving unit 131 may receive information regarding the evaluation of each predetermined unit.

[0178] Furthermore, the reception unit 131 may also receive information regarding user evaluations of the output information as an evaluation of the output information. For example, the reception unit 131 may receive information regarding the user's CTR (Click Through Rate) for content provided by the content provider as an evaluation from the user. For example, suppose the output information includes content provided by the content provider. In this case, the reception unit 131 may receive information regarding the user's CTR for the content as an evaluation from the user.

[0179] Furthermore, the reception unit 131 may also accept information regarding the user's CVR (Conversion Rate) for content provided by the content provider as user evaluation. For example, suppose the output information includes content provided by the content provider. In this case, the reception unit 131 may accept information regarding the user's CVR for the content as user evaluation. This allows the reception unit 131 to accept information regarding user evaluations as well as evaluations from content providers.

[0180] [5-9. Estimation Process] In the above embodiment, an example was described in which the estimation unit 138 estimates a modification policy for the LLM to generate desired output information based on sparse features that indicate a generation policy for the LLM to generate output information corresponding to predetermined input information, but the embodiment is not limited to this. For example, the estimation unit 138 may estimate the modification policy based on information about the content as content provision information. The estimation unit 138 may also estimate the modification policy based on information about other targets different from the target of the content as content provision information. The estimation unit 138 may also estimate the modification policy based on information about the fees paid by the content provider when providing the content to the user as content provision information. The estimation unit 138 may also estimate the modification policy based on the output information output by the LLM and the content provision information received from the content provider that provides the predetermined content.

[0181] [5-10. Setup Process] In the above embodiment, an example was given in which the setting unit 140 sets a fee based on a correction value for the content provider, but the embodiment is not limited to this. For example, the setting unit 140 may set a fee for the content provider according to the number of dimensions of the vector indicated by the sparse feature and the correction value. In this case, the setting unit 140 performs payment processing for the content provider at such a fee. For example, the setting unit 140 works in cooperation with the payment server 40 to perform payment processing for the fee for the content provider.

[0182] To give a more specific example, the configuration unit 140 performs payment processing with the payment server 40 for a fee each time, or at predetermined intervals. This allows the configuration unit 140 to perform payment processing with the content provider at a fee corresponding to the number of dimensions of the vector indicated by the sparse feature and the correction value. In this way, the configuration unit 140 can perform payment processing with the content provider at a suitable fee.

[0183] [5-11. Second Learning Model for Each Content Provider] In the above embodiment, an example was given in which the learning unit 135 trains the second learning model 127 on the relationship between the change policy and the evaluation, but it is not limited to this. For example, suppose that information on the evaluation is received from each content provider. In this case, the learning unit 135 trains the second learning model 127 on the relationship between the change policy and the evaluation for each content provider. As a result, the learning unit 135 can generate a second learning model 127 that is suitable for each content provider.

[0184] [5-12. Auction] Furthermore, the information processing device 100 may receive information from content providers regarding fees for correction values. For example, the receiving unit 131 receives information regarding the first fee from the first content provider. The receiving unit 131 also receives information regarding the second fee from the second content provider.

[0185] Here, the first fee is assumed to be higher than the second fee. In this case, the information processing device 100 generates sparse features that indicate information related to the content provided by the first content provider. The information processing device 100 may also decide that the content to be included in the output information is the content provided by the first content provider. In this way, the information processing device 100 may select from among the multiple content providers to provide the content to be included in the output information, based on the fees received from multiple content providers. In this case, the information processing device 100 modifies the sparse features that indicate information related to the content provided by the content provider with the higher fee by multiplying them by a transformation matrix in which the value of k shown by formula (1) has been changed.

[0186] In such cases, the fee may be adjusted according to the number of dimensions of the vector represented by the sparse feature. For example, suppose the first dimension of the vector represented by the first sparse feature is smaller than the second dimension of the vector represented by the second sparse feature. In this case, the first sparse feature may be priced higher than the second sparse feature.

[0187] Furthermore, the fees may vary depending on whether a single node constituting a sparse feature is a word, sentence, paragraph, or document. In this case, for example, the fee for a word may be lower than that for a paragraph.

[0188] Furthermore, the information processing device 100 may receive information from content providers regarding fees incurred when including content in the output information for a predetermined period. In this case, the fee is a fee corresponding to the predetermined period.

[0189] [5-13. Program] Furthermore, the information processing device 100 according to the above-described embodiment is realized by a computer 1000 having a configuration as shown in Figure 17. Figure 17 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which an arithmetic unit 1030, a cache 1040, a memory 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected by a bus 1090.

[0190] The arithmetic unit 1030 operates based on programs stored in the cache 1040 and memory 1050, as well as programs read from the input device 1020, and executes various processes. The cache 1040 is a memory device, such as RAM, that temporarily stores data used by the arithmetic unit 1030 for various calculations. The memory 1050 is a storage device where data used by the arithmetic unit 1030 for various calculations and various databases are registered, and is implemented using ROM (Read Only Memory), HDD (Hard Disk Drive), flash memory, etc.

[0191] Output IF1060 is an interface for transmitting information to be output to output devices 1010, such as monitors and printers, and is implemented using connectors of standards such as USB (Universal Serial Bus), DVI (Digital Visual Interface), and HDMI (High Definition Multimedia Interface). Input IF1070 is an interface for receiving information from input devices 1020, such as mice, keyboards, and scanners, and is implemented using, for example, USB.

[0192] The input device 1020 may also be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), DVD (Digital Versatile Disc), or PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), tape media, magnetic recording media, or semiconductor memory. Furthermore, the input device 1020 may be an external storage medium such as a USB memory stick.

[0193] Network IF1080 receives data from other devices via network N and sends it to the arithmetic unit 1030, and also transmits data generated by the arithmetic unit 1030 to other devices via network N.

[0194] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or memory 1050 onto the cache 1040 and executes the loaded program.

[0195] For example, when computer 1000 functions as an information processing device 100, the arithmetic unit 1030 of computer 1000 realizes the functions of the control unit 130 by executing a program loaded on the cache 1040.

[0196] [5-14. Others] Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0197] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0198] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0199] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit," etc. For example, a generation unit can be replaced with a generation means or a generation circuit.

[0200] [6. Effects] As described above, the information processing device 100 according to the embodiment includes a reception unit 131 and a learning unit 135. The reception unit 131 receives information regarding the evaluation of the output information output by the first learning model, which corresponds to the predetermined input information, when predetermined input information is input to a first learning model (corresponding to an LLM in the embodiment) that has been trained to generate an answer to a question input as output information. Based on the content provision information received from content providers that provide content in a predetermined service, the learning unit 135 trains a second learning model (corresponding to a second learning model 127 in the embodiment), which is a different learning model from the first learning model, on a second learning model (corresponding to a second learning model 127 in the embodiment) for each content provider, regarding the relationship between the modification policy for the first learning model to generate desired output information when the output information output by the first learning model is modified, and the evaluation.

[0201] The information processing device 100 according to this embodiment can perform a learning process suitable for a predetermined service.

[0202] Furthermore, in the information processing device 100 according to this embodiment, the receiving unit 131 receives information regarding the evaluation of the output information from the content provider as an evaluation of the output information.

[0203] As a result, the information processing device 100 according to the embodiment can receive appropriate evaluations of the output information.

[0204] Furthermore, in the information processing device 100 according to this embodiment, the reception unit 131 receives information regarding user evaluations of the output information as an evaluation of the output information.

[0205] As a result, the information processing device 100 according to the embodiment can receive appropriate evaluations of the output information.

[0206] Furthermore, in the information processing device 100 according to this embodiment, the reception unit 131 receives information regarding the user's CTR (Click-Through Rate) for content provided by the content provider as evaluation from the user.

[0207] As a result, the information processing device 100 according to the embodiment can receive appropriate evaluations of the output information.

[0208] Furthermore, in the information processing device 100 according to this embodiment, the reception unit 131 receives information regarding the user's conversion rate (CVR) for content provided by the content provider, as evaluation from the user.

[0209] As a result, the information processing device 100 according to the embodiment can receive appropriate evaluations of the output information.

[0210] Furthermore, in the information processing device 100 according to this embodiment, the receiving unit 131 receives information regarding the evaluation of each predetermined piece of information included in the output information as an evaluation of the output information.

[0211] As a result, the information processing device 100 according to the embodiment can receive appropriate evaluations of the output information.

[0212] Furthermore, the information processing device 100 according to the embodiment further includes a providing unit 133 that provides information on the change policy output to the content provider by inputting predetermined evaluation information to the second learning model trained by the learning unit 135.

[0213] As a result, the information processing device 100 according to the embodiment can provide the content provider with information regarding a suitable change policy.

[0214] Furthermore, the information processing device 100 according to the embodiment further includes an estimation unit 138 that estimates a change policy for a predetermined evaluation by inputting information related to a predetermined evaluation to the second learning model.

[0215] As a result, the information processing device 100 according to the embodiment can estimate a suitable change policy.

[0216] Although some embodiments of the present invention have been described in detail above with reference to the drawings, these are illustrative examples, and the present invention can be implemented in various other forms with modifications and improvements based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section of the invention. [Explanation of Symbols]

[0217] N Network 1. Information Processing System 10. User terminals 20 Generation AI Server 30. Content Provider Main Terminal 40 Payment Server 100 Information Processing Devices 110 Communications Department 120 Storage section 121 Learning Data Storage Unit 122 First Learning Model 123 Content Provision Information Storage Unit 124 User information storage unit 125 Correction Value Information Storage Unit 126 Evaluation Information Storage Unit 127 Second Learning Model 130 Control Unit 131 Reception Department 132 Acquisition Department 133 Provision Department 134 Conversion Unit 135 Learning Department 136 Changes 137 Generation part 138 Estimation Department 139 Decision Section 140 Setting section

Claims

1. A receiving unit receives information regarding the evaluation of the output information corresponding to the predetermined input information, when predetermined input information is input to a first learning model which has been trained to generate output information that answers questions input as input information, Based on content provision information received from content providers who provide content in a predetermined service, a learning unit that provides a modification policy for the first learning model to generate desired output information when the output information output by the first learning model is modified, and a second learning model, which is a different learning model from the first learning model, to learn the relationship with the evaluation for each content provider. An information processing device characterized by comprising:

2. The aforementioned reception unit is As an evaluation of the output information, information regarding the content provider's evaluation of the output information is received. The information processing apparatus according to feature 1.

3. The aforementioned reception unit is As an evaluation of the output information, information regarding user evaluations of the output information is received. The information processing apparatus according to feature 1.

4. The aforementioned reception unit is As part of the user evaluation, information regarding the user's CTR (Click Through Rate) for the content provided by the content provider will be accepted. The information processing apparatus according to claim 3.

5. The aforementioned reception unit is As part of the user's evaluation, information regarding the user's CVR (Conversion Rate) for the content provided by the content provider will be accepted. The information processing apparatus according to claim 3.

6. The aforementioned reception unit is As an evaluation of the output information, information regarding the evaluation of each predetermined piece of information included in the output information is received. The information processing apparatus according to feature 1.

7. The system further comprises a providing unit that provides the content provider with information on the change policy output by inputting predetermined evaluation information to the second learning model trained by the learning unit. The information processing apparatus according to feature 1.

8. The second learning model further comprises an estimation unit that estimates a policy for changing a predetermined evaluation by inputting information about that predetermined evaluation. The information processing apparatus according to feature 1.

9. A method of information processing performed by a computer, A receiving step in which, when predetermined input information is input to a first learning model that has been trained to generate an answer to a question input as output information, the receiving step receives information regarding the evaluation of the output information output by the first learning model that corresponds to the predetermined input information, Based on content provision information received from content providers who provide content in a predetermined service, the first learning model modifies the output information output by the first learning model to generate desired output information, and the learning process involves training a second learning model, which is a different learning model from the first learning model, on the relationship with the evaluation for each content provider. An information processing method characterized by including

10. A receiving procedure that, when predetermined input information is input to a first learning model trained to generate output information that answers questions input as input information, receives information regarding the evaluation of the output information output by the first learning model that corresponds to the predetermined input information, Based on content provision information received from content providers who provide content in a predetermined service, a modification policy for the first learning model to generate desired output information when the output information output by the first learning model is modified, and a learning procedure for training a second learning model, which is a different learning model from the first learning model, on the relationship with the evaluation for each content provider. An information processing program characterized by causing a computer to execute it.

Images