Information processing device, information processing method, and program

Abstract

An information processing device according to an aspect of the present disclosure comprises: a conversion unit that converts a plurality of queries in a vector format for a large-scale language model, into a query in a matrix format; and an inference unit that executes inference processing of the large-scale language model using the matrix-format query as input, by using a processor that executes matrix operations by parallel processing.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] Large language models (LLMs) are known as language models constructed using deep learning technology with large amounts of data (see, for example, Non-Patent Document 1), and are expected to be applied to various fields. For example, in contact centers (which may also be called "call centers"), large language models are expected to be used to generate information (e.g., summaries of conversations with customers) to support operators in answering calls.

[0003] Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, Jianfeng Gao, "Large Language Models: A Survey", arXiv:2402.06196 [cs.CL].

[0004] However, for example, when a large-scale language model is used simultaneously by many operators, the server that executes the inference process using the large-scale language model requires a large amount of hardware resources. On the other hand, when a large-scale language model is applied to a contact center, due to the nature of the contact center, it is expected that the server that executes the inference process using the large-scale language model will be built on-premise. For this reason, it is considered that scaling out or scaling up of hardware resources is often difficult.

[0005] The present disclosure has been made in consideration of the above points, and aims to provide a technology that can efficiently execute inference processing using a large-scale language model.

[0006] An information processing device according to one aspect of the present disclosure includes a conversion unit that converts multiple vector-format queries for a large-scale language model into matrix-format queries, and an inference unit that uses a processor that performs matrix operations in parallel to perform inference processing of the large-scale language model using the matrix-format queries as input.

[0007] Inference processing using large-scale language models can be performed efficiently.

[0008] 1 is a diagram illustrating an example of the overall configuration of a system according to the present embodiment; FIG. 2 is a diagram illustrating an example of the hardware configuration of a server according to the present embodiment; FIG. 3 is a diagram illustrating an example of the functional configuration of a server according to the present embodiment; FIG. 4 is a flowchart illustrating an example of the operation of a server when receiving a request; FIG. 5 is a flowchart illustrating an example of the operation of a server when executing inference processing using a large-scale language model; and FIG. 6 is a flowchart illustrating an example of the operation of a server when transmitting a response.

[0009] An embodiment of the present invention will be described in detail below with reference to the drawings. In the following embodiment, it is assumed that a large-scale language model is applied to a contact center, and information for supporting an operator's telephone response work (e.g., a summary of the contents of a call with a customer) is generated by the large-scale language model. However, the application of a large-scale language model is not limited to a contact center, and the following embodiment can be similarly applied to cases where a large-scale language model is applied to other applications.

[0010] <Proposed Method> Large-scale language models are generally realized by machine learning models such as a Transformer (Reference 1), which is a type of encoder-decoder model. In inference processing using a large-scale language model, when a text called a query is given, a text that satisfies the query is generated as a query result using pre-trained model parameters (weights). Hereinafter, the trained model parameters of a large-scale language model are referred to as {W ｉ |i=1,...,I}, where each W ｉ are matrices of a certain size with real-valued elements, and I is the number of matrices.

[0011] Inference processing using a large-scale language model requires the use of each model parameter W ｉ In other words, the inference process using a large-scale language model involves a linear transformation using f ｉ (x ｉ ) = W ｉ x ｉ Here, x ｉ is a linear transformation f ｉ In addition, in inference processing using a large-scale language model, when a query is given, text generation for the query is executed in one thread.

[0012] For this reason, when N queries are given, N threads are generally used, and inference processing is performed using a large-scale language model in each thread. Therefore, when N is large (for example, when many operators at a contact center use a large-scale language model almost simultaneously), there is a possibility that hardware resources such as memory in a graphics processing unit (GPU) may be insufficient. Note that the above N is also called "multiplicity" or "query multiplicity."

[0013] To address the issue of potential hardware resource shortages when the degree of redundancy is high, scaling out or scaling up hardware resources is one option. However, because personal information and other data is often exchanged between agents and customers in contact centers, the nature of such systems means that servers that perform inference processing using large-scale language models are expected to be built on-premise. For this reason, scaling out or scaling up hardware resources is often considered difficult due to factors such as cost.

[0014] Therefore, we propose a method for improving the efficiency of inference processing using a large-scale language model by executing inference processing for multiple queries in a single thread.

[0015] Let each of the N queries be q １ , ..., q Ｎ Each query q ｎis represented by a token sequence. A token is a portion of text that serves as a processing unit when processing text. Examples of tokens include words, phrases, and subwords. A dictionary that associates tokens with integer values ​​is called a "vocabulary set."

[0016] When the vocabulary set is V, each token is assigned an integer value between 1 and |V|. ｎ Is, L ｎ It can be considered as a dimensional (integer-valued) vector, where L ｎ is the query q ｎ is the number of tokens that make up the

[0017] So, the query q ｎ The matrix Q:=(q １ , ..., q Ｎ ) is created, and then inference processing is performed in one thread using a large-scale language model with this matrix Q as input. At this time, each linear transformation f ｉ So, f ｉ (X ｉ ) = W ｉ X ｉ Here, we calculate the linear transformation ｉ is a linear transformation f ｉ Query q ｎ The input corresponding to x ｉｎ As X ｉ :=(x ｉ１ , ..., x ｉＮ ) (i.e., the vector x ｉｎ is the matrix with f as the nth column. ｉ (X ｉ ) = W ｉ X ｉ The matrix operation can be efficiently calculated by a GPU.

[0018] From the above, N queries q １ , ..., q ＮIn contrast, it is possible to execute inference processing using a large-scale language model in a single thread. This improves the efficiency of inference processing using a large-scale language model, so that even when the degree of multiplicity is high (i.e., when N is large), it is possible to execute inference processing using a large-scale language model without scaling out or scaling up hardware resources.

[0019] A system 1 including a server 10 that executes inference processing using a large-scale language model according to the proposed method will be described below.

[0020] <Example of Overall Configuration of System 1> An example of the overall configuration of the system 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the overall configuration of the system 1 according to this embodiment.

[0021] 1, a system 1 according to this embodiment includes a server 10 and a plurality of clients 20. The server 10 and each client 20 are connected to each other so as to be able to communicate with each other via a network 30, which may include a LAN (Local Area Network) or the like.

[0022] The server 10 executes inference processing using a large-scale language model according to the proposed method, using a query included in a request received from the client 20. The server 10 also transmits a response including a query result generated by the inference processing using the large-scale language model to the client 20. The server 10 is realized by, for example, an information processing device (computer) such as a general-purpose server, or an information processing system (computer system) configured of such devices.

[0023] The client 20 transmits a request including a query to the server 10, and receives a response including a query result for the query from the server 10. The client 20 is realized by a device such as a PC (personal computer), a smartphone, a tablet terminal, a wearable device, etc. A specific example of the client 20 is a terminal used by an operator at a contact center.

[0024] 1 is an example, and the system 1 is not limited to this. For example, the system 1 may include various devices other than the server 10 and the client 20.

[0025] <Example of Hardware Configuration of Server 10> An example of the hardware configuration of the server 10 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the hardware configuration of the server 10 according to this embodiment.

[0026] 2, the server 10 according to this embodiment includes an external I / F 101, a communication I / F 102, a RAM (Random Access Memory) 103, a ROM (Read Only Memory) 104, a HDD (Hard Disk Drive) 105, a CPU (Central Processing Unit) 106, and a GPU 107. Each of these pieces of hardware is connected to each other via a bus 108 so as to be able to communicate with each other.

[0027] The external I / F 101 is an interface with an external device such as a recording medium 101a. The server 10 can read from and write to the recording medium 101a via the external I / F 101. Examples of the recording medium 101a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0028] The communication I / F 102 is an interface for connecting to the network 30. The RAM 103 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 104 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The HDD 105 is a storage device that stores programs and data. The CPU 106 is a computing device that reads programs and data from the ROM 104, HDD 105, etc. onto the RAM 103 and processes the data in accordance with the programs. The GPU 107 is a computing device that has multiple cores and is capable of performing parallel processing by the multiple cores working together. Compared to the CPU 106, the GPU 107 can perform relatively simple calculations such as matrix operations at higher speeds through parallel processing.

[0029] The server 10 according to the present embodiment has the hardware configuration shown in FIG. 2 , thereby enabling the operation described below. Note that the hardware configuration shown in FIG. 2 is merely an example, and the hardware configuration of the server 10 according to the present embodiment is not limited thereto. For example, the server 10 according to the present embodiment may have an input device such as a keyboard, a mouse, a touch panel, or physical buttons, or a display device such as a display or a display panel. Furthermore, for example, the server 10 according to the present embodiment may have another storage device such as an SSD (Solid State Drive) or flash memory instead of or in addition to the HDD 105. Furthermore, for example, the server 10 according to the present embodiment may have an NPU (Neural Network Processing Unit) or a TPU (Tensor Processing Unit) instead of or in addition to the GPU 107, or may have another computing device capable of performing at least matrix operations at high speed through parallel processing. Furthermore, for example, the server 10 according to this embodiment may have multiple CPUs 106, multiple GPUs 107, multiple HDDs 105, or various other hardware besides the hardware shown in the figure.

[0030] Hereinafter, the GPU 107, NPU, TPU, and other computing devices that can perform at least matrix operations at high speed through parallel processing will be collectively referred to as "parallel processing specialized processors."

[0031] <Example of Functional Configuration of Server 10> An example of the functional configuration of the server 10 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the functional configuration of the server 10 according to this embodiment.

[0032] 3, the server 10 according to this embodiment includes a request receiving unit 201, a request control unit 202, a conversion unit 203, an inference unit 204, a response control unit 205, and a response sending unit 206. Each of these units is realized by processing executed by a computing device such as the CPU 106 of one or more programs installed in the server 10. The server 10 according to this embodiment also includes a request temporary storage unit 301 and a response temporary storage unit 302. The request temporary storage unit 301 and the response temporary storage unit 302 are realized by storage areas such as the RAM 103 and the HDD 105, for example.

[0033] The request receiving unit 201 receives a request sent from a client 20. The request includes sender information, which is identification information for identifying the client 20 that sent the request, and a query specified or set by the client 20. Examples of sender information include a client ID and an IP (Internet Protocol) address. Examples of queries include a prompt consisting of an instruction such as "Please summarize the following input sentence in 30 characters or less" and an input sentence such as speech-recognized text that is the result of speech recognition of a voice conversation between an operator and a customer. Note that a prompt may also be called, for example, an "instruction," "command," or "question."

[0034] However, a request may further include information other than the sender information and the query, such as a priority indicating how much priority the request should be given.

[0035] Furthermore, the request receiving unit 201 stores the request received from the client 20 in the request temporary storage unit 301 .

[0036] The request control unit 202 acquires N requests from the request temporary storage unit 301. Hereinafter, the n-th request will be referred to as req ｎ Note that N, which represents the degree of multiplicity, is determined in advance based on, for example, the memory capacity of a parallel processing specialized processor such as the GPU 107 and the average length of a query (i.e., the average number of tokens constituting a query).

[0037] The conversion unit 203 converts the queries included in each of the N requests acquired by the request control unit 202 into a matrix format. ｎ Let q be the query contained in ｎ The conversion unit 203 converts N queries q １ , ..., q Ｎ Let Q:=(q １ , ..., q Ｎ ) to convert the query q ｎ The matrix Q=(q １ , ..., q Ｎ ) are created. Hereinafter, N queries q １ , ..., q Ｎ The matrix Q formed by the above is also referred to as a "query Q in matrix form."

[0038] The inference unit 204 receives a query Q in matrix form as an input, executes inference processing using a large-scale language model, and obtains a query result A in matrix form: A:=(a １ , ..., a Ｎ ) where the query result a ｎ is a query q ｎ At this time, the inference unit 204 performs the inference process using the large-scale language model to calculate the linear transformations f ｉ (X ｉ ) = W ｉ X ｉ is calculated by a processor specialized for parallel processing such as the GPU 107. ｉ (X ｉ ) = Wｉ X ｉ The matrix operation can be efficiently calculated by a processor specialized for parallel processing, such as the GPU 107.

[0039] The response control unit 205 receives the nth request req ｎ The sender information included in the nth request req ｎ The query q contained in ｎ Query result for a ｎ Response res containing ｎ In addition, the response control unit 205 creates each response res ｎ (n=1, . . . , N) is stored in the response temporary storage unit 302 .

[0040] The response sending unit 206 sends each response res stored in the response temporary storage unit 302 ｎ (n=1, . . . , N) to the client 20. That is, the response sending unit 206 sends the response res ｎ The response res is sent to the client 20 indicated by the sender information included in ｎ This sends a response res ｎ The client 20 receives the query result a ｎ can be obtained.

[0041] The request temporary storage unit 301 stores the request stored by the request receiving unit 201 .

[0042] The response temporary storage unit 302 stores the response stored by the response control unit 205 .

[0043] <Example of Operation of Server 10> <Example of Operation of Server 10 When Receiving a Request> Below, an example of operation of the server 10 when receiving a request transmitted from a client 20 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of operation of the server 10 when receiving a request. In the following, it is assumed that a request is transmitted from a certain client 20. Note that the following steps S101 to S102 are repeatedly executed each time a request is transmitted from the client 20.

[0044] The request receiving unit 201 receives a request sent from the client 20 (step S101).

[0045] The request receiving unit 201 stores the request received in step S101 in the request temporary storage unit 301 (step S102).

[0046] <<Example of Operation of Server 10 When Executing Inference Processing Using a Large-Scale Language Model>> An example of operation of the server 10 when executing inference processing using a large-scale language model will now be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of operation of the server 10 when executing inference processing using a large-scale language model. Note that the following steps S201 to S207 are repeatedly executed, for example, every time a predetermined time period (e.g., a few seconds) has elapsed, or every time a request is stored in the request temporary storage unit 301, etc.

[0047] The request control unit 202 determines whether an execution start condition is satisfied (step S201). The execution start condition is a condition for determining whether to start execution of inference processing using a large-scale language model. Various conditions can be used as the execution start condition, but for example, at least one of the following execution start conditions 1 to 4 can be used.

[0048] Execution start condition 1: N or more requests are stored in the request temporary storage unit 301.

[0049] Execution start condition 2: The number of requests stored in the request temporary storage unit 301 has reached a predetermined upper limit.

[0050] Execution start condition 3: A predetermined time has elapsed since the oldest request among the requests stored in the request temporary storage unit 301 was stored in the request temporary storage unit 301.

[0051] Execution start condition 4: The execution of the previous inference process using the large-scale language model has been completed, and it is now possible to execute the inference process using the large-scale language model.

[0052] However, the above-mentioned execution start conditions 1 to 4 are all examples, and conditions other than these may be used as the execution start conditions.

[0053] If it is not determined in step S201 that the execution start condition is satisfied, the request control unit 202 returns to step S201, whereby step S201 is repeatedly executed until the execution start condition is satisfied.

[0054] On the other hand, if it is determined in step S201 that the execution start condition is satisfied, the request control unit 202 retrieves N requests req from the request temporary storage unit 301. ｎ (n=1, . . . , N) (step S202). At this time, the request control unit 202 acquires N requests req according to a predetermined request acquisition standard. ｎ (n=1, ..., N). The request acquisition criteria are criteria for acquiring N requests from the request temporary storage unit 301. If the number of requests stored in the request temporary storage unit 301 is less than N, the request control unit 202 acquires all requests from the request temporary storage unit 301 (that is, if the number of requests stored in the request temporary storage unit 301 is N' (<N), it acquires N' requests). In the following, it is assumed that N requests req are acquired in step S202. ｎ The following description will be given on the assumption that (n=1, . . . , N) is acquired.

[0055] Various criteria can be used as the request acquisition criteria, but for example, any of the following request acquisition criteria 1 to 5 can be used.

[0056] Request acquisition standard 1: N requests are acquired randomly from the request temporary storage unit 301.

[0057] Request acquisition criterion 2: If a priority is included in a request, N requests are acquired in descending order of priority.

[0058] Request acquisition criterion 3: The shorter (or longer) the query length (i.e., the number of tokens that make up the query) included in the request, the higher the priority is set, and N requests are acquired in descending order of priority.

[0059] Request acquisition criterion 4: The closer the length of the query included in the request is to the average length of the queries, the higher (or lower) the priority is set, and N requests are acquired in descending order of priority.

[0060] Request acquisition criterion 5: A higher priority is assigned to a request that has been stored in the request temporary storage unit 301 for a longer period of time, and N requests are acquired in descending order of priority.

[0061] However, the above request acquisition criteria 1 to 5 are all examples, and criteria other than these may be used as the request acquisition criteria.

[0062] The conversion unit 203 converts the N requests req acquired in step S202 into ｎ (n=1, . . . , N) is converted into a matrix format (step S203). ｎ Let q be the query contained in ｎ The conversion unit 203 converts N queries q １ , ..., q Ｎ Let us consider a matrix form of the query Q=(q １ , ..., q Ｎ ) to

[0063] The inference unit 204 receives the matrix-form query Q obtained in step S203 and executes inference processing using a large-scale language model, and obtains a matrix-form query result A=(a １ , ..., a Ｎ ) included in the inference process using the large-scale language model (step S204). ｉ (X ｉ ) = W ｉ X ｉ is calculated by a processor specialized for parallel processing such as the GPU 107.

[0064] The response control unit 205 converts the matrix-format query result A obtained in step S204 into the original vector-format query result a １ , ..., a Ｎ (step S205).

[0065] The response control unit 205 receives the nth request req ｎ The sender information included in the nth request req ｎ The query q contained in ｎ Query result for a ｎ Response res containing ｎ is created (step S206).

[0066] The response control unit 205 receives the response res ｎ (n=1, . . . , N) is stored in the response temporary storage unit 302 (step S207).

[0067] <<Example of Operation of Server 10 When Sending a Response>> An example of operation of the server 10 when sending a response to the client 20 will now be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of operation of the server 10 when sending a response. Note that the following steps S301 to S302 are repeatedly executed, for example, until the response temporary storage unit 302 becomes empty (that is, until there are no responses stored in the response temporary storage unit 302).

[0068] The response sending unit 206 receives the response res from the response temporary storage unit 302.ｎ is acquired (step S301).

[0069] The response sending unit 206 sends the response res acquired in step S301. ｎ The response transmission unit 206 then transmits the response res ｎ The response res is sent to the client 20 indicated by the sender information included in ｎ Send.

[0070] <Modifications> Modifications of the above embodiment will be described below. Note that the following modifications can be combined with multiple modifications as appropriate, as long as they do not contradict each other.

[0071] Modification 1 In step S204 of FIG. 5, multiple inference processes using a large-scale language model with a matrix-format query as input may be executed in multiple threads. That is, M queries in matrix format may be input as Q （ｍ） (m=1, . . . , M), where M is an integer of 2 or more.

[0072] In this case, M threads are used to generate a matrix-form query Q （ｍ） Inference processing may be performed using a large-scale language model with the following as input.

[0073] Modification 2: When the server 10 has a plurality of processors specialized for parallel processing, the server 10 may have an inference unit 204 corresponding to each of the processors specialized for parallel processing. Hereinafter, the plurality of inference units 204 will be referred to as the inference units 204 , respectively. ｋ (k=1, . . . , K), where K is the number of inference units 204 (that is, the number of parallel processing specialized processors).

[0074] In this case, in step S202 of FIG. 5, each inference unit 204 ｋ For N ｋ requests are acquired from the request temporary storage unit 301. At this time, for each k=1, . . . , K, N queries with similar lengths are acquired. ｋrequests may be acquired because the execution time of inference processing by a large-scale language model that uses a matrix-format query as input is considered to depend on the maximum length of the column vectors that make up the matrix-format query (i.e., the original vector-format query).

[0075] Modification 3 In the above modification 2, if the server 10 is realized in an information processing system configured with a plurality of general-purpose servers, etc., a plurality of parallel processing specialized processors may be distributed and arranged among a plurality of general-purpose servers.

[0076] Variation 4 Although the above embodiment is directed to a large-scale language model, the above embodiment can also be applied to any machine learning model other than a large-scale language model (e.g., a language model other than a large-scale language model, a machine learning model that realizes a service called generative AI (Artificial Intelligence) or generative AI, etc.). For example, when a query is given by a prompt or the like to a machine learning model that realizes generative AI that generates content such as moving images, not just text, the above embodiment can be applied to multiplex queries. Note that a specific example of a machine learning model that realizes generative AI is a machine learning model that includes a Transformer, etc.

[0077] <Summary> As described above, when performing inference processing using a large-scale language model, the server 10 according to this embodiment converts multiple queries in vector format into matrix format and then performs inference processing using the matrix format queries as input. This allows multiple queries to be processed simultaneously using one thread of a parallel processing-specialized processor such as the GPU 107, thereby enabling efficient execution of inference processing using a large-scale language model. Therefore, even when the number of queries is high, inference processing using a large-scale language model can be performed without scaling out or up hardware resources.

[0078] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims.

[0079] The following supplementary items are further disclosed in relation to the above embodiments. (Supplementary Item 1) An information processing device including: a memory; and at least one processor connected to the memory, wherein the processor converts a plurality of vector-format queries for a large-scale language model into matrix-format queries, and performs inference processing for the large-scale language model using a parallel processing processor that performs matrix operations in parallel. (Supplementary Item 2) The information processing device according to Supplementary Item 1, wherein the processor acquires a plurality of requests from a storage unit that stores requests including vector-format queries for the large-scale language model based on predetermined criteria, and converts the plurality of queries into the matrix-format query using a query included in each of the acquired requests. (Supplementary Item 3) The information processing device according to Supplementary Item 2, wherein the criteria include acquiring the plurality of requests from the storage unit based on priority of the requests. (Supplementary Item 4) The information processing device according to Supplementary Item 3, wherein the priority is a priority included in the requests or a priority based on the length of the queries included in the requests. (Supplementary Item 5) The information processing device according to Supplementary Item 2, wherein the processor acquires the plurality of requests from the storage unit based on the criterion when a predetermined condition is satisfied. (Supplementary Item 6) The information processing device according to Supplementary Item 5, wherein the condition includes at least one of: the number of requests stored in the storage unit has reached a predetermined upper limit, a predetermined time has elapsed since the storage time of the oldest request among the requests stored in the storage unit, or the previous execution of inference processing for the large-scale language model has been completed, making it possible to execute the inference processing for the large-scale language model. (Supplementary Item 7) The information processing device according to Supplementary Item 1, wherein the processor converts a matrix-format query result obtained as a result of execution of inference processing for the large-scale language model using the matrix-format query as input into a vector-format query result, and transmits a response including the vector-format query result to a sender of a request that included a query corresponding to the query result.(Supplementary Item 8) A non-transitory storage medium storing a program executable by a computer to perform predetermined information processing, wherein the information processing includes converting a plurality of vector-format queries for a large-scale language model into matrix-format queries, and performing inference processing of the large-scale language model using a parallel processing processor that performs matrix operations in parallel, with the matrix-format queries as input.

[0080] [References] Reference 1: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, "Attention Is All You Need", arXiv:1706.03762 [cs.CL].

[0081] REFERENCE SIGNS LIST 1 System 10 Server 20 Client 30 Network 101 External I / F 101a Recording medium 102 Communication I / F 103 RAM 104 ROM 105 HDD 106 CPU 107 GPU 108 Bus 201 Request receiving unit 202 Request control unit 203 Conversion unit 204 Inference unit 205 Response control unit 206 Response sending unit 301 Request temporary storage unit 302 Response temporary storage unit

Claims

1. An information processing device having: a conversion unit that converts multiple vector-format queries for a large-scale language model into matrix-format queries; and an inference unit that uses a processor that performs matrix operations in parallel to perform inference processing on the large-scale language model using the matrix-format queries as input.

2. An information processing device as described in claim 1, further comprising a request control unit that acquires multiple requests from a memory unit that stores requests containing vector-format queries for the large-scale language model based on predetermined criteria, and wherein the conversion unit converts the multiple queries into the matrix-format query using a query contained in each of the multiple requests acquired by the request control unit.

3. The information processing device according to claim 2, wherein the criteria include retrieving the plurality of requests from the storage unit based on the priority of the requests.

4. An information processing method in which a computer executes a conversion procedure for converting multiple vector-format queries for a large-scale language model into matrix-format queries, and an inference procedure for executing inference processing of the large-scale language model using a processor that performs matrix operations in parallel, with the matrix-format queries as input.

5. A program that causes a computer to execute the following steps: a conversion procedure that converts multiple vector-format queries for a large-scale language model into matrix-format queries; and an inference procedure that uses a processor that performs matrix operations in parallel to perform inference processing on the large-scale language model using the matrix-format queries as input.

Images