Information processing method, information processing system, and information processing program

Abstract

To improve a processing speed in data generation using generation AI, and to improve a compression ratio in data accumulated in a storage area.SOLUTION: In the information processing method, an information processing system generates a feature amount of input data, and acquires recollection data corresponding to the feature amount from correspondence information between the feature amount and the recollection data of the input data. Then, the information process system inputs the obtained recall dataset to a generative AI, and obtains an answer dataset for the input dataset from the generative AI.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing method, an information processing system, and an information processing program. [Background technology]

[0002] In recent years, generative AI (Artificial Intelligence) such as LLMs (Large Language Models) has become widespread. Generative AI can improve the accuracy of answers by generating answers based on a prompt describing a user's question and input data based on search results of external information related to the prompt.

[0003] When the data volume used as external information is large, it is necessary to reduce the data volume when storing it in a storage area. Therefore, the conventional technology disclosed in Patent Document 1 reduces the data volume of compressed data stored in a storage area by compressing an image to a data volume according to the importance of each area of ​​the image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-145701 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above-mentioned conventional technology, the intermediate representation data used to generate input data to be input to the generation AI is generated each time using a neural network, so there was room for improvement in the processing speed of data generation using the generation AI.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to improve the processing speed of data generation using generation AI. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, one aspect of the present invention is an information processing method executed by an information processing system having a processor and a memory, characterized in that the processor comprises the steps of accepting input data, generating features of the input data, obtaining recall data corresponding to the features from correspondence information between the features and recall data of the input data, inputting the obtained recall data to a generation AI (Artificial Intelligence), and obtaining response data to the input data from the generation AI. [Effects of the Invention]

[0008] According to the present invention, it is possible to improve the processing speed of data generation using generation AI and improve the compression rate of data stored in a storage area. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the configuration of a computer according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing an overview of data accumulation processing and data generation processing in the computer according to the first embodiment. [Figure 3] FIG. 2 is a diagram showing the configuration of a recall data table according to the first embodiment. [Figure 4] FIG. 10 is a diagram showing an overview of data accumulation processing and data generation processing in a computer according to the second embodiment. [Figure 5] FIG. 10 is a diagram showing an overview of model learning processing in a computer according to the second embodiment. [Figure 6] 10 is a flowchart showing processing performed when data is accumulated in a computer according to the second embodiment. [Figure 7] 10 is a flowchart showing a process performed when generating data in a computer according to the second embodiment. [Figure 8] FIG. 11 is a diagram showing an overview of data accumulation processing and data generation processing in a computer according to a third embodiment. [Figure 9]FIG. 10 is a diagram showing an overview of learning processing in a computer according to a third embodiment. [Figure 10] 10 is a flowchart showing a data accumulation process according to the third embodiment. [Figure 11] 11 is a flowchart showing a process at the time of data generation according to the third embodiment. [Figure 12] FIG. 10 is a diagram showing an outline of a neural network model according to a fourth embodiment. [Figure 13] FIG. 10 is a diagram showing an outline of Causal Linear according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] In the following description, an "interface apparatus" may refer to one or more interface devices, which may be at least one of the following: One or more I / O (Input / Output) interface devices. The I / O (Input / Output) interface devices are interface devices for at least one of the I / O device and a remote display computer. The I / O interface device for the display computer may be a communications interface device. The at least one I / O device may be a user interface device, for example, either an input device such as a keyboard and a pointing device, or an output device such as a display device. One or more communication interface devices. The one or more communication interface devices may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)) or two or more heterogeneous communication interface devices (e.g., an NIC and an HBA (Host Bus Adapter)).

[0011] In the following description, "memory" refers to one or more memory devices, typically a primary storage device. At least one of the memory devices may be a volatile memory device or a non-volatile memory device.

[0012] In the following description, a "persistent storage device" refers to one or more persistent storage devices. A persistent storage device is typically a non-volatile storage device (e.g., an auxiliary storage device), and specifically, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0013] In the following description, a "storage device" may be a physical storage device such as a persistent storage device, or a logical storage device associated with a physical storage device.

[0014] In the following description, a "processor" refers to one or more processor devices. The at least one processor device is typically a microprocessor device such as a CPU (Central Processing Unit), but may also be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may also be a processor core. The at least one processor device may also be a processor device in a broader sense, such as a hardware circuit (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0015] In the following description, information that provides an output for an input may be described using expressions such as "xxx table," but this information may be data of any structure, or may be a learning model such as a neural network that generates an output for an input. Therefore, an "xxx table" may be referred to as "xxx information." In the following description, the structure of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0016] In the following description, functions may be described using the expressions "xxx device" and "xxx unit." A function may be realized by one or more computer programs being executed by a processor, or by one or more hardware circuits (e.g., FPGA or ASIC). When a function is realized by a program being executed by a processor, the specified processing is performed using a storage device and / or an interface device, etc., as appropriate, and therefore the function may be considered to be at least a part of the processor. Processing described using a function as the subject may be processing performed by a processor or a device having the processor. A program may be installed from a program source. The program source may be, for example, a computer from which the program is distributed or a computer-readable recording medium (e.g., a non-transitory recording medium). The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

[0017] In the following embodiments, the image data may be either a still image or a moving image.

[0018] [Embodiment 1] (Configuration of the computer 10 according to the first embodiment) FIG. 1 is a diagram showing the configuration of a computer 10 according to the first embodiment. The computer 10 is an example of an information processing system. Processing by the computer 10 is executed by a processor 14 and a parallel processing device 15, which will be described later. The computer 10 can process multiple batches in parallel. The computer 10 is a computer or storage device in an on-premise environment or a cloud environment. The input device 20 is a computer in an on-premise environment or a cloud environment. The persistent storage device 12 may also be storage in an on-premise environment or a cloud environment that is communicatively connected to the computer 10.

[0019] The computer 10 has interfaces 11a and 11b (an example of an interface device), a persistent storage device 12, a memory 13, a processor 14, a parallel processing device 15, and a bus 16 that connects these. The interfaces 11a and 11b, the persistent storage device 12, and the parallel processing device 15 are connected to the processor 14 via the bus 16, for example, so as to be able to communicate with each other.

[0020] The interface 11a is connected to an input device 20. The input device 20 inputs data to the computer 10. The input device 20 may be a sensor device (for example, an optical camera or a gravity sensor), a portable storage medium, or another computer.

[0021] The interface 11b is connected to the terminal 30. The terminal 30 inputs a prompt to the computer 10 when the user makes a query to the generated AI (not shown). The computer 10 generates an answer based on the prompt and outputs it to the terminal 30. A general-purpose computer can be used as the terminal 30.

[0022] Data to be compressed is input to the parallel processing device 15 from the input device 20 via the interface 11b, either via the processor 14 or not. In this embodiment, the data to be compressed is image data representing an image (still image), but any type of data may be used. The parallel processing device 15 has a memory 151 and multiple cores 152.

[0023] The memory 13 stores computer programs executed by the processor 14 and data input and output by the processor 14.

[0024] The processor 14 executes at least a part of the processing executed by the computer 10 by reading and executing a program from the memory 13. The processor 14 and the parallel processing device 15 are realized as an elicitor 14a and a generator 14b by executing the program. The processing of the elicitor 14a and the generator 14b will be described in detail later.

[0025] For example, a system may be configured using a plurality of computers 10. Furthermore, when configuring a system using a plurality of computers 10, the computers 10 including some of the persistent storage devices 12 may be configured as a storage system, and the storage system may be used as the storage media (such as the persistent storage devices 12 and memory 13) of the remaining computers 10. Furthermore, some of the processing described in the following embodiments may be executed by the parallel processing device 15 or processor 14 of the computer 10 on the storage system side, thereby improving efficiency by centralizing and executing processing close to the data storage destination.

[0026] (Outline of Data Accumulation Processing and Data Creation Processing According to the First Embodiment) FIG. 2 is a diagram showing an overview of the data accumulation process and the data generation process in the computer 10 according to the first embodiment.

[0027] (Outline of Data Accumulation Processing According to the First Embodiment) First, with reference to FIG. 2(a), the process performed by the image generator 14a according to the first embodiment when storing data will be described.

[0028] In the image generator 14a, the feature generation processing unit 14a1 generates feature D14a1 from original data (image data) input from the input device 20. The intermediate representation generation model processing unit 14a2 has an intermediate representation generation model, and uses the intermediate representation generation model to generate intermediate representation data D14a2 from the image data input from the input device 20. The intermediate representation generation model held by the intermediate representation generation model processing unit 14a2 is, for example, a neural network model.

[0029] When stored in the recalled data table T1, the feature amount D14a1 and the intermediate representation data D14a2 are compressed (entropy coded). The feature amount D14a1 and the intermediate representation data D14a2 of the compressed image data are associated with each other for the same image data and recorded in the recalled data table T1 stored in the memory 13, 151. The intermediate representation data D14a2 is also saved in a memory area of ​​the storage.

[0030] (Outline of Data Generation Processing According to the First Embodiment) Next, with reference to FIG. 2(b), a data generation process in the image generator 14a and the generator 14b according to the first embodiment will be described.

[0031] First, in the recaller 14a, the feature generation processing unit 14a1 receives the original data (image data) or the intermediate layer data as input, and generates the feature D14a1 from the original data (image data) or the intermediate layer data.

[0032] Next, the recaller 14a (or the generator 14b) refers to the recalled data table T1 based on the feature amount D14a1, and acquires the corresponding intermediate representation data D14a2.

[0033] When the later-described generative model processing unit 14b1 has an input layer, one or more intermediate layers, and an output layer, the feature generation processing unit 14a1 takes as input data the original data (image data) for the input layer, and as input data for the intermediate and output layers the data generated by the previous input layer or intermediate layer (intermediate layer data).

[0034] Next, in the generator 14b, the generative model processing unit 14b1 acquires intermediate representation data D14a2 by referring to the recall data table T1. The generative model processing unit 14b1 then inputs the acquired intermediate representation data D14a2, a prompt, and intermediate layer data to the generative model (generative AI), and acquires generated data generated by the generative model (generative AI). The generative model processing unit 14b1 has a generative model (generative AI) such as an LLM, at least a portion of which is configured using a neural network model. If the generated data is the output of an intermediate layer of the generative model (generative AI), it becomes intermediate layer data, which is the input for processing the next intermediate layer. However, if it is the output of the final layer of the generative model (generative AI), it becomes answer data for the generative model (generative AI).

[0035] The generative model processing unit 14b1 may be provided in another computer that is different from the computer 10 and can communicate with the computer 10 via a network, instead of the generator 14b.

[0036] (Configuration of Recalled Data Table T1 According to First Embodiment) 3 is a diagram showing the configuration of the recall data table T1 according to the first embodiment. The recall data table T1 is stored in the memory 13, 151. Cosine similarity may be used as the similarity between the features. In addition, to speed up the determination of the similarity, LSH (Locality Sensitive Hashing) or a match determination based on a quantized value of the feature D14a1 may be used.

[0037] The recall data table T1 has columns for "features" and "compressed data (intermediate representation data)." "Features" are image data features expressed as consecutive natural numbers, etc. The intermediate representation data is compressed data obtained by compressing the original data. The image data of the original data typically has a size of C (number of channels) x N (length) (N is undefined).

[0038] The recall data table T1 is a table for outputting "compressed data (intermediate representation data)" corresponding to the "feature" that has the highest similarity to the input feature. Instead of the intermediate representation data, the "compressed data (intermediate representation data)" may store a pointer that indicates a storage location in storage where the entity of the intermediate representation data is stored.

[0039] (Effects of the First Embodiment) In the first embodiment, recalled data corresponding to the feature amounts is obtained from a recalled data table or the like that stores correspondence information between the feature amounts of the input data and the recalled data of the input data, and inputs the data to the generation AI, and response data to the input data is obtained from the generation AI. Therefore, according to the first embodiment, recalled data such as intermediate representation data used when generating input data for the generation AI is converted in advance, eliminating the need to generate it each time, and thus reducing the processing speed of data generation.

[0040] [Embodiment 2] In the second embodiment, the differences from the first embodiment will be mainly described, and overlapping descriptions will be omitted.

[0041] (Outline of Data Accumulation Processing and Data Creation Processing According to Second Embodiment) FIG. 4 is a diagram showing an outline of the data accumulation process and the data generation process in the computer 10B according to the second embodiment.

[0042] (Outline of Data Accumulation Processing According to Second Embodiment) First, with reference to FIG. 4(a), a data storage process in the recaller 14Ba according to the second embodiment will be described.

[0043] In the image generator 14Ba, the intermediate representation generation model processing unit 14a2 generates intermediate representation data D14a2 from original data (image data) input from the input device 20. The feature generation processing unit 14a1 has an intermediate representation generation model, and uses the intermediate representation generation model to generate feature amounts D14a1 from the intermediate representation data D14a2 generated by the intermediate representation generation model processing unit 14a2. The intermediate representation generation model held by the intermediate representation generation model processing unit 14a2 is, for example, a neural network model.

[0044] On the other hand, the auxiliary input conversion processing unit 14a3 converts the intermediate representation data D14a2 generated by the intermediate representation generation model processing unit 14a2 to generate auxiliary input data D14a3. The auxiliary input data D14a3 is data that is input together with a prompt to the generative model (generative AI) of the generative model processing unit 14b1 so that the generation data generated by the generative model (generative AI) has high accuracy as answer data.

[0045] The entropy predictor 14a4 predicts the probability distribution f of each symbol, which is a data unit of compression, for the auxiliary input data D14a3 using predictions such as an autoregressive model. The entropy predictor 14a4 then calculates the cumulative distribution function CDF (Cumulative Distribution Function) of the probability distribution f. The probability distribution f and cumulative distribution function CDF for each symbol are called the predicted probability (CDF, ​​f) of each symbol.

[0046] The entropy encoder 14a5 encodes each symbol based on the auxiliary input data D14a3 and the predicted probability (CDF, ​​f) of each symbol from the entropy predictor 14a4, and outputs compressed data D14a5. The compressed data D14a5 is stored in a storage area.

[0047] The feature D14a1 is compressed (entropy coded) when recorded in the recall data table T2. The feature D14a1 of the compressed image data and the auxiliary input data D14a3 of the same image data are associated with each other and recorded in the recall data table T2 stored in the memory 13, 151. The recall data table T2 is obtained by replacing the "compressed data (intermediate representation data)" of the recall data table T1 with "compressed data (input auxiliary data)." The intermediate representation data D14a2 is saved in a memory area of ​​the storage.

[0048] (Outline of Data Generation Processing According to Second Embodiment) Next, with reference to FIG. 4(b), a data generation process in the computer 10B according to the second embodiment will be described.

[0049] First, in the recaller 14Ba, the feature generation processing unit 14a1 generates a feature D14a1 from input data (such as a prompt input to a generative model included in the generative model processing unit 14b1).

[0050] Next, the recaller 14Ba (or the generator 14Bb) refers to the recall data table T2 based on the feature amount D14a1, and acquires the corresponding auxiliary input data D14a3.

[0051] Next, in the generator 14Cb, the generative model processing unit 14b1 inputs the auxiliary input data D14a3 and the prompt acquired by referring to the recall data table T2 to the generative model (generative AI). Then, the generative model processing unit 14b1 acquires the generated data (answer data) generated by the generative model (generative AI).

[0052] (Outline of Model Learning Process According to Second Embodiment) FIG. 5 is a diagram showing an outline of the model learning process in the computer 10B according to the second embodiment.

[0053] In the recaller 14Ca, the intermediate representation generation model processing unit 14a2 generates intermediate representation data D14a2 from input original data (image data). The auxiliary input conversion processing unit 14a3 converts the intermediate representation data D14a2 generated by the intermediate representation generation model processing unit 14a2 into auxiliary input data D14a3.

[0054] The entropy predictor 14a4 and the generative model processing unit 14b1 of the generator 14Bb learn the intermediate representation data D14a2 by backpropagation. Through learning, the generative model processing unit 14b1 generates or updates a generative model (generative AI) that the generative model processing unit 14b1 has.

[0055] (Data accumulation processing according to the second embodiment) 6 is a flowchart showing the data storage process in the computer 10B according to the second embodiment. The data storage process corresponds to FIG. 4(a). In the data storage process, steps S11 to S15 are executed for each input image data.

[0056] First, in step S11, the intermediate representation generation model processing unit 14a2 of the recaller 14Ba generates intermediate representation data D14a2. Next, in step S12, the auxiliary input conversion processing unit 14a3 of the recaller 14Ba converts the intermediate representation data D14a2 generated in step S11 into auxiliary input data D14a3.

[0057] Next, in step S13, the feature generation processing unit 14a1 of the recaller 14Ba generates feature D14a1 from the intermediate representation data D14a2 generated in step S11. Next, in step S14, the entropy encoder 14a5 of the recaller 14Ba encodes (compresses) the auxiliary input data D14a3 converted in step S12 to generate compressed data D14a5. Next, in step S15, the recaller 14B associates the feature D14a1 generated in step S13 with the compressed data D14a5 compressed in step S14 and stores them in the recall data table T2.

[0058] (Data Generation Processing According to the Second Embodiment) 7 is a flowchart showing the data generation process in the computer 10B according to embodiment 2. The data generation process corresponds to FIG.

[0059] First, in step S21, the feature generation processing unit 14a1 of the recaller 14Ba generates feature D14a1 of the input data (such as a prompt input to the generative model of the generative model processing unit 14b1) from the input data. Next, in step S22, the recaller 14Ba (or the generator 14Bb) refers to the recall data table T2 based on the feature D14a1 generated in step S21, and acquires corresponding auxiliary input data D14a3.

[0060] Next, in step S23, the recaller 14Ba (or the generator 14Bb) entropy-decodes the auxiliary input data D14a3 acquired in step S22. Next, in step S24, the generator 14Bb creates input data for the generative model (generative AI) from the auxiliary input data D14a3 entropy-decoded in step S23 and the prompt entered by the user.

[0061] Next, in step S25, the generative model processing unit 14b1 of the generator 14Bb inputs the input data created in step S24 to its own generative model (generative AI) and acquires the answer data generated by the generative model (generative AI).

[0062] The above-described second embodiment is suitable for on-demand processing of video data and the like.

[0063] (Effects of the second embodiment) In the second embodiment, a prompt to ask a question to the generation AI is used as input data, and compressed auxiliary input data based on this input data is used as recall data. Intermediate representation data is then generated from the prompt to be input to the generation AI, features are generated from this intermediate representation data, the intermediate representation data is converted into auxiliary input data, and the features and auxiliary input data are associated to generate correspondence information. Therefore, according to the second embodiment, when the generation AI is used, the auxiliary input data is compressed and stored when data is stored, thereby reducing the amount of data stored in the memory area.

[0064] In addition, in the second embodiment, intermediate representation data is generated from prompts to ask questions to the generation AI, the intermediate representation data is converted into auxiliary input data, and the auxiliary input data is learned to generate a generative model possessed by the generation AI. Therefore, according to the second embodiment, the generative model is learned using the auxiliary input data, so that the generative model can be made compact.

[0065] In the second embodiment, the auxiliary input data is trained to generate an entropy predictor corresponding to the intermediate representation data, the auxiliary input data is compressed using the entropy predictor, and the feature values ​​are associated with the compressed auxiliary input data to generate correspondence information (recalled data table). Thus, according to the second embodiment, the data volume of the recalled data table can be reduced.

[0066] Furthermore, in the second embodiment, the feature quantities and intermediate representation data are generated using a neural network model. Therefore, according to the second embodiment, correspondence information between the feature quantities and auxiliary input data is calculated in advance, and the auxiliary input data is obtained from the correspondence information based on the feature quantities, eliminating the need to generate intermediate representation data each time to calculate auxiliary input data. This makes it possible to more significantly suppress a decrease in the processing speed of data generation.

[0067] [Embodiment 3] In the third embodiment, the differences from the first and second embodiments will be mainly described, and overlapping descriptions will be omitted.

[0068] (Outline of Data Accumulation Processing and Data Creation Processing in Computer 10C According to Third Embodiment) FIG. 8 is a diagram showing an outline of the data accumulation process and the data generation process in the computer 10C according to the third embodiment.

[0069] (Outline of Data Accumulation Processing According to the Third Embodiment) First, referring to Fig. 8(a), a description will be given of the data storage process in the computer 10C according to the third embodiment. The data storage process is executed for all natural numbers that can be expressed in k bits.

[0070] The image generator 14Ca receives input of image data and a k-bit natural number N, which is the number of symbols for entropy coding. The image generator 14Ca generates a feature quantity (compression) D14a6 of the image data from the image data.

[0071] The entropy predictor 14a4 predicts the predicted probability (CDF, ​​f) of each symbol, which is a data unit of compression, from the natural number N and the feature amount (compressed) D14a6.

[0072] The entropy decoder 14a6 expands the feature (compressed) D14a6 based on the predicted probability (CDF, ​​f) of each symbol obtained by the entropy predictor 14a4, and obtains the feature D14a1.

[0073] The input data converter 14a7 converts the feature quantity D14a1 expanded by the entropy decoder 14a6 into input conversion data D14a7, which is in the input format of the intermediate representation generation model processing unit 14a2.

[0074] The intermediate representation generation model processing unit 14a2 has an intermediate representation generation model, and uses the intermediate representation generation model to generate intermediate representation data D14a2 from input converted data D14a7 input from the input data converter 14a7. The intermediate representation generation model that the intermediate representation generation model processing unit 14a2 has is, for example, a neural network model.

[0075] When stored in the recall data table T3, the intermediate representation data D14a2 may be compressed (entropy coded). The feature (compressed) D14a6 and the intermediate representation data D14a2 are associated with each other and recorded in the recall data table T3 stored in the memory 13, 151. The recall data table T3 is obtained by replacing the "feature" in the recall data table T1 with "feature (compressed)." The intermediate representation data D14a2 is saved in a memory area of ​​the storage.

[0076] (Outline of Data Generation Processing According to the Third Embodiment) Next, with reference to FIG. 8(b), a data generation process in the computer 10C according to the third embodiment will be described.

[0077] The feature generation processing unit 14a1 receives the original data (image data) or the intermediate layer data as input data, and generates the feature (compressed) D14a6 from the original data (image data) or the intermediate layer data.

[0078] The entropy predictor 14a4 predicts the predicted probability (CDF, ​​f) of each symbol, which is a data unit of compression, for the feature D14a1 using prediction based on an autoregressive model, etc. The entropy encoder 14a5 encodes each symbol based on the feature D14a1 and the predicted probability (CDF, ​​f) of each symbol from the entropy predictor 14a4, and outputs the compressed feature D14a6.

[0079] Next, the recaller 14Ca (or the generator 14Cb) refers to the recalled data table T3 based on the feature amount (compressed) D14a6, and acquires the corresponding intermediate representation data D14a2.

[0080] Next, in the generator 14Cb, the generative model processing unit 14b1 inputs the intermediate representation data D14a2 and the prompt acquired by referring to the recall data table T3 to the generative model (generative AI). The generative model processing unit 14b1 then acquires the generated data (next-layer intermediate layer data or answer data) generated by the generative model (generative AI). At this time, the generative model processing unit 14b1 calculates part of the matrix of the neural model in the generative model (generative AI).

[0081] (Outline of Model Learning Process According to Third Embodiment) FIG. 9 is a diagram showing an outline of a model learning process in the computer 10C according to the third embodiment.

[0082] The feature generation processing unit 14a1 generates feature D14a1 from input data (image data). The input data converter 14a7 converts the feature D14a1 into input converted data D14a7. The intermediate representation generation model processing unit 14a2 receives the input converted data D14a7 as input and generates intermediate representation data D14a2.

[0083] The entropy predictor 14a4 of the recaller 14Ca and the generative model processing unit 14b1 of the generator 14Cb learn the feature quantity D14a1 and the intermediate representation data D14a2 by backpropagation, respectively. Through learning, the generative model processing unit 14b1 generates or updates a generative model (generative AI) that the generative model processing unit 14b1 possesses.

[0084] (Data accumulation processing according to the third embodiment) Fig. 10 is a flowchart showing the data storage process according to the third embodiment. The data storage process corresponds to Fig. 8(a). The data storage process is executed for all natural numbers expressed in k bits. In the data storage process, steps S31 to S14 are executed for each input feature (compressed).

[0085] First, in step S31, the entropy decoder 14a6 of the recaller 14Ca expands the feature (compressed) D14a6 to generate the feature D14a1. Next, in step S22, the input data converter 14a7 converts the feature D14a1 generated in step S31 into input converted data D14a7.

[0086] Next, in step S33, the intermediate representation generation model processing unit 14a2 of the recaller 14Ba receives the input converted data D14a7 generated in step S32 as input and generates intermediate representation data D14a2. Next, in step S34, the recaller 14B associates the input feature (compressed) D14a6 with the intermediate representation data D14a2 generated in step S33 and stores them in the recalled data table T3.

[0087] (Data Generation Processing According to the Third Embodiment) 11 is a flowchart showing the data generation process in the computer 10C according to the third embodiment. The data generation process corresponds to FIG. 8(b). Steps S41 to S45 of the data generation process are executed for each layer of the generative model processing unit 14b1: the input layer, one or more intermediate layers, and the output layer.

[0088] First, in step S41, the feature generation processing unit 14a1 of the recaller 14Ca generates a feature (compressed) D14a6 of the image data from the original image data (image data) that is input data, together with the entropy predictor 14a4 and the entropy encoder 14a5. In step S41, the input layer uses the original data (image data) as input data, and the intermediate layer or output layer uses data generated by the previous input layer or intermediate layer (intermediate layer data) as input data.

[0089] Next, in step S42, the recaller 14Ba determines whether the feature (compressed) D14a6 generated in step S41 is equal to or less than a predetermined value (k bits). If the feature (compressed) D14a6 is equal to or less than the predetermined value (YES in step S42), the recaller 14Ba proceeds to step S43, and if it is greater than the predetermined value (NO in step S42), the recaller 14Ba proceeds to step S46.

[0090] In step S43, the feature generation processing unit 14a1 refers to the recalled data table T3 and acquires the intermediate representation data D14a2 corresponding to the feature (compressed) D14a6.

[0091] Next, in step S44, the generative model processing unit 14b1 of the generator 14Cb inputs the intermediate representation data D14a2 acquired in step S43 into the generative model possessed by the generative model processing unit 14b1 to generate generated data. Next, in step S45, the recaller 14Ca (or the generator 14Cb) determines whether processing of all target layers (input layer, intermediate layer, and output layer) has been executed. If processing of all target layers has been executed (step S45 YES), the recaller 14Ca (or the generator 14Cb) ends the data generation processing. On the other hand, if there is a layer for which processing has not been executed (step S45 NO), the recaller 14Ca (or the generator 14Cb) returns the processing to step S41.

[0092] In step S46, the input data converter 14a7 of the recaller 14Ca converts the feature quantity D14a1 decompressed by the entropy decoder 14a6 into input transformed data D14a7. Next, in step S47, the intermediate representation generation model processing unit 14a2 of the recaller 14Ca generates intermediate representation data D14a2 from the input transformed data D14a7 converted in step S46. When step S47 is completed, the process proceeds to step S44.

[0093] 11, when the size of the feature (compressed) D14a6 is k bits or less, the intermediate representation data D14a2 is acquired by referring to the recall data table T3. At this time, a high-speed memory of the memories 13 and 151 may be used. On the other hand, when the size of the feature (compressed) D14a6 generated in step S41 is larger than k bits, the intermediate representation generation model processing unit 14b2 generates the intermediate representation data D14a2.

[0094] For efficient use of the memories 13 and 151, the persistent storage device 12 may be used as a working memory when the size of the feature (compressed) D14a6 is greater than k bits, and the memories 13 and 151 may be used as working memory when the size of the feature (compressed) D14a6 is k bits or less.

[0095] The above-described third embodiment is suitable for batch processing of stored data.

[0096] (Effects of the third embodiment) In the third embodiment, image data to be used to query the generation AI is used as input data, and compressed data of intermediate representation data based on this image data is used as recall data. Features are then generated from the image data to be input to the generation AI, and feature (compressed) features are generated from these feature values. Correspondence information is then generated by associating the feature (compressed) with the intermediate representation data. Therefore, according to the third embodiment, when the generation AI is used, the intermediate representation data is compressed and stored when data is stored, thereby reducing the amount of data stored in the memory area.

[0097] In addition, in the third embodiment, features are generated from image data, these features are converted into input converted data, intermediate representation data is generated from the input converted data, and this intermediate representation data is learned to generate a generative model possessed by the generative AI. Therefore, according to the third embodiment, the generative model is learned using the intermediate representation data, so that the generative model can be made compact.

[0098] In the third embodiment, the feature is learned and an entropy predictor corresponding to the feature is generated. The feature is then compressed using the entropy predictor, the feature (compressed) is decompressed, and the feature (compressed) is associated with compressed data of the intermediate representation data to generate correspondence information (recalled data table). Therefore, according to the third embodiment, the data volume of the recalled data table can be reduced.

[0099] Furthermore, in the third embodiment, the feature amounts and intermediate representation data are generated using a neural network model. Therefore, according to the third embodiment, correspondence information between the feature amounts (compressed) and the intermediate representation data is calculated in advance, and the intermediate representation data is obtained from the correspondence information based on the feature amounts (calculated), eliminating the need to generate intermediate representation data each time. This makes it possible to more significantly suppress a decrease in the processing speed of data generation.

[0100] In addition, in the third embodiment, depending on the feature (compressed) size, the process switches between generating intermediate representation data to be input to the intermediate layer of the generative model and inputting it to the generative model, or generating input data to be input to the generative model and inputting it to the generative model. Therefore, according to the third embodiment, when the feature (compressed) is small, below a threshold, the recall data table is referenced to acquire intermediate representation data. On the other hand, when the feature (compressed) exceeds the threshold and a certain amount is accumulated, intermediate representation data is generated using a neural network. This can improve the processing speed of data generation while suppressing a decrease in the quality accuracy of the generated data. Furthermore, since the recall process only checks whether the feature (compressed) values ​​match, comparison with multiple values ​​using cosine similarity or the like is unnecessary, thereby speeding up the recall process. Furthermore, since the method using feature (compressed) uses values ​​after entropy coding, the spatial density of the feature is considered to be higher than methods using LSH or quantized values ​​of the feature, and more data may be indexed efficiently.

[0101] [Embodiment 4] In the fourth embodiment, differences from the first, second, and third embodiments will be mainly described, and redundant description will be omitted. Note that in the fourth embodiment, the configuration and processing of a neural network model will be described as a specific implementation form of the feature generation processing unit 14a1, the intermediate representation generation model processing unit 14a2, the auxiliary input conversion processing unit 14a3, the entropy predictor 14a4, the input data converter 14a7, the generation model processing unit 14b1, and the like described in the first, second, and third embodiments.

[0102] (Outline of neural network model according to embodiment 4) FIG. 12 shows an example of a neural network model implementation in the feature generation processing unit 14a1, intermediate representation generation model processing unit 14a2, auxiliary input conversion processing unit 14a3, entropy predictor 14a4, input data converter 14a7, and generative model processing unit 14b1 described in the first, second, and third embodiments. The tokenizer 51 tokenizes character string data. It receives input data [B,P] (hereinafter, for example, [X,Y,Z] represents a rank-3 tensor with shape X,Y,Z. The data formats before and after each process are also described in the figure using a similar notation.) and outputs a one-hot vector [B,N,T]. Here, B represents the number of batches, P represents the number of input characters, N represents the number of tokens, and T represents the number of token types. The embedding 52 converts the tokenized data into a tensor data size appropriate for subsequent processing. It receives [B,N,T] as input and outputs [B,N,C]. Here, C is the channel size (also called the hidden dimension size). In this example, a case where the input data is a character string is described, but if image data is input, the tokenizer 51 and embedding 52 may be replaced with a process of patching into a token format by convolution processing.

[0103] Next, the scale-down block 53 (53A, 53B) is a processing block that receives [B, N, C] as input and outputs [B, N / 2, C]. The scale-down block 53 may be configured by concatenating multiple blocks (D blocks in the figure). When D blocks are concatenated, the final output is [B, N / 2^(D-1), C]. Here, there is a number of groups G for each block. G may be set to 2^(D-1). G represents the number of groups in the channel dimension in the input data of that block, and each block may independently process the number of groups G in the channel dimension. This will be described more specifically with reference to FIG. 13.

[0104] The Scale Down Block 53 is composed of multiple processes (processes from Action 531 to Down 536). Normalization 531 and 533 are processes for normalizing input. Normalization may be performed in the channel direction. However, the channels may be divided into groups G in the block, and normalization may be performed for each divided group of the channels. Attention 532 performs self-attention processing on the data. Self-attention may be performed by outputting data Q, data K, and data V using three Causal Linears 581, 582, and 583, processing the data using a multi-head Scaled Dot-Product Attention 584 having heads for the number of groups G, and processing the results using Causal Linear 585. Furthermore, when the target process is a decoder (e.g., when used as the entropy predictor 14a4 or the generative model processing unit 14b1), a mask may be applied to the attention to perform prediction processing, and only past data may be referenced in the direction of the token sequence. The Feed Forward 534 is a process that is configured by, for example, two Causal Linears 571 and 573 and an activation function 572 .

[0105] As shown in FIG. 12, each of the above processes may be implemented as a residual network by inserting a data path that bypasses the process. Split 535 is a shortcut path from a scale-down block 53 (e.g., 53A) to a corresponding scale-up block 54 (e.g., 54A). It splits the input data in half along the channel dimension and sends it to the corresponding scale-up block 54. This shortcut path is intended to achieve the same effect as a residual network and, when the data granularity (token direction) is coarsened by the scale-down block 53, retain fine-grained data information and proceed with processing, thereby improving the overall accuracy of the neural network. Here, if G is greater than 1, the data may be divided in half for each group along the channel dimension, maintaining the relationship between the number of groups. Next, Down 536 may take, for example, [B, N, C / 2] as input and convert two pieces of data adjacent along the token dimension in the channel direction, resulting in [B, N / 2, C] as output.

[0106] Next, the Scale Up Block 54 (54A, 54B) is a processing block that takes [B, N / 2, C] as input and outputs [B, N, C]. Hereinafter, the differences with the Scale Down Block 53 will be mainly explained. First, Up 546 is the reverse process of Down 536. For example, it may take [B, N / 2, C] as input and convert two data of the same group in the channel direction into token dimensions to output [B, N, C / 2]. Cat 545 is a process that concatenates data received from the corresponding Scale Down Block 53 in the channel direction.

[0107] Next, Linear55 is a layer using linear matrix operations with [B,N,C] as input and [B,N,T] as output. Softmax56 calculates Softmax with [B,N,T] as input and outputs [B,N,T] as the occurrence probability of each token. In the case of an encoder, Softmax56 is not necessary, and the output size of Linear55 may be changed as appropriate.

[0108] In the above-described neural network configuration, the accuracy may be improved by appropriately inserting the Scale Down Block 53 excluding the Split 535 and Down 536 to create a model with more parameters.

[0109] Furthermore, in the case of an encoder (such as when used as the feature generation processing unit 14a1, the intermediate representation generation model processing unit 14a2, the auxiliary input conversion processing unit 14a3, or the input data converter 14a7), the output data may be quantized. The purpose of quantization may be to perform processing in the entropy encoder 14a5 or the entropy predictor 14a4, or to reduce the amount of data. Furthermore, a neural network may be inserted before or after the encoder to use a pre-trained model or to reduce the amount of data.

[0110] (Outline of Causal Linear according to the fourth embodiment) Figure 13 shows an example of implementation of Causal Linear61 (571, 573, 581, 582, 583, 585 in Figure 12), which is a component of some of the neural network in Figure 12. The processing example in this figure explains an example in which the number of groups G is 4.

[0111] In the example of Figure 13, Causal Linear 61 takes [B, S, Cin] as input and [B, S, Cout] as output. The hidden dimension of the input data is divided into G parts and managed as shown by the numbers in the figure (for example, for data 611, 0, 1, 2, and 3 are identifiers indicating groups corresponding to the token dimension length N). Causal Linear 61 first executes (1) an expansion process for the input data. This process replicates each piece of data in the sequence dimension (611, 612, 613, etc.) into the number of groups G while shifting the sequence by a fixed length.

[0112] For example, the first data 611 of the sequence is duplicated into G groups (data 621, 622, 623, 624) using padding "p" as shown in FIG. 13, with a shift. This duplication may be implemented by copying in memory, or may be implemented by duplicating only the references, thereby reducing memory usage and memory transfer volume. Next, the Causal Linear 61 (2) performs weight multiplication. A matrix multiplication operation is performed on each of the divided data (e.g., 621, 622, 623, 624) using weights (e.g., data 631, 632, 633, 634) divided by the number of groups. As a result, outputs divided for each group are obtained, and these are combined to form the final output. The Causal Linear 61 may also perform bias processing (e.g., weight addition) on the output.

[0113] 13 shows a pseudo program example 62 in the style of PyTorch (registered trademark) as a more specific example of how to implement Causal Linear 61. The shape of the processed data is shown as a comment on the right side of each line.

[0114] In the configuration described above, even in the case of an encoder (when used as the feature generation processing unit 14a1, the intermediate representation generation model processing unit 14a2, the auxiliary input conversion processing unit 14a3, or the input data converter 14a7) or a decoder (when used as the entropy predictor 14a4 or the generation model processing unit 14b1), if padding is performed before the processing of Up546A, a general Linear layer may be used instead of a Causal Linear layer.

[0115] (Effects of the fourth embodiment) The following describes examples of the effects achieved by the configuration and processing of the neural network model described above. By gradually reducing the size of the output data (e.g., the scale of the token dimension) using Scale Down Block 53, layer processing can be accelerated while a shortcut path allows processing while retaining fine-grained information in the data, thereby improving the overall accuracy of the neural network. Furthermore, when using a neural network as a decoder, maintaining causality in the direction of the token dimension during Up546 requires padding with a conventional linear algorithm, which can result in leakage of nearby receptive fields and inefficient improvement of accuracy. However, by introducing group counts into Scale Down Block 53 and Scale Up Block 54 and introducing Causal Linear, even if the token dimension is gradually reduced using Down536, the data elements of the token dimension converted to the channel dimension by Down536 correspond to groups in the channel dimension, preserving fine-grained causality even after Up546 is performed. This prevents leakage of the receptive field and may efficiently improve the accuracy of the neural network as a whole.

[0116] The present invention is not limited to the above-described embodiments, and includes various modifications. Furthermore, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the described configurations. Furthermore, not only are such configurations removed, but replacement or addition of configurations is also possible. Furthermore, forms in which some or all of the above-described embodiments are appropriately combined to be consistent are also included in the embodiments of the present invention.

[0117] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be implemented by software program code that implements the functions of the embodiments. In this case, a recording medium on which the program code is recorded is provided to a computer, and a processor within the computer reads the program code stored in the recording medium.

[0118] In this case, the program code itself read from the recording medium will realize the functions of the above-described embodiment, and the program code itself and the recording medium on which it is stored constitute the present invention. Examples of recording media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

[0119] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as Python (registered trademark), assembler, C / C++, perl, Shell, PHP, Java (registered trademark), etc.

[0120] In the above-described embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected. [Explanation of symbols]

[0121] 10, 10B, 10C: computer, 13: memory, 14: processor, 15: parallel processing device.

Claims

1. An information processing method executed by an information processing system having a processor and a memory, comprising: the processor: Accepts input data, generating features of the input data; acquiring the recalled data corresponding to the feature amount from correspondence relationship information between the feature amount and the recalled data of the input data; The acquired recall data is input into a generating AI (Artificial Intelligence), Response data to the input data is obtained from the generation AI. An information processing method characterized by comprising each process.

2. 2. The information processing method according to claim 1, The input data is a prompt to ask the generated AI a question, The recalled data is compressed data of auxiliary input data based on the input data.

1. An information processing method comprising:

3. 3. The information processing method according to claim 2, The processor: generating intermediate representation data from the prompt; generating the feature amount from the intermediate representation data; converting the intermediate representation data into the auxiliary input data; The feature amount and the auxiliary input data are associated with each other to generate the association information.

1. An information processing method comprising:

4. 4. The information processing method according to claim 3, The processor: The auxiliary input data is learned to generate a generative model that the generative AI has.

1. An information processing method comprising:

5. 5. The information processing method according to claim 4, The processor: learning the auxiliary input data to generate an entropy predictor corresponding to the intermediate representation data; compressing the auxiliary input data using the entropy predictor; The feature amount and the compressed auxiliary input data are associated with each other to generate the association information.

1. An information processing method comprising:

6. 4. The information processing method according to claim 3, The processor: The feature amount and the intermediate representation data are generated using a neural network model.

1. An information processing method comprising:

7. 2. The information processing method according to claim 1, the input data is image data, The recalled data is compressed data of intermediate representation data based on the image data.

1. An information processing method comprising:

8. 8. The information processing method according to claim 7, The processor: generating a feature (compressed) from the image data, the feature being a compressed feature; Decompressing the feature (compressed) to generate the feature; converting the feature amount into input transformed data; generating the intermediate representation data from the input transformation data; The feature amount (compressed) and the intermediate representation data are associated with each other to generate the association information.

1. An information processing method comprising:

9. 9. The information processing method according to claim 8, The processor: The intermediate representation data is learned to generate a generative model that the generative AI has.

1. An information processing method comprising:

10. 9. The information processing method according to claim 8, The processor: learning the feature values ​​to generate an entropy predictor corresponding to the feature values; Decompressing the feature (compressed) using the entropy predictor to generate the feature; Compress the feature using the entropy predictor to generate the feature (compressed).

1. An information processing method comprising:

11. 8. The information processing method according to claim 7, The processor: The feature amount and the intermediate representation data are generated using a neural network model.

1. An information processing method comprising:

12. 12. The information processing method according to claim 11, The processor: If the size of the feature (compressed) is equal to or smaller than a predetermined value, the intermediate representation data is obtained from the correspondence relationship information; If the size of the feature (compressed) is larger than a predetermined value, the intermediate representation data is generated using the neural network model.

1. An information processing method comprising:

13. An information processing system having a processor and a memory, The processor: Accepts input data, generating features of the input data; acquiring the recalled data corresponding to the feature amount from correspondence relationship information between the feature amount and the recalled data of the input data; The acquired recall data is input into a generating AI (Artificial Intelligence), Response data to the input data is obtained from the generation AI. An information processing system comprising:

14. On the computer, Accepts input data, generating features of the input data; acquiring the recalled data corresponding to the feature amount from correspondence relationship information between the feature amount and the recalled data of the input data; The acquired recall data is input into a generating AI (Artificial Intelligence), Response data to the input data is obtained from the generation AI. An information processing program that causes each process to be executed.

Images