Data processing device and method

Abstract

A data processing method performed by a data processing device according to an embodiment comprises the steps of: acquiring the data of a single batch corresponding to a data set as an output of a first multilayer perceptron model with respect to the data set input to the first multilayer perceptron model; inputting the batch data and noise to an induced self-attention model having a permutation-invariant structure to generate intermediate data including the batch data and pseudo context data for the batch data as an output of the induced self-attention model; and inputting the intermediate data and unknown data into a second multilayer perceptron model and outputting the mean and variance of the unknown data as an output of the second multilayer perceptron model. and inputting the intermediate data and the unknown data into a second multilayer perceptron model and outputting a mean and a variance of the unknown data as an output of the second multilayer perceptron model.

Description

Data processing device and method

[0001] The present invention relates to a data processing device and method including a neural process model.

[0002] This invention is a research (No. 2022-0-00713-002, Development of Meta-Learning Technology Applicable to Real-World Problems) conducted with support from the Information and Communications Technology Planning and Evaluation Institute with funding from the government (Ministry of Science and ICT) in 2023.

[0003] Gaussian processes, which are traditional Bayesian models of stochastic processes, suffer from computational complexity that increases proportionally to the square of the amount of data. Consequently, neural processes, which mimic stochastic processes using artificial neural network models, have been proposed.

[0004] Previous neural process models have not been sufficiently effective in extracting functional uncertainty. This has led to models failing to accurately capture uncertainty in output values ​​relative to input values, ultimately hindering their performance.

[0005] According to an embodiment, a data processing device and method for applying a martingale posterior distribution method to a neural process model are provided.

[0006] The problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0007] As a first aspect, a data processing method performed by a data processing device includes the steps of: obtaining a single bundle of data corresponding to a data set input to a first multilayer perceptron model as an output of the first multilayer perceptron model; inputting the bundled data and noise into an induced self-attention model having a permutation invariant structure to generate intermediate data including pseudo context data for the bundled data and the bundled data as an output of the self-attention induced model; and inputting the intermediate data and unknown data into a second multilayer perceptron model to output the mean and variance of the unknown data as an output of the second multilayer perceptron model.

[0008] As a second aspect, a data processing device includes a memory storing at least one command; and a processor executing the command to obtain a single bundle of data corresponding to a data set input to a first multilayer perceptron model as an output of the first multilayer perceptron model, input the bundled data and noise into a self-attention induction model having a permutation invariant structure, generate intermediate data including similar context data for the bundled data and the bundled data as an output of the self-attention induction model, and input the intermediate data and unknown data into a second multilayer perceptron model to output the mean and variance of the unknown data as an output of the second multilayer perceptron model.

[0009] As a third aspect, a computer-readable recording medium storing a computer program includes instructions for causing the processor to perform the data processing method when the computer program is executed by the processor.

[0010] In one embodiment, the Martingale posterior distribution method is applied to a neural process model. This solves the problem of conventional neural process models failing to accurately extract functional uncertainty, and sufficiently captures the uncertainty in the output value for given input data. Therefore, it effectively enables more accurate decision-making when making predictions using neural process models.

[0011] Figure 1 is a configuration diagram of a data processing device according to an embodiment of the present invention.

[0012] Figure 2 is a configuration diagram of a martingale posterior neural model mounted on the data processing device illustrated in Figure 1.

[0013] Figure 3 is a flowchart for explaining a data processing method according to an embodiment of the present invention.

[0014] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Like reference numerals designate like elements throughout the specification.

[0015] When describing embodiments of the present invention, detailed descriptions of known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined in light of their functions in the embodiments of the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0016] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

[0017] The terms used in this specification will be briefly explained, and the present invention will be described in detail.

[0018] The terms used in this invention have been selected from widely used, current terms, taking into account the functions of the invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the invention.

[0019] When a part of a specification is said to 'include' a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

[0020] Also, the term 'part' used in the specification means a software or hardware component such as an FPGA or ASIC, and the 'part' performs certain functions. However, the 'part' is not limited to software or hardware. The 'part' may be configured to reside on an addressable storage medium or may be configured to play one or more processors. Thus, as an example, the 'part' includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

[0021] FIG. 1 is a configuration diagram of a data processing device according to an embodiment of the present invention, and FIG. 2 is a configuration diagram of a martingale posterior neural model mounted on the data processing device illustrated in FIG. 1.

[0022] Referring to FIGS. 1 and 2, the data processing device (100) includes a memory (110) and a processor (120) equipped with a Martingale post-neural model (111), and may further include an input unit (130) and / or an output unit (140).

[0023] The memory (110) is equipped with a Martingale post-neural model (111) that includes instructions that can be executed by the processor (120), and can further store various types of information necessary for executing the Martingale post-neural model (111).

[0024] The Martingale post-neural model (111) includes a compression unit (210), a generation unit (220), and an output unit (230).

[0025] The compression part (210) of the Martingale posterior neural model (111) is a data set (X c , Y c ) is input to the first multilayer perceptron model, and the data set (X) is input as the output of the first multilayer perceptron model. c , Y c ) compressed into a single bundle of data (R c ) is provided.

[0026] The generation unit (220) of the Martingale posterior neural model (111) is a bundle data (R c ) and noise (ε) are input to the self-attention induction model of permutation invariant structure, and the bundle data (R) is input as the output of the self-attention induction model. c ) for similar context data (R c ') and the above bundled data (R c ) containing intermediate data (R c , R c ') is created.

[0027] The output part (230) of the Martingale post-neural model (111) is the intermediate data (R c , R c ') and unknown data (X t ) is input to the second multilayer perceptron model, and the unknown data (X) is input as the output of the second multilayer perceptron model. t ) of the average (μ t ) and variance (σ t ) is printed.

[0028] The processor (120) loads the Martingale posterior neural model (111) by executing the command stored in the memory (110) and performs data processing according to the Martingale posterior neural model (111). This processor (120) may be composed of one or more processors. For example, one or more processors may be a general-purpose processor such as a central processing unit (CPU), a digital signal processor (DSP), a graphics-only processor such as a graphics processing unit (GPU), a vision processing unit (VPU), or an artificial intelligence-only processor such as a neural processing unit (NPU). This processor (120) may process the data set (X) through the Martingale posterior neural model (111). c , Y c ) is input to the first multilayer perceptron model, and the data set (X) is input as the output of the first multilayer perceptron model. c , Y c ) compressed into a single bundle of data (R c ) is provided. And, the processor (120) provides bundled data (R c ) and noise (ε) are input to the self-attention induction model of permutation invariant structure, and the bundle data (R) is input as the output of the self-attention induction model. c ) for similar context data (R c ') and the above bundled data (R c ) containing intermediate data (R c , R c ') is generated. And, the processor (120) generates intermediate data (R c , R c ') and unknown data (X t ) is input to the second multilayer perceptron model, and the unknown data (X) is input as the output of the second multilayer perceptron model. t ) of the average (μ t ) and variance (σ t) is output. Here, the noise is N, and the processor (120) outputs similar context data (R) N times. c ') and the above intermediate data (R c , R c ') can be created N times.

[0029] The input unit (130) can receive various information required for the processor (120) to execute the Martingale post-neural model (111). Such various information can be input in real time, and if input in advance, various information can be stored in the memory (110). For example, the input unit (130) can receive a data set (X c , Y c ) and noise (ε) and unknown data (X t ) can be entered.

[0030] The output unit (140) can output various processing data generated as the processor (120) executes the Martingale post-neural model (111). For example, the output unit (140) can include a data interface capable of outputting various processing data to external peripheral devices, a communication module capable of transmitting various processing data through a communication channel, etc.

[0031] Figure 3 is a flowchart for explaining a data processing method according to an embodiment of the present invention.

[0032] Hereinafter, with reference to FIGS. 1 to 3, a method for processing data through a data processing device (100) including a Martingale post-neural model (111) using the Martingale post-neural model (111) will be described.

[0033] First, the processor (120) of the data processing device (100) can load the Martingale post-neural model (111) by executing a command stored in the memory (110), thereby performing data processing according to the Martingale post-neural model (111).

[0034] The compressed part (210) of the Martingale posterior neural model (111) loaded by the processor (120) is a data set (X c , Y c ) is input to the first multilayer perceptron model, and the data set (X) is input as the output of the first multilayer perceptron model. c , Y c ) compressed into a single bundle of data (R c ) is provided (S310).

[0035] And, the generation unit (220) of the Martingale post-neural model (111) is a bundle data (R c ) and noise (ε) are input to the self-attention induction model of permutation invariant structure, and the bundle data (R) is input as the output of the self-attention induction model. c ) for similar context data (R c ') and the above bundled data (R c ) containing intermediate data (R c , R c ') is created (S320).

[0036] And, the output part (230) of the Martingale post-neural model (111) is the intermediate data (R c , R c ') and unknown data (X t ) is input to the second multilayer perceptron model, and the unknown data (X) is input as the output of the second multilayer perceptron model. t ) of the average (μ t ) and variance (σ t ) is printed (S330).

[0037] Meanwhile, N noises (ε) can be input one by one N times to the self-attention induction model of the generation unit (220), and the self-attention induction model can input N intermediate data (R) N times. c , R c ') can be generated, and the output unit (230) outputs N unknown data (X) N times. t ) of the average (μ t ) and variance (σt ) can be output. Such parallel computation can solve the problem of not being able to accurately extract functional uncertainty without significantly increasing the computational complexity of conventional neural process models. In addition, by using an appropriate loss function of the martingale posterior neural model (111), the quality of similar context data generated through the martingale posterior distribution can be improved and the performance of the model can be increased.

[0038] As described above, the embodiments of the present invention apply the Martingale posterior distribution method to neural process models. This solves the problem of conventional neural process models failing to accurately extract functional uncertainty, and sufficiently captures the uncertainty in the output value for given input data. Therefore, it enables more accurate decision-making when making predictions using neural process models.

[0039] Meanwhile, each step included in the data processing method using the Martingale post-neural model according to the above-described embodiment can be implemented in a computer-readable recording medium that records a computer program programmed to perform these steps.

[0040] In addition, each step included in the data processing method using the Martingale post-neural model according to the above-described embodiment can be implemented in the form of a computer program stored on a computer-readable recording medium programmed to perform such steps.

[0041] The combination of each step of each flowchart attached to the present invention may be performed by computer program instructions. These computer program instructions may be installed in a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment create a means for performing the functions described in each step of the flowchart. These computer program instructions may also be stored in a computer-usable or computer-readable recording medium that can direct a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-usable or computer-readable recording medium can also produce a manufactured article that includes an instruction means for performing the functions described in each step of the flowchart. Since the computer program instructions can also be installed on a computer or other programmable data processing device, a series of operational steps can be performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform can also provide steps for performing the functions described in each step of the flowchart.

[0042] Additionally, each step may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative embodiments, the functions described in the steps may occur out of order. For example, two steps depicted in succession may actually be performed substantially concurrently, or the steps may sometimes be performed in reverse order, depending on the corresponding function.

[0043] The above description is merely an illustrative illustration of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims

1. A data processing method performed by a data processing device, A step of obtaining a single bundle of data corresponding to the data set input to a first multilayer perceptron model as an output of the first multilayer perceptron model; A step of inputting the above bundled data and noise into an induced self-attention model of a permutation-invariant structure, thereby generating pseudo context data for the above bundled data and intermediate data including the above bundled data as outputs of the above self-attention induced model: and A step of inputting the intermediate data and unknown data into a second multilayer perceptron model and outputting the mean and variance of the unknown data as output of the second multilayer perceptron model; How to process data.

2. In paragraph 1, The above noise is N, and N similar context data and intermediate data are generated over N times. How to process data.

3. Memory that stores at least one instruction; By executing the above command, a processor is included that obtains a single bundle data corresponding to the data set as an output of the first multilayer perceptron model for the data set input to the first multilayer perceptron model, inputs the bundle data and noise into a self-attention induction model having a permutation invariant structure, generates intermediate data including similar context data for the bundle data and the bundle data as an output of the self-attention induction model, and inputs the intermediate data and unknown data into a second multilayer perceptron model, and outputs the mean and variance of the unknown data as an output of the second multilayer perceptron model. Data processing unit.

4. In paragraph 3, The above noise is N, and N similar context data and intermediate data are generated over N times. Data processing unit.

5. A computer-readable recording medium storing a computer program, The above computer program, when executed by a processor, A step of obtaining a single bundle of data corresponding to the data set input to the first multilayer perceptron model as an output of the first multilayer perceptron model; A step of inputting the above bundled data and noise into a self-attention induction model having a permutation-invariant structure, thereby generating similar context data for the above bundled data and intermediate data including the above bundled data as outputs of the self-attention induction model: and A step of inputting the intermediate data and unknown data into a second multilayer perceptron model and outputting the mean and variance of the unknown data as output of the second multilayer perceptron model; A computer-readable recording medium containing instructions for causing the processor to perform a data processing method.

Images