Data processing method and computing device for NPU-based batch inference optimization

Abstract

Disclosed are a data processing method and computing device for NPU-based batch inference optimization. The data processing method according to an embodiment of the present invention includes: a step of receiving one or more inputted data items; a step of generating a batch by using the one or more data items and determining whether the batch satisfies batch processing conditions on the basis of the time elapsed from the input time of the first data item that was inputted first among the one or more data items included in the batch, a computation processing time, and a time limit; and a data processing step of either performing NPU computation processing through batch processing or waiting for the next data input, depending on the result of the determination.

Description

Data processing method and computing device for optimizing NPU-based batch inference

[0001] The disclosed embodiments relate to a batch inference optimization technique for NPUs.

[0002] A neural processing unit (NPU) may be hardware designed to accelerate the computations required for artificial neural networks. NPUs are optimized for deep learning algorithms and can perform matrix multiplication, convolution, and other operations at higher speeds and with more energy efficiency than central processing units (CPUs) and graphics processing units (GPUs).

[0003] The neural network processing device described above can operate by implementing mathematical operations required for deep learning algorithms using highly parallel processing elements.

[0004] The disclosed embodiments are intended to provide a data processing method and computing device for NPU-based batch inference optimization to improve computational processing capabilities in an NPU (neural processing unit).

[0005] A data processing method according to one embodiment is a method performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, the method comprising: receiving at least one input data; generating a batch using the at least one data, and determining whether a batch processing condition is satisfied based on an elapsed time from an input time of first data input first among the at least one data included in the batch, an operation processing time, and a time limit; and performing NPU operation processing through batch processing or waiting for the next data input based on a result of the determination.

[0006] The above operation processing time may be the time required for operation processing of at least one data.

[0007] The step of checking whether the above batch processing condition is satisfied may include the step of checking whether the time obtained by subtracting the elapsed time and the operation processing time from the time limit is less than a preset first time; if the result of the checking is not less than the first time, the step of calculating a probability value that a probability variable following a Poisson distribution of the time obtained by subtracting the elapsed time and the operation processing time from the time limit is not 0; and the step of checking whether the probability value that the probability variable following the Poisson distribution is not 0 exceeds a preset threshold value. The data processing step may include the step of waiting for input of the next data if the result of the checking is greater than the threshold value.

[0008] As a result of the above verification, if the threshold value is not exceeded, the data processing method may further include a step of performing NPU operation processing through the batch processing.

[0009] As a result of the verification, if it is less than the first time, the data processing method may further include a step of performing NPU operation processing through the batch processing.

[0010] The step of checking whether the above batch processing condition is satisfied may include a step of listing the at least one or more data according to the data acquisition time; and a step of identifying pre-input data up to a preset number among the at least one or more data, and the data processing step may include a step of performing NPU operation processing on the pre-input data up to the preset number through the batch processing according to the identification result.

[0011] The above data processing method may further include a step of processing the input data exceeding the preset number by waiting for the input of the next data, based on the result of the identification.

[0012] The above data processing method can perform the NPU operation processing or wait for the next data input within a time period in which the elapsed time of the first data does not exceed the time limit.

[0013] According to one embodiment, a computing device includes one or more processors; and a memory storing one or more programs executed by the one or more processors, wherein the one or more processors receive at least one or more input data, generate a batch using the at least one or more data, and determine whether a batch processing condition is satisfied based on an elapsed time from an input time of first data input first among the at least one or more data included in the batch, an operation processing time, and a time limit, and performs NPU operation processing through batch processing or performs data processing to wait for the next data input based on a result of the determination.

[0014] The computing device, when checking whether the batch processing condition is satisfied, checks whether the time obtained by subtracting the elapsed time and the operation processing time from the time limit is less than a preset first time, and if the result of the check is not less than the first time, calculates a probability value that a probability variable following a Poisson distribution of the time obtained by subtracting the elapsed time and the operation processing time from the time limit is not 0, and checks whether the probability value that the probability variable following the Poisson distribution is not 0 exceeds a preset threshold, and when performing the data processing, if the result of the check exceeds the threshold, input of the next data can be waited for.

[0015] The computing device, when checking whether the batch processing condition is satisfied, checks whether the time obtained by subtracting the elapsed time and the operation processing time from the time limit is less than a preset first time, and if the result of the check is not less than the first time, calculates a probability value that a probability variable following a uniform distribution of the time obtained by subtracting the elapsed time and the operation processing time from the time limit is not 0, and checks whether the probability value that the probability variable following the uniform distribution is not 0 exceeds a preset threshold value, wherein the uniform distribution is a distribution in an elapsed reference range time including a preset time before and after based on a preset elapsed time that has elapsed from a time point at which the first data is input, and when performing the data processing, if the result of the check exceeds the threshold value, the input of the next data can be waited for.

[0016] The computing device, when checking whether the batch processing condition is satisfied, includes listing the at least one or more data according to the data acquisition time, and identifying pre-input data up to a preset number among the at least one or more data, and when performing the data processing, can perform NPU operation processing through the batch processing on the pre-input data up to the preset number as a result of the identification.

[0017] The computing device can perform the NPU operation processing or wait for the next data input within a time period in which the elapsed time of the first data does not exceed the time limit.

[0018] In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the disclosed embodiment may be further provided.

[0019] According to the disclosed embodiments, it is expected that the NPU (neural processing unit) operation processing performance can be further improved by more systematically performing batch processing of data including images input to the NPU.

[0020] Figure 1 is a block diagram illustrating a data processing device according to one embodiment.

[0021] Figures 2 and 3 are exemplary diagrams for explaining a data processing method according to one embodiment.

[0022] Figure 4 is a flowchart for explaining a data processing method according to one embodiment.

[0023] Figures 5 and 6 are flowcharts for explaining the data processing method of Figure 4 in more detail.

[0024] FIG. 7 is a block diagram illustrating a computing environment including a computing device according to one embodiment.

[0025] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, devices, and / or systems described herein. However, these are merely examples and the present invention is not limited thereto.

[0026] In describing embodiments of the present invention, if a detailed description of a known technology related to the present invention is judged to unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of their functions in the present invention, and this may vary depending on the intention or custom of the user or operator. Therefore, the definitions should be made based on the contents throughout this specification. The terminology used in the detailed description is only for the purpose of describing embodiments of the present invention and should not be limited in any way. Unless clearly used otherwise, the singular form includes the plural form. In this description, expressions such as "comprises" or "having" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, and should not be construed to exclude the presence or possibility of one or more other features, numbers, steps, operations, elements, parts or combinations thereof other than those described.

[0027] FIG. 1 is a block diagram illustrating a data processing device according to one embodiment.

[0028] Hereinafter, a description will be given with reference to FIGS. 2 and 3, which are exemplary diagrams for explaining a data processing method according to one embodiment.

[0029] Referring to FIG. 1, the data processing device (100) includes a batch processing unit (110) and an input inference unit (120). The components illustrated in FIG. 1 are not essential for implementing the data processing device (100) according to the present disclosure, and thus, the data processing device (100) described herein may have more or fewer components than the components listed above. The components illustrated in FIG. 1 may be communicatively connected to each other via a communication network (not shown). In some embodiments, the communication network may include the Internet, one or more local area networks, wire area networks, a cellular network, a mobile network, other types of networks, or a combination of these networks.

[0030] For example, the situation conditions according to the present embodiment may be based on NPU hardware having deterministic latency capable of batch processing.

[0031] The data processing device (100) may have an internal memory within the chip that does not function as a cache. For example, the internal memory within the chip may be L2SRAM.

[0032] The data processing device (100) may have data that can be used repeatedly for each core. For example, data may be brought between DRAM and a chip only once through weight data and branch DMA, and then copied and transferred to each core.

[0033] Due to the above-described characteristics of the data processing device (100), the present embodiment may be advantageous in terms of speed and power during batch processing.

[0034] If up to 3 batches can be processed and the data is image data, the computational processing time for 1 image data is A1ms, the computational processing time for 2 image data is A2ms, the computational processing time for 3 image data is A3ms, and the computational processing time for 4 or more image data is A 4 = ∞ It can be ms. In this case, 0 < A1< A2< A 3, It could be 2A1> A2, 3A2> 2A3.

[0035] As another example, a situation condition according to the present embodiment may have a limit latency within which processing must be completed for all data (image data).

[0036] When data (image data) is input into a queue with Exp() distribution from multiple cameras, the data processing device (100) must complete NPU operation processing for each data before the time limit (K ms) elapses.

[0037] D, as described below, represents the elapsed time, which may refer to the time elapsed from the input point of the first data (image data, first data) included in the batch. n may refer to the number of data (image data) included in the batch. N may refer to the maximum number of batches, which may refer to the maximum number of batches that can be processed.

[0038] Below, an example will be described in which input data is received, batch-processed depending on whether batch processing conditions are satisfied, and NPU operation processing is performed, or the input of the next data is waited for.

[0039] The batch processing unit (110) can receive at least one piece of input data. In the present embodiment, the data may refer to image data, but is not limited thereto.

[0040] The batch processing unit (110) can generate a batch using at least one piece of data. The batch processing unit (110) can store the input data in a buffer (not shown). In this case, the buffer (not shown) can temporarily store the input data before batch processing when the batch processing conditions are satisfied.

[0041] The batch processing unit (110) can perform NPU operation processing through batch processing, or perform data processing while waiting for the next data input, depending on the result of the confirmation of whether the batch processing conditions are satisfied in the input inference unit (120). Although not shown, the data processing device (100) may additionally be equipped with an operation unit for NPU operation processing, but is not limited thereto, and the operation unit may be implemented separately and independently from the data processing device (100).

[0042] The batch processing unit (110) can perform NPU operation processing or wait for the next data input within a time period in which the elapsed time of the first data does not exceed the time limit. The first data may refer to the first input data included in the batch.

[0043] The input inference unit (120) can determine whether the batch processing conditions are satisfied based on the elapsed time from the input time of the first data input among at least one or more data included in the batch, the operation processing time, and the time limit. The operation processing time may refer to the time required for the operation processing of at least one or more data included in the batch. For example, in the case of one piece of data, the operation processing time may refer to the operation processing time of one piece of data, and in the case of two pieces of data, the operation processing time may refer to the operation processing time of two pieces of data.

[0044] Below, we will explain how to process data depending on whether the batch processing conditions are satisfied.

[0045] For example, the input inference unit (120) may wait until the latest input data (image data) (n+1 in mathematical expression 1) among the data included in the batch satisfies mathematical expression 1, and then perform batch processing through the batch processing unit (110).

[0046] (Equation 1)

[0047] D+A n-1 =K

[0048] The above D is the elapsed time, the time elapsed from the input time of the first data (image data, first data) included in the batch, the above n is the number of data (image data) included in the batch, and the above A n-1 is the computational processing time for n+1 data, and the above K may mean the time required for each input data to complete NPU computational processing from the input time.

[0049] As another example, the input inference unit (120) can determine whether to batch process the input data based solely on the current situation. This allows batch processing to be performed relatively early based on the queue distribution, without waiting for up to K hours.

[0050] This embodiment does not always run the watch dog, but can proceed with the batch processing procedure only when a condition is satisfied through the inverse calculation of the time that new data (image) is input (n update) or the time that can wait for the input of new data (the inverse function of the variance using p). To simplify the calculation of the watch dog, etc., a first time (ε) may be required. The first time (ε) may be arbitrarily set by the operator. For example, the first time can be linearly obtained from one inference time.

[0051] New data (image data) is input, so n is updated, and Poi (KDA n+1)| !0 When < 0.3, the input inference unit (120) can perform batch processing through the batch processing unit (110) without waiting for the input of other data, and proceed with NPU operation processing because the probability that one more data will be input and operate by forming an n-1 batch is less than 0.3.

[0052] Specifically, referring to mathematical expression 2, when checking whether the batch processing condition is satisfied, the input inference unit (120) can check whether the time obtained by subtracting the elapsed time and the operation processing time from the time limit is less than the preset first time.

[0053] (Equation 2)

[0054] KDA n-1 <ε

[0055] The above K represents a time limit, which may refer to the time within which each input data must complete NPU computational processing from the time of input. The above D represents an elapsed time, which may refer to the time elapsed from the time the first data included in the batch was input. For example, referring to FIG. 2, in a situation where multiple image data such as Image 1, Image 2, and Image 3 are input, the elapsed time from the time the first input data (Image 1) was input to the present time (now) may be D.

[0056] A above n+1 is the operation processing time, which may refer to the time required for NPU operation processing of n+1 input data. The n may refer to the number of data included in the batch. ε may refer to a preset first time.

[0057] As a result of the verification, if it is not less than the first time, the input inference unit (120) calculates the Poisson distribution (Poi(KDA)) of the time obtained by subtracting the elapsed time and the operation processing time from the time limit. n-1)) can be used to calculate the probability value that the random variable following is not 0. This can be expressed as mathematical expression 3.

[0058] (Equation 3)

[0059] Poi(KDA n-1 )| !0

[0060] The Poisson distribution described above can refer to a discrete probability distribution that can be used when an event occurs randomly in a certain unit of time or unit of space.

[0061] Referring to mathematical expression 4, the input inference unit (120) can check whether the probability value of a random variable following a Poisson distribution that is not 0 exceeds a preset threshold value.

[0062] (Equation 4)

[0063] Poi(KDA n-1 )| !0 > p

[0064] The above p may denote a threshold. At this time, p may be a value that can change depending on the situation and may be arbitrarily set by the operator. This is because p does not have a trade-off, but rather, there may be an optimal p for each data processing device (100). For example, the threshold p may be arbitrarily set by the operator to a probability of 99% or a probability of 95%, and may be subsequently changed or reset depending on the characteristics of the overall system to which it is applied.

[0065] If the probability value of a random variable following a Poisson distribution that is not 0 exceeds a preset threshold, the batch processing unit (110) can wait for the input of the next data.

[0066] Meanwhile, if the probability value of a random variable following a Poisson distribution not being 0 does not exceed a threshold, NPU operation processing can be performed through batch processing via a batch processing unit (110). Although not shown, the data processing device (100) can additionally implement an operation unit for NPU operation processing.

[0067] On the other hand, if the above-described verification result is less than the first time, the input inference unit (120) can perform NPU operation processing through batch processing via the batch processing unit (110).

[0068] On the other hand, the distribution in which input data accumulates in a queue is not limited to the Poisson distribution described above, and may also include distributions that depend on the current elapsed time. For example, in the case of a camera, input data may also exist as a distribution that depends on the current elapsed time (hereinafter, a uniform distribution).

[0069] Accordingly, as another example, when checking whether the batch processing condition is satisfied, the input inference unit (120) may consider a uniform distribution of the elapsed reference range time in which the elapsed time from the time when the input data is accumulated in the queue is preset, rather than a Poisson distribution. For example, in the case of input data from a camera, the input inference unit (120) may consider a uniform distribution in the elapsed reference range time (e.g., [25 ms, 35 ms]) including the time before and after the elapsed time (e.g., 30 ms) from the time when the input data is input. That is, the uniform distribution can replace the above-described Poisson distribution.

[0070] Specifically, according to the above-described mathematical expression 2, when checking whether the batch processing condition is satisfied, the input inference unit (120) can check whether the time obtained by subtracting the elapsed time and the operation processing time from the time limit is less than the preset first time.

[0071] As a result of the verification, if it is not less than the first time, the input inference unit (120) can calculate a probability value that the probability variable following the uniform distribution of the input data is not 0. In this case, the uniform distribution may mean a distribution in the elapsed reference range time including the preset time before and after the preset elapsed time based on the preset elapsed time from the time when the first data was input.

[0072] The input inference unit (120) can check whether the probability value of a random variable following a uniform distribution that is not 0 exceeds a preset threshold value.

[0073] If the probability value that a random variable following a uniform distribution is not 0 exceeds a preset threshold, the batch processing unit (110) can wait for the input of the next data. If the probability value that a random variable following a uniform distribution is not 0 does not exceed the threshold, the input inference unit (120) can perform NPU operation processing through batch processing via the batch processing unit (110).

[0074] As another example, referring to FIG. 3, when determining whether a batch processing condition is satisfied, the input inference unit (120) may list at least one piece of data according to its data acquisition time. In this case, the data acquisition time may refer to a past data acquisition time.

[0075] The input inference unit (120) can identify pre-input data up to a preset number among at least one or more pieces of data.

[0076] The batch processing unit (110) can perform NPU operation processing through batch processing of line input data (Image 1, Image 2 of FIG. 3) up to a preset number. The batch processing unit (110) waits for the input of the next data (next queue), and when the next data is input, it can batch process it together with the remaining subsequent input data.

[0077] Specifically, the batch processing unit (110) may, based on input data having a time difference exceeding a threshold among a plurality of input data listed in time series order, give priority to batch-processing of pre-input data, and then batch-process data input after the time difference exceeds the threshold together with the subsequent input data. Referring to FIG. 3, if Image 1, Image 2, and Image 3 are currently accumulated in the queue, the batch processing unit (110) may not batch-process three images (Image 1, Image 2, and Image 3) at once, but may batch-process only two images (Image 1 and Image 2) first, and then process Image 3 together with the input to be input later. When the batch processing unit (110) measures the waiting time limit K between Image 2 and Image 3, if there is a large time difference, it can process Image 1 and Image 2 first to increase the waiting time by recalculating from Image 3, and then process Image 3 together with the input to be input later.

[0078] FIG. 4 is a flowchart illustrating a data processing method according to one embodiment. The method illustrated in FIG. 4 may be performed, for example, by the aforementioned data processing device (100). While the illustrated flowchart describes the method as divided into multiple steps, at least some of the steps may be performed in a different order, combined with other steps and performed together, omitted, divided into substeps and performed, or one or more steps not illustrated may be added and performed.

[0079] At step 1100, the data processing device (100) can receive at least one piece of input data.

[0080] At step 1200, the data processing device (100) can generate a batch using at least one data, and check whether the batch processing condition is satisfied based on the elapsed time from the input time of the first data input first among at least one data included in the batch, the operation processing time, and the time limit.

[0081] The above operation processing time may be the time required for operation processing of at least one data.

[0082] In steps 1300 and 1400, the data processing device (100) may perform NPU operation processing through batch processing according to the verification result, or perform data processing while waiting for the next data input.

[0083] The data processing device (100) can perform the NPU operation processing or wait for the next data input within a time period in which the elapsed time of the first data does not exceed the time limit.

[0084] Figure 5 is a flowchart to explain the data processing method of Figure 4 in more detail.

[0085] At step 2100, the data processing device (100) can receive at least one piece of input data.

[0086] At step 2200, the data processing device (100) can check whether the time obtained by subtracting the elapsed time and the operation processing time from the time limit is less than a preset first time (Mathematical Expression 1).

[0087] At step 2300, if the result of the verification at step 2200 is not less than the first time, the data processing device (100) can calculate a probability value that the probability variable following the Poisson distribution of the time obtained by subtracting the elapsed time and the operation processing time from the time limit is not 0.

[0088] At step 2400, the data processing device (100) can check whether the probability value of a random variable following a Poisson distribution that is not 0 exceeds a preset threshold value.

[0089] At step 2500, if the verification result of step 2400 exceeds the threshold, the data processing device (100) can wait for input of the next data.

[0090] At step 2600, if the verification result at step 2400 does not exceed the threshold, the data processing device (100) can perform NPU operation processing through batch processing.

[0091] Meanwhile, if the result of the verification at step 2200 is less than the first time, the data processing device (100) can perform NPU operation processing through batch processing at step 2600.

[0092] Figure 6 is a flowchart to explain the data processing method of Figure 4 in more detail.

[0093] At step 3100, the data processing device (100) can receive at least one piece of input data.

[0094] At step 3200, the data processing device (100) can list at least one data according to the data acquisition time (image acquisition time of FIG. 6).

[0095] The above data acquisition time may refer to the point in time when input data (image) is accumulated in a queue. The data processing device (100) can measure the time for which input data is currently stored and waiting in the queue by subtracting the data acquisition time (the point in time when the data is accumulated in the queue) from the current time through the data acquisition time (current time - data acquisition time).

[0096] At step 3300, the data processing device (100) can identify at least one or more pre-input data up to a preset number.

[0097] At step 3400, based on the identification result of step 3300, the data processing device (100) can perform NPU operation processing through batch processing of line input data up to a preset number.

[0098] At step 3500, based on the result of the identification at step 3300, the data processing device (100) can process the input data exceeding the preset number by waiting for the input of the next data.

[0099] FIG. 7 is a block diagram illustrating a computing environment including a computing device according to one embodiment. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

[0100] The illustrated computing environment (10) includes a computing device (12). The computing device (12) may be one or more components included in a data processing device (100) according to one embodiment.

[0101] A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate according to the exemplary embodiments mentioned above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations according to the exemplary embodiments.

[0102] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (12) and store desired information, or a suitable combination thereof.

[0103] A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).

[0104] The computing device (12) may also include one or more input / output interfaces (22) that provide interfaces for one or more input / output devices (24) and one or more network communication interfaces (26). The input / output interfaces (22) and the network communication interfaces (26) are connected to the communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) via the input / output interfaces (22). Exemplary input / output devices (24) may include input devices such as pointing devices (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as display devices, printers, speakers and / or network cards. The exemplary input / output devices (24) may be included within the computing device (12) as a component constituting the computing device (12), or may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0105] The NPU hardware applied to this embodiment may have the following differences compared to batch processing of general hardware (GPU hardware). For example, in the case of GPU, memory may not be used deterministically, and the internal memory may be in the form of a cache. In this embodiment, since the internal memory and DMA are deterministically applied at compiler time, the batch size has an immediate effect on latency, so it can be predicted and calculated in advance. In GPU, there is no deterministic latency according to batch, the batch is not a real-time queue, and the hardware utility is relatively very low, so the utility can be increased by using software batch calculation. For example, since the GPU receives multiple images, adds an overlapping dimension, and calculates to increase the batch, an out-of-memory (OOM) phenomenon may occur.

[0106] The disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0107] While representative embodiments of the present invention have been described in detail above, those skilled in the art will appreciate that various modifications to the above-described embodiments are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims set forth below but also by equivalents thereof.

Claims

1. One or more processors, and A method performed on a computing device having a memory storing one or more programs executed by one or more processors, A step of receiving at least one or more input data; A step of generating a batch using at least one or more of the above data, and checking whether the batch processing condition is satisfied based on the elapsed time from the input time of the first data input first among the at least one or more of the above data included in the batch, the operation processing time, and the time limit; and A data processing method including a data processing step of performing NPU operation processing through batch processing or waiting for the next data input according to the verification result.

2. In claim 1, A data processing method, wherein the above operation processing time is the time required for operation processing of at least one data.

3. In claim 1, The step of checking whether the above batch processing conditions are satisfied is: A step of checking whether the time obtained by subtracting the elapsed time and the operation processing time from the above time limit is less than a preset first time; As a result of the verification, if it is not less than the first time, a step of calculating a probability value that is not 0 of a probability variable that follows a Poisson distribution of the time obtained by subtracting the elapsed time and the operation processing time from the limited time; and A step of checking whether the probability value of the random variable following the above Poisson distribution is not 0 exceeds a preset threshold value, The above data processing steps are: A data processing method, comprising a step of waiting for input of the next data if the verification result exceeds the threshold value.

4. In claim 3, A data processing method further comprising a step of performing NPU operation processing through the batch processing if the above verification result does not exceed the threshold value.

5. In claim 4, A data processing method further comprising a step of performing NPU operation processing through batch processing if the result of the verification is less than the first time.

6. In claim 1, The step of checking whether the above batch processing conditions are satisfied is: A step of listing at least one of the above data according to data acquisition time; and Comprising a step of identifying pre-input data up to a preset number of at least one of the above data, The above data processing steps are: A data processing method, comprising a step of performing NPU operation processing on line input data up to the preset number through batch processing according to the identification result.

7. In claim 6, A data processing method further comprising a step of processing subsequent input data exceeding the preset number based on the results of the analysis by waiting for the input of the next data.

8. In claim 1, A data processing method, wherein the NPU operation processing is performed or the next data input is waited for within a time period in which the elapsed time of the first data does not exceed the time limit.

9. One or more processors; and A memory storing one or more programs executed by said one or more processors, One or more of the above processors, Receive at least one piece of input data, Generate a batch using at least one of the above data, Check whether the batch processing conditions are satisfied based on the elapsed time from the input time of the first data input among at least one or more data included in the above batch, the operation processing time, and the time limit, and A computing device that performs NPU operation processing through batch processing based on the verification result, or performs data processing while waiting for the next data input.

10. In claim 9, When checking whether the above batch processing conditions are satisfied, Check whether the time obtained by subtracting the elapsed time and the operation processing time from the above time limit is less than the preset first time, As a result of the verification, if it is not less than the first time, the probability value that the probability variable following the Poisson distribution of the time obtained by subtracting the elapsed time and the operation processing time from the limited time is not 0 is calculated, and Including checking whether the probability value of the random variable following the above Poisson distribution is not 0 exceeds a preset threshold, When performing the above data processing, A computing device that waits for input of the next data if the verification result exceeds the above threshold.

11. In claim 9, When checking whether the above batch processing conditions are satisfied, Check whether the time obtained by subtracting the elapsed time and the operation processing time from the above time limit is less than the preset first time, As a result of the verification, if it is not less than the first time, the probability value that the probability variable that follows the uniform distribution of the time obtained by subtracting the elapsed time and the operation processing time from the limited time is not 0 is calculated, and It includes checking whether the probability value of the random variable following the above uniform distribution is not 0 exceeds a preset threshold value, The above uniform distribution is a distribution in the elapsed reference range time including the preset time before and after the preset elapsed time from the time the first data is input, When performing the above data processing, A computing device that waits for input of the next data if the verification result exceeds the above threshold.

12. In claim 9, When checking whether the above batch processing conditions are satisfied, Listing at least one of the above data according to data acquisition time, and Including identifying pre-input data up to a preset number of at least one of the above data, When performing the above data processing, A computing device that performs NPU operation processing on line input data up to the preset number through batch processing as a result of the determination.

13. In claim 9, A computing device that performs the NPU operation processing or waits for the next data input within a time period in which the elapsed time of the first data does not exceed the time limit.

Images