Electronic device for processing stream data

Abstract

Disclosed is an electronic device comprising: a memory, which stores data for computational processing of a processor; a stream input buffer, which sequentially stores input stream data input in a stream data format, converts the input stream data into a processor data format that can be processed by the processor, and transmits the input stream data having the processor data format to the memory; the processor which acquires the input stream data having the processor data format from the memory, and which stores, in the memory, computational processing result data about the input stream data having the processor data format; and a stream output buffer which stores, in a processor data format, the computational processing result data received from the memory, and which converts the computational processing result data into a stream data format and sequentially outputs same.

Description

Electronic devices that process stream data

[0001] The present disclosure relates to an electronic device for processing stream data.

[0002] Continuous vision (CV) systems process image / video data in a streaming manner, which is widely used in multimedia system-on-chip (SoC) such as camera image signal processors (ISPs), advanced driver-assistance systems (ADAS), and augmented / virtual reality (AR / VR). Digital television (DTV) SoCs are also representative commercial products that use CV systems. To maintain high user satisfaction, it is essential to display images / videos smoothly within a deterministic data processing time. Consequently, the CV systems in multimedia SoCs are implemented as pipelined fixed-functional modules, called vision pipelines, that are interconnected using a unidirectional first-in first-out (FIFO) order and process data in a raster-scanned manner. However, this structure is inflexible and has difficulty accommodating various vision processing algorithms.

[0003] Meanwhile, the increasing demand for high-performance algorithms in devices requiring real-time stream data processing, such as DTVs and monitors, has necessitated higher-performance and more flexible methods than previously available. This has led to the need for stream data SoCs that utilize processors rather than application-specific integrated circuits (ASICs). Existing stream data and processor data format transfer protocols have presented challenges, including (1) increased buffer size and (2) increased interface (buffer)-memory occupancy time.

[0004] Aspects of the present disclosure will be set forth in part in the description that follows, and in part will become apparent from the detailed description or may be learned by practicing the embodiments set forth herein.

[0005] An electronic device according to an embodiment of the present disclosure may include a memory that stores data for computational processing of a processor; a stream input buffer that sequentially stores input stream data input in a stream data format, converts the input stream data into a processor data format that can be processed by the processor, and transmits the input stream data in the processor data format to the memory; a processor that obtains input stream data in the processor data format from the memory and stores computational processing result data for the input stream data converted into the processor data format in the memory; a stream output buffer that stores computational processing result data received from the memory in the processor data format, converts the computational processing result data into the stream data format, and sequentially outputs the converted data; and a controller that determines access rights to the memory of the processor, the stream input buffer, and the stream output buffer based on a state of the processor, a state of the stream input buffer, and a state of the stream output buffer.

[0006] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0007] FIG. 1 is a diagram illustrating a system for processing stream data according to one embodiment of the present disclosure.

[0008] FIG. 2 is a block diagram illustrating an electronic device for processing stream data according to one embodiment of the present disclosure.

[0009] FIG. 3 is a diagram for explaining the configuration of a stream input buffer and a stream output buffer according to one embodiment of the present disclosure.

[0010] FIG. 4 is a diagram for explaining the operation of a stream input buffer according to one embodiment of the present disclosure.

[0011] FIG. 5 is a diagram for explaining the operation of a stream output buffer according to one embodiment of the present disclosure.

[0012] FIG. 6 is a diagram for explaining the amount of data change in a stream input buffer according to one embodiment of the present disclosure.

[0013] FIG. 7 is a diagram for explaining the amount of data change in a stream output buffer according to one embodiment of the present disclosure.

[0014] FIG. 8 is a flowchart illustrating a method by which a controller manages access rights to memory according to one embodiment of the present disclosure.

[0015] FIG. 9 is a diagram for explaining access priorities for memory according to one embodiment of the present disclosure.

[0016] FIG. 10 is a diagram for explaining access priorities for memory according to one embodiment of the present disclosure.

[0017] FIG. 11 is a diagram for explaining access priorities for memory according to one embodiment of the present disclosure.

[0018] FIG. 12 is a diagram for explaining access priorities for memory according to one embodiment of the present disclosure.

[0019] FIG. 13 is a diagram for explaining the processing time of a processor according to one embodiment of the present disclosure.

[0020] The terms used in this disclosure will be briefly explained, and one embodiment of the present disclosure will be specifically described.

[0021] The terms used in this disclosure are selected from widely used, current terms, taking into account the functions of one embodiment of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant embodiments of the disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of the disclosure.

[0022] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0023] Throughout this disclosure, when a part is said to "include" a certain component, this does not mean that other components are excluded, but rather that other components may be included, unless otherwise specifically stated. Furthermore, terms such as "part," "module," etc., used in this disclosure refer to a unit that processes at least one function or operation, and "part" and "module" may be implemented as hardware or software, or as a combination of hardware and software.

[0024] It should be understood that the blocks and combinations of flowcharts in each flowchart can be executed by one or more computer programs containing computer-executable instructions. The one or more computer programs may be stored entirely in a single memory, or may be stored in separate portions across multiple different memories.

[0025] Unless the context clearly dictates otherwise, singular forms (e.g., "a," "an," and "the") are to be understood to include plural referents. Thus, for example, the description "a component surface" may also include reference to one or more of such surfaces.

[0026] All functions or operations described in this document may be performed by a single processor or a combination of processors. A single processor or a combination of processors is a circuitry that performs processing, and may include circuitry such as an Application Processor (AP), a Communication Processor (CP), a Graphical Processing Unit (GPU), a Neural Processing Unit (NPU), a Microprocessor Unit (MPU), a System on Chip (SoC), or an Integrated Chip (IC).

[0027] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present disclosure. However, one embodiment of the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted to clearly describe one embodiment of the present disclosure, and similar parts are designated with similar drawing reference numerals throughout the present disclosure.

[0028] FIG. 1 is a diagram illustrating a system for processing stream data according to one embodiment of the present disclosure.

[0029] According to one embodiment of the present disclosure, stream data may refer to data that is continuously transmitted through an interface. For example, stream data may refer to data that flows as the smallest unit of individual data (e.g., 1 pixel in display). Stream data may include, but is not limited to, image data, audio data, video data, multimedia data, spatiotemporal data (e.g., AR / VR content), etc. For convenience of explanation, the following description will be made using an example in which the stream data is image data. According to one embodiment of the present disclosure, stream data may also have channel data. For example, if the stream data is image data, the stream data may include R channel pixels, G channel pixels, and B channel pixels.

[0030] According to one embodiment of the present disclosure, stream data received from the outside may be stored in a buffer and then transmitted to memory (200). In addition, stream data stored in the memory (200) and processed by the processor may be output to the outside through the buffer. The buffer may be a space that temporarily stores data while transmitting the data from one place to another. Since the processor cannot process stream data immediately, the stream data temporarily stored in the buffer may be transmitted to the memory (200) for processing by the processor. Hereinafter, stream data received from the outside will be defined as input stream data, and stream data output to the outside will be defined as output stream data. In addition, the buffer that stores the input stream data will be defined as a stream input buffer (100), and the buffer that stores the output stream data will be defined as a stream output buffer (300).

[0031] According to one embodiment of the present disclosure, stream data can be sequentially transmitted to the stream input buffer (100) in minimum units (e.g., pixels). Here, the minimum unit (e.g., pixels) of individual data input to the stream input buffer (100) is referred to as a stream data format. That is, the input stream data can be sequentially stored in the stream input buffer (100) in a stream data format (e.g., pixels).

[0032] According to one embodiment of the present disclosure, input stream data stored in the stream input buffer (100) may be transmitted to the memory (200) in a format that can be processed by the processor. The format that the processor can process may refer to the minimum unit that the processor can process. In other words, the format that the processor can process may be a unit of data (e.g., a word) that the processor handles at one time. A word may be created by stacking unit data of stream data (e.g., a pixel) into a single block. Hereinafter, the format that the processor can process will be referred to as a processor data format.

[0033] In general, the size of a processor data format (e.g., word) may be larger than the size of a stream data format (e.g., pixel). Therefore, when input stream data in a stream data format is sequentially input, the stream input buffer (100) may convert the input stream data in the stream data format into a processor data format and transmit the converted input stream data to the memory (200). According to one embodiment of the present disclosure, converting the stream data format into a processor data format may include storing (accumulating) the input stream data that is sequentially input according to the processor data format. That is, converting the stream data format into a processor data format may include collecting unit data (e.g., pixel) input as stream data and forming it into a block form (e.g., word) of a certain form. For example, as shown in FIG. 1, a word may be 8 width X2 channel When composed of pixels, the stream input buffer (100) sequentially inputs 8 pixels, which are unit data. width x2 channel Can be stacked in the form of

[0034] The input stream data stored in the memory (200) can be processed by the processor. For example, the processor can receive the input stream data from the memory (200), perform the computational processing, and then transmit the computational processing result data to the memory (200). At this time, the computational processing result data can be transmitted to the memory (200) in a processor data format (e.g., word). The stream output buffer (300) can receive the computational processing result data in the processor data format from the memory (200), convert it into a stream data format, and output it to the outside. Converting the processor data format to the stream data format can mean dividing a large block, a word, into unit data (e.g., pixel). For example, if a word is 8 width X2 channelWhen composed of pixels, the stream output buffer (300) is 8 from the memory (200). width X2 channel It can receive and store pixels at a time, and output them one pixel at a time.

[0035] Meanwhile, a system for processing stream data must store stream data in memory (200) without omission in accordance with the synchronization signal of the internal / external interface, read the result of the computational processing back from the memory (200), and transmit it to the outside without synchronization delay. For example, a system for processing stream data can process stream data without omission in accordance with the synchronization signal by using a line buffer method.

[0036] The line buffer method may be a method of storing and converting the entire line data when converting an input stream data format (e.g., pixel) to a processor data format (e.g., word), and converting and exporting the entire line data when converting the processor data format (e.g., word) to an output stream data format (e.g., pixel). Here, the line data may refer to the entire data transmitted between synchronization signals. The amount of line data may be fixed. For example, the entire pixels (e.g., 1920 pixels) between the first synchronization signal and the second synchronization signal may be 1 line data.

[0037] In the case of the line buffer method, the stream input buffer (100) can store the entire line data in a stream data format (e.g., pixel), convert the entire line data into a processor data format (e.g., word), and transmit it to the memory (200). At this time, the size of the line data and word can be expressed as the product of the width and the channel, but is not limited thereto. The size of the line data and word can also be expressed as the product of the width and the channel and the height.

[0038] 1 Input Line Data = Width input X Channel input

[0039] 1 Word = Width input X Channel input

[0040] Let us define an equation in which N line data (N Input Stream Line Data) and M words (M words) are mutually converted in an interface. In the following, the interface may be a stream input buffer (100) and / or a stream output buffer (300). Since width information and channel information cannot be mutually substituted, N and M are respectively N width * N channel , M width * M channel can be expressed as

[0041] N width XN channel X Input Line Data = M width XM channel X words

[0042] In the case of the line buffer method, all of the required N line data must be stored in the stream input buffer (100) and then replaced with M word data, or all of the M word data must be stored in the stream output buffer (300) and then replaced with N line data. Therefore, a relatively large buffer size is required, which has the disadvantage of increasing the chip area.

[0043] In addition, in the case of the line buffer method, there is a disadvantage that the exclusive occupation time for the memory (200) of the interface increases while transmitting the entire line data to the memory (200). For example, the total time required for one input stream line data (1 Input Stream Line Data) to be transmitted from the stream input buffer (100) to the memory (200) is as follows. This is because the stream input buffer (100) converts the line data into word units and then transmits it to the memory (200).

[0044] T whole,input = Input Stream Line Data / word =

[0045] Additionally, the total time required for one output stream line data (1 Output Stream Line Data) to be transmitted from the memory (200) to the stream output buffer (300) is as follows.

[0046] T whole,output = Output Stream Line Data / word =

[0047] Therefore, the total occupancy time for the memory (200) that the interface (stream input buffer (100) and stream output buffer (300)) must access starting from the synchronization signal is T access = T whole, input + T whole, output am.

[0048] For flexible processing of the processor, the word size must become smaller. However, according to the above formula, as the word size becomes smaller, the total occupancy time (T) for the memory (200) of the interface becomes shorter. access ) increases. Therefore, in the current development direction of the processor, the line buffer method has a disadvantage in that the total occupancy time for the interface memory (200) will inevitably continue to increase.

[0049] To compensate for the above shortcomings of the line buffer method, two more methods can be considered.

[0050] First, to address the drawback of increasing buffer size in the line buffer method, a method called the Priority to Interface Method can be considered. With this method, line data can be split into smaller units for use. In other words, only the smallest unit that can be transmitted can be used as a buffer.

[0051] N width XN channel X Width input X Channel input

[0052] = M width XM channel X Width word X Channel word

[0053]

[0054] For external stream data, transmission is possible by splitting it into 1-pixel units. However, since words cannot be split into smaller words, N / M can only have values ​​greater than or equal to 1. Therefore, the minimum stream input buffer unit for converting stream data format to processor data format can be as follows.

[0055] Interface Input Buffer =

[0056] The method of giving priority to the interface has the advantage of being able to use the minimum size buffer for data format conversion, but since the memory (200) must be forcibly occupied every time the buffer becomes full for data transmission, there is still a disadvantage that the processor cannot use the memory (200) during the time the interface forcibly occupies the memory (200).

[0057] The time it takes for the interface and memory (200) to transfer 1 word per access is T unit Assuming that, the time (T) for the interface to access memory (200) access ) is the number of accesses and T unit It can be expressed as a product of .

[0058] T access = Num access XT unit (1 word transaction per 1 access)

[0059] Secondly, to address the drawback of increased memory occupancy time of the interface, a method of granting priority to the processor (Priority to Processor Method) can be considered. The Priority to Processor Method can be a method in which the processor always has priority access to the memory (200), and the interface only accesses the memory (200) when the processor is not using the memory (200).

[0060] The processor accesses the memory (200) to perform each operation, but there are brief periods where the memory (200) is not accessed. For example, the processor may not access the memory (200) while processing data obtained from the memory (200) or when the type of operation changes. Therefore, the Priority to Processor Method has the advantage that the interface does not interfere with the processor's access to the memory (200). However, since it is not possible to know exactly when the processor will not access the memory (200), there still exists a disadvantage that the buffer size must be continuously maintained large to prepare for the worst case scenario.

[0061] Therefore, according to one embodiment of the present disclosure, in order to improve the performance degradation of the processor due to the access of the memory (200) of the interface and to optimize the chip area, a hardware structure including a separate controller that manages access rights to the memory (200) between the interface and the processor can be provided. The controller according to one embodiment of the present disclosure can complement the shortcomings of the line buffer method, the priority to interface method, and the priority to processor method by dynamically controlling the access rights to the memory (200) of the interface and the processor.

[0062] Below, with reference to Fig. 2, we will examine in detail a hardware structure that reduces the overall processing time of stream data by dynamically adjusting the memory (200) occupancy time of the interface while reducing the buffer size.

[0063] FIG. 2 is a block diagram illustrating an electronic device (1000) that processes stream data according to one embodiment of the present disclosure.

[0064] An electronic device (1000) according to one embodiment of the present disclosure may be a device that processes stream data. For example, the electronic device (1000) may include, but is not limited to, a digital TV, an Augmented Reality (AR) device, a Virtual Reality (VR) device, an automobile control device (e.g., an Advanced Driver-Assistance System (ADAS)), a wearable device, a mobile terminal (e.g., a smartphone, a tablet PC), a PC, etc.

[0065] According to one embodiment of the present disclosure, an electronic device (1000) may include, but is not limited to, a stream input buffer (100), a memory (200), a stream output buffer (300), a processor (400), and a controller (500). According to one embodiment of the present disclosure, the stream input buffer (100), the memory (200), the stream output buffer (300), the processor (400), and the controller (500) may be configured as a system on a chip (SoC). Each configuration will be described in turn.

[0066] According to one embodiment of the present disclosure, the stream input buffer (100) may include a minimum unit buffer area and an extra buffer area. The minimum unit buffer area may be a minimum unit size buffer for converting between a stream data format and a processor data format. The minimum unit buffer area may have a fixed size. For example, the size of the minimum unit buffer area may be an integer multiple of a word size, which is the smallest unit that the processor (400) can process. Accordingly, when the minimum unit buffer area is full, an integer multiple of a word size may be generated. For example, when 1 line of data is 1920 width X 16 channels and 1 word is 192 width X 2 channels, the size of the minimum unit buffer area may be (192 width X 2 channels) X 8. That is, the size of the minimum unit buffer area may be 8 times the word size. At this time, when the minimum unit buffer area is full, the input stream data can be converted into 8 words. When the stream input buffer (100), the stream output buffer (300), the minimum unit buffer area, or the additional buffer area is "fully occupied," it means that the storage has reached the upper limit for which the buffer is configured to store data. Therefore, referring to the storage space "available" in the buffer means availability for the upper limit for which the buffer is configured to store data.

[0067] The additional buffer area may be a buffer space for storing additional incoming input stream data when access to the memory (200) of the stream input buffer (100) is not granted even though the minimum unit buffer area is full. The additional buffer area may have various sizes. For example, the additional buffer area may be smaller or larger than the minimum unit buffer area. Furthermore, the size of the additional buffer area may not be proportional to the word size. Meanwhile, the size of the additional buffer area may be proportional to the time for which the stream input buffer (100) can maintain a low access priority to the memory (200). For example, the longer the time for which the stream input buffer (100) can maintain a low access priority to the memory (200), the larger the size of the additional buffer area may be. Therefore, the user can set a desired processing time by adjusting the size of the stream input buffer (100).

[0068] According to one embodiment of the present disclosure, the stream input buffer (100) can sequentially store input stream data in a stream data format input from the outside. For example, the stream input buffer (100) can sequentially store input stream data starting from the minimum unit buffer area. When the minimum unit buffer area is full, the stream input buffer (100) can sequentially store input stream data in an additional buffer area.

[0069] According to one embodiment of the present disclosure, the stream input buffer (100) can convert input stream data into a processor data format that can be processed by the processor (400). The processor data format may include a word format, which is the smallest unit that can be processed by the processor (400). Therefore, according to one embodiment of the present disclosure, the stream input buffer (100) can convert input stream data input in pixel units into word units. Here, converting the stream data format (pixel) into the processor data format (word) may include stacking each pixel in word format. For example, the stream input buffer (100) may stack sequentially input pixels in the minimum unit buffer area in the width and channel directions. At this time, since the size of the minimum unit buffer area is an integer multiple of a word, when the input stream data is filled in the minimum unit buffer area, the input stream data stored in the minimum unit buffer area can be naturally converted into the processor data format (word).

[0070] According to one embodiment of the present disclosure, when the stream input buffer (100) is granted access to the memory (200), it can transmit input stream data converted into a processor data format to the memory (200). That is, according to one embodiment of the present disclosure, the stream input buffer (100) can access the memory (200) under the control of the controller (500). For example, when the stream input buffer (100) is granted access to the memory (200) from the controller (500), it can convert the input stream data stored in the minimum unit buffer area into a processor data format and transmit it to the memory (200). On the other hand, when the stream input buffer (100) is not granted access to the memory (200) from the controller (500), it cannot transmit the input stream data to the memory (200) and can sequentially store the input stream data in an additional buffer area.

[0071] According to one embodiment of the present disclosure, the stream input buffer (100) can continuously transmit status information (e.g., free space, storage status, etc.) of the stream input buffer (100) to the controller (500). For example, when the minimum unit buffer area is full with input stream data, the stream input buffer (100) can transmit information that the minimum unit buffer area is full to the controller (500). Since the size of the minimum unit buffer area is an integer multiple of the word size, the state in which the minimum unit buffer area is full may be a state in which input stream data can be converted into a processor data format. In addition, the stream input buffer (100) can also transmit information to the controller (500) that the minimum unit buffer area is full and the additional buffer area has reached a threshold state. The stream input buffer (100) can also transmit information to the controller (500) that there is free storage space in the minimum unit buffer area or the additional buffer area.

[0072] According to one embodiment of the present disclosure, the stream input buffer (100) may transmit a signal to the controller (500) requesting to increase the access priority to the memory (200). For example, when the minimum unit buffer area is full and the additional buffer area has reached a critical state, the stream input buffer (100) may transmit a signal to the controller (500) requesting to increase the access priority to the memory (200). The fact that the additional buffer area of ​​the stream input buffer (100) has reached a critical state may mean that the free storage space of the additional buffer area has become below a critical value. Since the stream input buffer (100) is about to become full and thus input stream data storage will soon be impossible, the stream input buffer (100) may transmit a signal to the controller (500) requesting to increase the access priority to the memory (200) of the stream input buffer (100) to the maximum. In this case, the stream input buffer (100) is granted access to the memory (200) from the controller (500) and can access the memory (200). Accordingly, the stream input buffer (100) can convert the input stream data stored in the minimum unit buffer area into a processor data format and transmit it to the memory (200). In this case, since the minimum unit buffer area of ​​the stream input buffer (100) becomes empty, the stream input buffer (100) can move the input stream data stored in the additional buffer area to the minimum unit buffer area. In addition, the stream input buffer (100) can sequentially store the input stream data in the remaining minimum unit buffer areas.

[0073] The memory (200) may store a program for processing and controlling the processor (400), and may also store input / output data. For example, the memory (200) may store stream data converted into a processor data format (e.g., word format) in the stream input buffer (100). In addition, the memory (200) may store computational processing result data for the stream data of the processor (400), and may store intermediate computational result data of the processor (400). The memory (200) may transfer stream data in a processor data format to the processor (400), and may also transfer computational processing result data in a processor data format to the stream output buffer (300).

[0074] The memory (200) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0075] The memory (200) may not exist separately and may be configured to be included in the processor (400). The memory (200) may be configured as volatile memory, non-volatile memory, or a combination of volatile memory and non-volatile memory. The memory (200) may store a program or at least one instruction for performing operations according to an embodiment of the present disclosure. The memory (200) may also provide stored data to the processor (400) upon request from the processor (400).

[0076] The stream output buffer (300) may include a minimum unit buffer area and an extra buffer area. The minimum unit buffer area may be a minimum unit size buffer for converting between a processor data format and a stream data format. The minimum unit buffer area may have a fixed size. For example, the size of the minimum unit buffer area may be an integer multiple of a word size, which is the smallest unit that the processor (400) can process. The extra buffer area of ​​the stream output buffer (300) may be a buffer area that receives data to be exported to the outside from the minimum unit buffer area in advance so that the minimum unit buffer area can receive data in the processor data format (word) from the memory (200). The extra buffer area may have various sizes. For example, the extra buffer area may be smaller or larger than the minimum unit buffer area. In addition, the size of the extra buffer area may not be proportional to the word size. Meanwhile, the size of the additional buffer area of ​​the stream output buffer (300) may be different from the size of the additional buffer area of ​​the stream input buffer (100).

[0077] According to one embodiment of the present disclosure, the stream output buffer (300) can store the computational processing result data in the processor data format received from the memory (200). For example, when the stream output buffer (300) receives the computational processing result data from the memory (200), the stream output buffer (300) can first store the computational processing result data in the processor data format (e.g., word format) in the minimum unit buffer area. Since the minimum unit buffer area is an integer multiple of the word size, when the computational processing result data in the word format is stored in the minimum unit buffer area, the minimum unit buffer area may become full. In addition, the stream output buffer (300) can push the computational processing result data stored in the minimum unit buffer area to an additional buffer area as much as possible and store it across the minimum unit buffer area and the additional buffer area.

[0078] According to one embodiment of the present disclosure, the stream output buffer (300) can convert the operation processing result data into a stream data format and sequentially output it to the outside. For example, if the operation processing result data is stored across a minimum unit buffer area and an additional buffer area, the stream output buffer (300) can convert the operation processing result data stored in the minimum unit buffer area into a stream data format (e.g., pixel unit) and sequentially output it to the outside. Generally, since the processor data format (e.g., word) is larger than the stream data format (e.g., pixel), converting the processor data format into the stream data format may include dividing data in the stream data format to fit the stream data format. For example, if a word is composed of 10 pixels, the stream output buffer (300) can receive one word from the memory (200), store 10 pixels, and output one pixel at a time to the outside.

[0079] According to one embodiment of the present disclosure, the stream output buffer (300) can continuously transmit status information (e.g., free space, storage status, etc.) of the stream output buffer (300) to the controller (500). For example, the stream output buffer (300) can transmit information that the minimum unit buffer area and the additional buffer area are empty to the controller (500). The stream output buffer (300) can transmit information that the minimum unit buffer area is empty to the controller (500). Since the size of the minimum unit buffer area is an integer multiple of the word size, the state in which the minimum unit buffer area is empty may be a state in which the computational processing result data of the processor data format (word) can be received. The stream output buffer (300) can transmit information that the minimum unit buffer area is empty and the additional buffer area is in a critical state to the controller (500). The fact that the additional buffer area is in a critical state may mean that there is almost no data left to be output to the outside in the additional buffer area. The stream output buffer (300) may also transmit information to the controller (500) that the minimum unit buffer area or additional buffer area is full.

[0080] According to one embodiment of the present disclosure, the stream output buffer (300) can receive the computational processing result data in the processor data format from the memory (200) by being granted access to the memory (200) by the controller (500). For example, if the stream output buffer (300) is granted access to the memory (200) by the controller (500), it can access the memory (200). At this time, the stream output buffer (300) can receive the computational processing result data in the processor data format from the memory (200) and store it in the minimum unit buffer area. On the other hand, if the stream output buffer (300) is not granted access to the memory (200) by the controller (500), it cannot additionally receive the computational processing result data from the memory (200). For example, if the minimum unit buffer area is empty but access to the memory (200) is not granted from the controller (500), the stream output buffer (300) can convert the operation processing result data stored in the additional buffer area into a stream data format and sequentially output it to the outside.

[0081] The stream output buffer (300) may transmit a signal to the controller (500) to increase the access priority to the memory (200) when the minimum unit buffer area is empty and the additional buffer area has reached a critical state. For example, the stream output buffer (300) may transmit a signal to the controller (500) to increase the access priority of the stream output buffer (300) to the memory (200) to the maximum because the data stored in the minimum unit buffer area and the additional buffer area is about to run out. The stream output buffer (300) may be forcibly granted access to the memory (200) by the controller (500). In this case, the stream output buffer (300) may access the memory (200) and receive the result data of the computation in the processor data format from the memory (200).

[0082] The processor (400) can write data to the memory (200) or read data stored in the memory (200). In particular, it can process data according to predefined operation rules or artificial intelligence models by executing a program or at least one instruction stored in the memory (200).

[0083] The processor (400) can perform a function of processing data. For example, the processor (400) can obtain input stream data converted into a processor data format from the memory (200). In addition, the processor (400) can perform computational processing on the input stream data converted into the processor data format. The processor (400) can store the computational processing result data on the input stream data converted into the processor data format in the memory (200) in the processor data format. In addition, the processor (400) can also execute an OS (Operating System) and various applications stored in the memory (200) when a preset condition is satisfied.

[0084] The processor (400) may be composed of one or more processors. The one or more processors included in the processor (400) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. The one or more processors included in the processor (400) may be a general-purpose processor such as a Central Processing Unit (CPU), a Micro Processor Unit (MPU), an Application Processor (AP), a Digital Signal Processor (DSP), a graphics-only processor such as a Graphics Processing Unit (GPU), a Vision Processing Unit (VPU), an artificial intelligence-only processor such as a Neural Processing Unit (NPU), or a communication-only processor such as a Communication Processor (CP). When the one or more processors included in the processor (400) are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model. The processor (400) may be implemented as a single core processor or a multicore processor.

[0085] According to one embodiment of the present disclosure, in order to process stream data, each of a stream input buffer (100), a stream output buffer (300), and a processor (400) must access a memory (200). However, it is impossible for the stream input buffer (100), the stream output buffer (300), and the processor (400) to access the same memory address simultaneously. Therefore, it is necessary to manage access priorities for the memory (200) among the stream input buffer (100), the stream output buffer (300), and the processor (400).

[0086] A controller (500) according to one embodiment of the present disclosure may be configured to control access priorities for memory (200) between an interface and a processor (400). For example, the controller (500) may grant access rights to the memory (200) of a stream input buffer (100), a processor (400), and a stream output buffer (300) based on access priorities for the memory (200). Hereinafter, the controller (500) may be expressed as a priority controller, and the stream input buffer (100), the processor (400), and the stream output buffer (300) may be expressed as modules.

[0087] According to one embodiment of the present disclosure, the controller (500) can determine access rights to the memory (200) of each module based on the status information of each module or the request of each module. For example, the controller (500) can receive status information from each of the processor (400), the stream input buffer (100), and the stream output buffer (300). In addition, the controller (500) can determine access rights to the memory (200) of the processor (400), the stream input buffer (100), and the stream output buffer (300) based on the status of the processor (400), the status of the stream input buffer (100), and the status of the stream output buffer (300). For example, the controller (500) may determine an access priority for the memory (200) among the processor (400), the stream input buffer (100), and the stream output buffer (300) based on the state of the processor (400), the state of the stream input buffer (100), and the state of the stream output buffer (300), and grant access to the memory (200) to the module with the highest access priority.

[0088] According to one embodiment of the present disclosure, when the controller (500) receives a request signal requesting access to the memory (200) from one of the modules of the processor (400), the stream input buffer (100), and the stream output buffer (300), the controller (500) may grant access to the memory (200) to the module that transmitted the request signal. For example, when only the stream input buffer (100) has requested access to the memory (200) at the current point in time, the controller (500) may grant access to the memory (200) to the stream input buffer (100).

[0089] According to one embodiment of the present disclosure, when the controller (500) receives a request signal requesting access to the memory (200) from two or more modules among the processor (400), the stream input buffer (100), and the stream output buffer (300), the controller (500) may grant access to the memory (200) to a module having a higher access priority to the memory (200) among the modules that transmitted the request signal.

[0090] In general, the processor (400) may have a higher priority for accessing the memory (200) than the stream input buffer (100) and the stream output buffer (300). In addition, the smaller the buffer size, the higher the priority for accessing the memory (200). That is, when the stream input buffer (100) and the stream output buffer (300) are in a normal state and the size of the stream input buffer (100) is smaller than that of the stream output buffer (300), the priority may be determined in the order of the processor (400), the stream input buffer (100), and the stream output buffer (300). The fact that the stream input buffer (100) is in a normal state may mean that there is free storage space in the additional buffer area or that the additional buffer area is not in a critical state. The fact that the stream output buffer (300) is in a normal state may mean that there is enough data to be consumed in the additional buffer area or that the additional buffer area is not in a critical state. Accordingly, when the stream input buffer (100) in a normal state and the processor (400) simultaneously request access to the memory (200), the controller (500) can allow access to the processor (400). In addition, when the processor (400) is occupying the memory (200) when the stream input buffer (100) requests access to the memory (200), the controller (500) can disallow access of the stream input buffer (100) to the memory (200). For example, when the controller (500) receives information from the stream input buffer (100) that the minimum unit buffer area is full but there is free space in the additional buffer area, the controller (500) can determine access rights of the stream input buffer (100) to the memory (200) based on the state of the processor (400).When the processor (400) occupies the memory (200), the controller (500) may not grant the stream input buffer (100) access to the memory (200), and when the processor (400) does not occupy the memory (200), the controller (500) may grant the stream input buffer (100) access to the memory (200). In addition, when the controller (500) receives information from the stream output buffer (300) that the additional buffer area contains operation processing result data but the minimum unit buffer area is empty, the controller (500) may determine the access rights to the memory (200) of the stream output buffer (300) based on the state of the processor (400). That is, when the processor (400) occupies the memory (200), the controller (500) may not grant the stream output buffer (300) access to the memory (200), and when the processor (400) does not occupy the memory (200), the controller (500) may grant the stream output buffer (300) access to the memory (200).

[0091] According to one embodiment of the present disclosure, when the stream input buffer (100) or the stream output buffer (300) is in a critical state, the controller (500) may determine the highest access priority for the memory (200) of the stream input buffer (100) or the stream output buffer (300). For example, when both the minimum unit buffer area and the additional buffer area of ​​the stream input buffer (100) are about to be full, the stream input buffer (100) may transmit a signal to the controller (500) requesting an increase in the access priority for the memory (200). At this time, the controller (500) may adjust the access priority of the stream input buffer (100) to the highest. Accordingly, when the processor (400) is occupying the memory (200), the controller (500) may suspend the access of the processor (400) and then allow the access of the stream input buffer (100) to the memory (200). In addition, when the data resulting from the operation processing of the minimum unit buffer area and the additional buffer area of ​​the stream output buffer (300) is about to run out, the stream output buffer (300) can transmit a signal to the controller (500) requesting to increase the access priority to the memory (200). At this time, the controller (500) can adjust the access priority of the stream output buffer (300) to the highest. Accordingly, when the processor (400) is occupying the memory (200), the controller (500) can suspend the access of the processor (400) and then allow the stream output buffer (300) to access the memory (200).

[0092] According to one embodiment of the present disclosure, when both the stream input buffer (100) and the stream output buffer (300) are in a critical state, the controller (500) may determine a higher access priority for a buffer with a smaller size among the stream input buffer (100) and the stream output buffer (300). For example, when the controller (500) simultaneously receives a signal requesting to increase the access priority for the memory (200) from the stream input buffer (100) and the stream output buffer (300), the controller (500) may allow the buffer with a smaller size among the stream input buffer (100) and the stream output buffer (300) to access the memory (200). Accordingly, when the stream input buffer (100) is smaller in size than the stream output buffer (300), the controller (500) may determine the access priorities in the order of the stream input buffer (100), the stream output buffer (300), and the processor (400).

[0093] According to one embodiment of the present disclosure, the controller (500) maximizes the access of the processor (400) to the memory (200) and, only when the stream input buffer (100) or the stream output buffer (300) reaches a critical state, raises the access priority of the stream input buffer (100) or the stream output buffer (300) to stop the processor (400) from accessing the memory (200), thereby minimizing the processing time of the processor (400). That is, according to one embodiment of the present disclosure, by separately arranging the controller (500) that manages the access priority to the memory (200), it is possible to minimize the performance degradation of the processor (400) due to the interface's access to the memory (200). In addition, according to one embodiment of the present disclosure, it is possible to adaptively respond to critical situations by providing additional buffer areas in the stream input buffer (100) and the stream output buffer (300), and since a buffer of a size close to the minimum unit buffer area is used, the chip area can be optimized.

[0094] FIG. 3 is a drawing for explaining the configuration of a stream input buffer (100) and a stream output buffer (300) according to one embodiment of the present disclosure.

[0095] An electronic device (1000) according to one embodiment of the present disclosure can apply a dynamic priority method (320) that combines the advantages of a method of granting priority to a processor (400) (Priority to Processor Method) (310) and a method of granting priority to an interface (Priority to Interface Method) (330).

[0096] In the case of the method (310) of giving priority to the processor (400), in order to not interfere with the processor's (400) access to the memory (200) as much as possible, the buffer may have a space capable of storing the entire line data. For example, the buffer of the method (310) of giving priority to the processor (400) may have M minimum unit buffer areas (301) for converting the stream data format into the processor data format according to the line data size. If the line data consists of 1920 pixels and the processor data format (word) consists of 192 pixels, the buffer of the method (310) of giving priority to the processor (400) may have 10 minimum unit buffer areas (301). In the case of the method (310) of giving priority to the processor (400), the size of the buffer may increase, but there is an advantage of minimizing the performance degradation of the processor (400).

[0097] In the case of the method (330) of giving priority to the interface, the buffer may include one minimum unit buffer area (301) for converting the stream data format into the processor data format. That is, according to the method (330) of giving priority to the interface, the buffer can directly access the memory (200) when the minimum stream data required for conversion into the processor data format is stored. Therefore, according to the method (330) of giving priority to the interface, the performance of the processor (400) may be degraded, but there is an advantage in that the buffer size may be minimized.

[0098] A dynamic priority method (320) according to one embodiment of the present disclosure may combine the advantages of a method (310) of granting priority to a processor (400) (e.g., minimizing performance degradation of the processor (400)) and a method (330) of granting priority to an interface (e.g., minimizing buffer size). According to the dynamic priority method (320), the stream input buffer (100) and the stream output buffer (300) may include one minimum unit buffer area (301) and one additional buffer area (302). The minimum unit buffer area (301) has a fixed size, but the additional buffer area (302) may have various sizes. The additional buffer area (302) may be smaller or larger than the minimum unit buffer area (301). If the additional buffer area (302) is larger than the minimum unit buffer area (301), the stream input buffer (100) and the stream output buffer (300) may include two or more minimum unit buffer areas (301). Therefore, according to one embodiment of the present disclosure, when designing hardware, the size of the additional buffer area (302) is appropriately set in advance according to the network desired by the user, so that an interface optimized for the appropriate size of the chip and the performance of the processor (400) suitable for the system desired by the user can be configured.

[0099] According to one embodiment of the present disclosure, the stream input buffer (100) can store additional incoming stream data in an additional buffer area (302) when access to the memory (200) is not granted even though the minimum unit buffer area (301) is full of stream data. The stream output buffer (300) can move data to be exported to the outside from the minimum unit buffer area (301) to the additional buffer area (302) so that data (words) in a processor data format can be received from the memory (200). Therefore, according to the dynamic priority method (320) according to one embodiment of the present disclosure, by providing an additional buffer area (302), it is possible to minimize performance degradation of the processor (400) similar to the method (310) of granting priority to the processor (400). In addition, according to the dynamic priority method (320) according to one embodiment of the present disclosure, a buffer having a size almost similar to the minimum unit buffer area (301) can be used, so that the chip area can be minimized, as in the method of giving priority to an interface (330).

[0100] FIG. 4 is a diagram for explaining the operation of a stream input buffer (100) according to one embodiment of the present disclosure.

[0101] According to one embodiment of the present disclosure, the stream input buffer (100) may include a minimum unit buffer area (401) and an additional buffer area (402).

[0102] In the first state (State 1) (410), both the minimum unit buffer area (401) and the additional buffer area (402) of the stream input buffer (100) may be empty. At this time, when input stream data is sequentially input from the outside, the stream input buffer (100) may be switched to the second state (State 2) (420). In the second state (420), the stream input buffer (100) may sequentially store input stream data (e.g., pixel units) input from the outside in the minimum unit buffer area (401).

[0103] The third state (430) may be a state in which the minimum unit buffer area (401) is filled with input stream data (e.g., pixels). That is, the third state (430) may be a state in which conversion into a processor data format (e.g., a word) is possible, and data in the processor data format can be recorded in the memory (200). For example, if a word is composed of 192 pixels, data that is an integer multiple of 192 pixels may be filled in the minimum unit buffer area (401).

[0104] In the third state (430), the stream input buffer (100) can transmit information to the controller (500) that the minimum unit buffer area (401) is full of input stream data. If the processor (400) is not occupying the memory (200), the controller (500) can grant the stream input buffer (100) access to the memory (200). In this case, the stream input buffer (100) can convert the input stream data into a processor data format and write it to the memory (200). For example, the stream input buffer (100) can transmit all pixels stored in the minimum unit buffer area (401) to the memory (200) by configuring them into one or more words. If the stream input buffer (100) transmits the input stream data stored in the minimum unit buffer area (401) to the memory (200), the stream input buffer (100) can be switched to the first state (410).

[0105] Meanwhile, when the stream input buffer (100) is in the third state (430) and the processor (400) occupies the memory (200), the controller (500) may not grant the stream input buffer (100) access to the memory (200). In this case, the stream input buffer (100) may sequentially store input stream data in the additional buffer area (402). When the processor (400) temporarily suspends the occupation of the memory (200) while the stream input buffer (100) stores the input stream data in the additional buffer area (402), the controller (500) may grant the stream input buffer (100) access to the memory (200). In this case, the stream input buffer (100) may convert the input stream data stored in the minimum unit buffer area (401) into a processor data format and write it to the memory (200). And the stream input buffer (100) can move the input stream data stored in the additional buffer area (402) to the minimum unit buffer area (401) and switch to the second state (420).

[0106] The fourth state (440) may be a state in which the stream input buffer (100) cannot continuously access the memory (200), and the minimum unit buffer area (401) and the additional buffer area (402) are about to become full (i.e., the additional buffer area (402) has reached a critical state). The stream input buffer (100) may transmit a signal to the controller (500) requesting an increase in the access priority to the memory (200) because it will soon no longer be able to store input stream data. The stream input buffer (100) may, taking into account the access time to the memory (200), transmit a signal to the controller (500) requesting an increase in the access priority to the memory (200) before the additional buffer area (402) becomes full. When the controller (500) receives a signal from the stream input buffer (100) to increase the access priority to the memory (200), it can stop the processor (400) from accessing the memory (200) and allow the stream input buffer (100) to access the memory (200). After the stream input buffer (100) is forcibly granted high memory access rights by the controller (500), it can convert the input stream data stored in the minimum unit buffer area (401) into a processor data format and write it to the memory (200). While the stream input buffer (100) accesses the memory (200), the additional buffer area (402) may become full.

[0107] The fifth state (450) represents a state in which only the additional buffer area (402) is full of input stream data after the input stream data stored in the minimum unit buffer area (401) is transmitted to the memory (200). At this time, the stream input buffer (100) can move the input stream data stored in the additional buffer area (402) to the minimum unit buffer area (401) for conversion into a later processor data format and switch to the second state (420). In the second state (420), the stream input buffer (100) can sequentially store input stream data coming from the outside in the minimum unit buffer area (401).

[0108] Although FIG. 4 illustrates an example where the additional buffer area (402) is smaller than the minimum unit buffer area (401), the present disclosure is not limited thereto. The additional buffer area (402) may be larger than the minimum unit buffer area (401). Furthermore, it should be understood that the fact that either the minimum unit buffer area (401) or the additional buffer area (402) is “fully occupied” does not necessarily mean that the buffer has reached its maximum occupancy capacity. Rather, in some cases, the minimum unit buffer area (401) or the additional buffer area (402) may be “full” when it reaches the upper limit at which the described operation is performed. FIG. 4 illustrates a case where the minimum unit buffer area (401) and the additional buffer area (402) are each represented as separate buffer areas. However, the present disclosure is not limited thereto, and the minimum unit buffer area (401) or the additional buffer area (402) may include two or more buffer areas.

[0109] According to one embodiment of the present disclosure, the stream input buffer (100) can access the memory (200) at intervals when the processor (400) is not occupying the memory (200), thereby minimizing the interference with the processor's (400) access to the memory (200).

[0110] FIG. 5 is a diagram for explaining the operation of a stream output buffer (300) according to one embodiment of the present disclosure.

[0111] According to one embodiment of the present disclosure, the stream output buffer (300) may include a minimum unit buffer area (501) and an additional buffer area (502).

[0112] In the first state (State 1) (510), both the minimum unit buffer area (501) and the additional buffer area (502) of the stream output buffer (300) may be empty. In the first state (510), since there is no data to be transmitted to the outside, the stream output buffer (300) may transmit a signal to the controller (500) requesting to increase the access priority to the memory (200). The controller (500) may grant the stream output buffer (300) access to the memory (200). When the stream output buffer (300) is granted access to the memory (200), it may read data in a processor data format (e.g., word format) from the memory (200). The data in the processor data format may be the result data of an operation processed by the processor (400) on input stream data written to the memory (200) from the stream input buffer (100).

[0113] In the second state (State 2) (520), the stream output buffer (300) can store the computational processing result data in the processor data format read from the memory (200) in the minimum unit buffer area (501). For example, if the computational processing result data in the processor data format (e.g., word format) is composed of 192 pixel data, the stream output buffer (300) can receive a word composed of 192 pixel data at once and store the 192 pixel data in the memory (200).

[0114] In the third state (State 3) (530), the stream output buffer (300) can store the operation processing result data stored in the minimum unit buffer area (501) by pushing it as much as possible into the additional buffer area (502). The stream output buffer (300) can store the operation processing result data stored in the minimum unit buffer area (501) by pushing it as much as possible into the additional buffer area (502) in order to continuously read the operation processing result data in the processor data format from the memory (200) at a later time. When the additional buffer area (502) is smaller than the minimum unit buffer area (501), the operation processing result data may be partially stored in the minimum unit buffer area (501) in addition to the additional buffer area (502).

[0115] In the fourth state (State 4) (540), the stream output buffer (300) can sequentially convert the operation processing result data stored in the minimum unit buffer area (501) into a stream data format (e.g., pixel unit) and output it to the outside. For example, the stream output buffer (300) can transmit the operation processing result data to the outside one pixel at a time.

[0116] The fifth state (State 5) (550) may be a state in which all of the operation processing result data stored in the minimum unit buffer area (501) of the stream output buffer (300) is transmitted to the outside, so that no operation processing result data remains in the minimum unit buffer area (501). In this case, since the stream output buffer (300) is in a state in which it can receive the operation processing result data in the processor data format from the memory (200), it may transmit information to the controller (500) that the minimum unit buffer area (501) is empty. At this time, the controller (500) may grant the stream output buffer (300) access to the memory (200) when the stream input buffer (100) and the processor (400) are not accessing the memory (200). The stream output buffer (300), when granted access to the memory (200), can additionally receive the computational processing result data in the processor data format (word format) from the memory (200) and transition to the 6-1 state (State 6-1) (561). The 6-1 state (561) may be a state in which both the minimum unit buffer area (501) and the additional buffer area (502) of the stream output buffer (300) are full. In the 6-1 state (561), the stream output buffer (300) can exhaust the computational processing result data stored in the additional buffer area (502) as output stream data. For example, since the data stored in the additional buffer area (502) is older than the data stored in the minimum unit buffer area (501), the stream output buffer (300) can output the computational processing result data stored in the additional buffer area (502) in a stream data format (e.g., pixel unit). When all data stored in the additional buffer area (502) is exhausted, the stream output buffer (300) can be switched to the second state (520).Accordingly, the stream output buffer (300) can store the operation processing result data stored in the minimum unit buffer area (501) by pushing it as much as possible to the additional buffer area (502) and switch to the third state (530).

[0117] Meanwhile, in the fifth state (550), if the stream input buffer (100) or the processor (400) is occupying the memory (200), the controller (500) may not grant the stream output buffer (300) access to the memory (200). In this case, the stream output buffer (300) may exhaust the operation processing result data stored in the additional buffer area (502) as output stream data and transition to the 6-2 state (State 6-2) (562). Meanwhile, if the stream input buffer (100) and the processor (400) stop occupying the memory (200) while exhausting the operation processing result data stored in the additional buffer area (502), the controller (500) may grant the stream output buffer (300) access to the memory (200). The stream output buffer (300), when granted access to the memory (200), can receive the computational processing result data in the processor data format (word format) from the memory (200) and store it in the minimum unit buffer area (501). The stream output buffer (300) can switch to the second state (520) after exhausting all data stored in the additional buffer area (502).

[0118] The 6-2 state (562) may be a state in which the stream output buffer (300) cannot continuously access the memory (200), and thus the data to be exhausted as output stream data will soon run out (i.e., the additional buffer area (502) has reached a critical state). The stream output buffer (300) may transmit a signal to the controller (500) requesting an increase in the access priority to the memory (200). The stream output buffer (300) may, taking into account the access time to the memory (200), transmit a signal to the controller (500) requesting an increase in the access priority to the memory (200) before all the data in the additional buffer area (502) is exhausted. When the controller (500) receives a signal from the stream output buffer (300) to increase the access priority to the memory (200), it can stop the access of the stream input buffer (100) or the processor (400) to the memory (200) and allow the stream output buffer (300) to access the memory (200). After the stream output buffer (300) is forcibly granted high memory access rights by the controller (500), it can read the computational processing result data in the processor data format (e.g., word format) from the memory (200). The stream output buffer (300) can store the computational processing result data in the minimum unit buffer area (501), and while the stream output buffer (300) accesses the memory (200), it can exhaust all the data stored in the additional buffer area (502) as output stream data, thereby returning to the second state (520).

[0119] In FIG. 5, the case where the additional buffer area (502) is smaller than the minimum unit buffer area (501) is described as an example, but is not limited thereto. The additional buffer area (502) may be larger than the minimum unit buffer area (501). It should be understood that the fact that either the minimum unit buffer area (501) or the additional buffer area (502) is “fully occupied” does not necessarily mean that the buffer has reached its maximum occupancy capacity. Rather, in some cases, the minimum unit buffer area (501) or the additional buffer area (502) may be “full” when the upper limit at which the described operation is performed is reached. FIG. 5 exemplifies the case where the minimum unit buffer area (501) and the additional buffer area (502) are each represented as separate buffer areas. However, the present disclosure is not limited thereto, and the minimum unit buffer area (501) or the additional buffer area (502) may include two or more buffer areas.

[0120] According to one embodiment of the present disclosure, the stream output buffer (300) can access the memory (200) at intervals when the processor (400) is not occupying the memory (200), thereby minimizing the interference with the processor's (400) access to the memory (200).

[0121] FIG. 6 is a diagram for explaining the amount of data change in a stream input buffer (100) according to one embodiment of the present disclosure.

[0122] Referring to FIG. 6, when input stream data is continuously input into the stream input buffer (100) in section (a), the amount of data in the stream input buffer (100) may increase. The stream input buffer (100) may sequentially store input stream data starting from the minimum unit buffer area (401). When the minimum unit buffer area (401) is full of input stream data, the controller (500) may grant the stream input buffer (100) access to the memory (200) at a time when the processor (400) does not occupy the memory (200) for a while. For example, when the processor (400) does not occupy the memory (200) in section (b), the controller (500) may grant the stream input buffer (100) access to the memory (200). When the stream input buffer (100) is granted access to the memory (200), it can convert the input stream data into a processor data format (e.g., word) and transmit it to the memory (200). Therefore, in section (b), the amount of data in the stream input buffer (100) can be reduced by the unit of word.

[0123] (c) In the section, the stream input buffer (100) can store input stream data again. At this time, if the minimum unit buffer area (401) is full but the processor (400) is continuously occupying the memory (200), so that the stream input buffer (100) cannot access the memory (200), the stream input buffer (100) can store input stream data in the additional buffer area (402). Therefore, the amount of data in the stream input buffer (100) can continue to increase.

[0124] (d) If there is no more space to store input stream data inside the stream input buffer (100) in the section (if the minimum unit buffer area (401) is full and the additional buffer area (402) has reached a critical state), the controller (500) can change the priority of the memory (200) of the stream input buffer (100) from Low to High. Then, the controller (500) can stop the processor (400) from accessing the memory (200) and then allow the stream input buffer (100) to access the memory (200). The stream input buffer (100) can convert the input stream data stored in the minimum unit buffer area (401) into a processor data format (e.g., word format) and write it to the memory (200). Therefore, the amount of data in the stream input buffer (100) can be reduced by word units.

[0125] (e) Since there is free space again in the stream input buffer (100) in the section, the controller (500) can change the priority of the memory (200) of the stream input buffer (100) from High to Low and transfer access rights to the memory (200) to the processor (400).

[0126] Therefore, according to one embodiment of the present disclosure, the stream input buffer (100) can access the memory (200) only when the processor (400) does not occupy the memory (200) for a while or when the additional buffer area (402) reaches a critical state, thereby minimizing interference with the computational processing of the processor (400).

[0127] FIG. 7 is a diagram for explaining the amount of data change in a stream output buffer (300) according to one embodiment of the present disclosure.

[0128] Referring to FIG. 7, in section (a), the stream output buffer (300) can continuously exhaust the operation processing result data stored therein as output stream data. Accordingly, the amount of data in the stream output buffer (300) can be reduced. The stream output buffer (300) can convert data stored in the minimum unit buffer area (501) into a stream data format (e.g., pixel unit) and output it.

[0129] (b) If all data in the minimum unit buffer area (501) in the section is exhausted, the controller (500) can grant the stream output buffer (300) access to the memory (200) at a time when the processor (400) does not occupy the memory (200) for a while. If the stream output buffer (300) is granted access to the memory (200), it can read the computational processing result data in the processor data format (e.g., word) from the memory (200). Accordingly, the amount of data in the stream output buffer (300) can increase by the unit of word.

[0130] (c) In the section, the stream output buffer (300) can sequentially output the operation processing result data to the outside. At this time, if the operation processing result data of the minimum unit buffer area (501) is all exhausted, but the processor (400) is continuously occupying the memory (200), so that the stream output buffer (300) cannot access the memory (200), the stream output buffer (300) can exhaust the operation processing result data stored in the additional buffer area (502). Therefore, the amount of data of the stream output buffer (300) can continue to decrease.

[0131] (d) If there is no more computational processing result data to be consumed inside the stream output buffer (300) in the section (if the minimum unit buffer area (501) is empty and the additional buffer area (502) has reached a critical state), the controller (500) can change the priority of the memory (200) of the stream output buffer (300) from Low to High. Then, the controller (500) can stop the access of the processor (400) to the memory (200) and then allow the access of the stream output buffer (300) to the memory (200). At this time, the stream output buffer (300) can receive the computational processing result data in a processor data format (e.g., word format) from the memory (200) and store it in the minimum unit buffer area (501). Therefore, the amount of data of the stream output buffer (300) can increase by the unit of words.

[0132] (e) Since the data to be consumed in the stream input buffer (100) is filled in the section, the controller (500) can change the priority of the memory (200) of the stream output buffer (300) from High to Low and transfer access rights to the memory (200) to the processor (400).

[0133] Therefore, according to one embodiment of the present disclosure, the stream output buffer (300) can access the memory (200) only when the processor (400) does not occupy the memory (200) for a while or when the additional buffer area (502) reaches a critical state, thereby minimizing interference with the computational processing of the processor (400).

[0134] FIG. 8 is a flowchart illustrating a method for a controller (500) to manage access rights to a memory (200) according to one embodiment of the present disclosure.

[0135] Referring to FIG. 8, the method by which the controller (500) manages access rights to the memory (200) may include steps S810 to S830. The method by which the controller (500) manages access rights to the memory (200) is not limited to that illustrated in FIG. 8, and in one or more embodiments, steps not illustrated in FIG. 8 may be further included, or some steps may be omitted.

[0136] In step S810, the controller (500) according to one embodiment of the present disclosure may receive a request signal requesting access to status information or memory (200) from the processor (400), the stream input buffer (100), and the stream output buffer (300).

[0137] According to one embodiment of the present disclosure, the controller (500) can periodically receive status information from each of the processor (400), the stream input buffer (100), and the stream output buffer (300). Each of the processor (400), the stream input buffer (100), and the stream output buffer (300) may periodically transmit its own status information to the controller (500), or may transmit its own status information to the controller (500) when a specific event occurs. For example, the processor (400) may transmit information that it is occupying the memory (200), information that it has stopped occupying the memory (200), and the like to the controller (500). The stream input buffer (100) can transmit information that the minimum unit buffer area (401) is full, information that there is free space in the additional buffer area (402), information that the additional buffer area (402) is in a critical state, the total amount of data in the stream input buffer (100), the amount of data in the minimum unit buffer area (401), the amount of data in the additional buffer area (402), etc. to the controller (500). The stream output buffer (300) can transmit information that the minimum unit buffer area (501) is empty, information that there is data in the additional buffer area (502), information that the additional buffer area (502) is in a critical state, the total amount of data in the stream input buffer (100), the amount of data in the minimum unit buffer area (501), the amount of data in the additional buffer area (502), etc. to the controller (500). At this time, the controller (500) can continuously monitor the status of each of the processor (400), stream input buffer (100), and stream output buffer (300).

[0138] According to one embodiment of the present disclosure, the controller (500) may receive a request signal requesting access to the memory (200). For example, when the minimum unit buffer area (401) of the stream input buffer (100) is full and the additional buffer area (402) has reached a critical state (i.e., when there is almost no space to store input stream data in the stream input buffer (100), the controller (500) may receive a signal from the stream input buffer (100) to increase the access priority to the memory (200). When the minimum unit buffer area (501) of the stream output buffer (300) is empty and the additional buffer area (502) has reached a critical state (i.e., when there is almost no data to be output to the outside in the stream output buffer (300), the controller (500) may receive a signal from the stream output buffer (300) to increase the access priority to the memory (200).

[0139] Below, each of the processor (400), stream input buffer (100), and stream output buffer (300) can be expressed as a module.

[0140] In step S820, the controller (500) according to one embodiment of the present disclosure may determine an access priority for the memory (200) based on status information or a request signal of the processor (400), the stream input buffer (100), and the stream output buffer (300).

[0141] Referring to FIG. 9, according to one embodiment of the present disclosure, the controller (500) may determine a higher access priority for the memory (200) of the processor (400) than the stream input buffer (100) and the stream output buffer (300) when the stream input buffer (100) and the stream output buffer (300) are in a normal state. The normal state may mean that the additional buffer areas (501, 502) are not in a critical state. For example, the normal state of the stream input buffer (100) may be a state in which there is free space to store input stream data in the minimum unit buffer area (401) or the additional buffer area (402) of the stream input buffer (100). The normal state of the stream output buffer (300) may be a state in which the operation processing result data to be consumed as output stream data is stored with plenty of space in the minimum unit buffer area (501) or the additional buffer area (502).

[0142] Meanwhile, when both the stream input buffer (100) and the stream output buffer (300) are in a normal state, a buffer with a smaller size may be given a higher priority between the stream input buffer (100) and the stream output buffer (300). For example, when the size of the stream output buffer (300) is smaller than that of the stream input buffer (100), the priority of the stream output buffer (300) may be higher than that of the stream input buffer (100).

[0143] In conclusion, referring to FIG. 9, the controller (500) can determine the access priority for the memory (200) in the order of the processor (400), the stream output buffer (300), and the stream input buffer (100) when the stream input buffer (100) and the stream output buffer (300) are in a normal state and the stream output buffer (300) is smaller than the stream input buffer (100).

[0144] Referring to FIG. 10, the controller (500) according to one embodiment of the present disclosure may determine, when one of the stream input buffer (100) and the stream output buffer (300) is in a critical state, the highest access priority for the memory (200) of the buffer in the critical state. For example, when the stream input buffer (100) is in a critical state, the controller (500) may determine the access priorities for the memory (200) in the order of the stream input buffer (100), the processor (400), and the stream output buffer (300). The critical state of the stream input buffer (100) may mean a state in which the free space for storing input stream data in the stream input buffer (100) (i.e., the free space of the additional buffer area (402)) is less than or equal to the critical value. On the other hand, when the stream output buffer (300) is in a critical state, the controller (500) can determine the access priority for the memory (200) in the order of the stream output buffer (300), the processor (400), and the stream input buffer (100). The critical state of the stream output buffer (300) can mean a state in which the result data of the computation to be consumed as output stream data in the stream output buffer (300) is below a critical value.

[0145] Referring to FIG. 11, when both the stream input buffer (100) and the stream output buffer (300) are in a critical state, the controller (500) according to one embodiment of the present disclosure may determine a higher access priority to the memory (200) of a buffer with a smaller size among the stream input buffer (100) and the stream output buffer (300). For example, when the stream input buffer (100) is smaller than the stream output buffer (300), the controller (500) may determine the access priority to the memory (200) in the order of the stream input buffer (100), the stream output buffer (300), and the processor (400). On the other hand, when the stream output buffer (300) is smaller than the stream input buffer (100), the controller (500) may determine the access priority to the memory (200) in the order of the stream output buffer (300), the stream input buffer (100), and the processor (400).

[0146] Returning to FIG. 8 again, in step S830, the controller (500) according to one embodiment of the present disclosure may grant access to the memory (200) starting from a module with a higher access priority to the memory (200).

[0147] According to one embodiment of the present disclosure, when the controller (500) receives a request signal requesting access to the memory (200) from two or more modules among the processor (400), the stream input buffer (100), and the stream output buffer (300), the controller (500) may grant access to the memory (200) to a module having a higher access priority to the memory (200) among the modules that transmitted the request signal. For example, when the controller (500) receives a signal from the stream input buffer (100) requesting access to the memory (200) because the minimum unit buffer area (401) is full, the controller (500) may determine access to the memory (200) of the stream input buffer (100) based on the state of the processor (400) having a higher access priority to the memory (200) than the stream input buffer (100). The controller (500) may not grant access to the memory (200) of the stream input buffer (100) when the processor (400) occupies the memory (200), and may grant access to the memory (200) of the stream input buffer (100) when the processor (400) does not occupy the memory (200).

[0148] In addition, when the controller (500) receives a signal from the stream output buffer (300) requesting access to the memory (200) because the additional buffer area (502) still has the result data of the operation processing but the minimum unit buffer area (501) is empty, the controller (500) may determine the access right of the stream output buffer (300) to the memory (200) based on the state of the processor (400) having a higher access priority to the memory (200) than the stream output buffer (300). For example, when the processor (400) is occupying the memory (200), the controller (500) may not grant the stream output buffer (300) access to the memory (200), and when the processor (400) is not occupying the memory (200), the controller may grant the stream output buffer (300) access to the memory (200).

[0149] Meanwhile, if both the stream input buffer (100) and the stream output buffer (300) are in a critical state and the stream input buffer (100) is smaller than the stream output buffer (300), the controller (500) can stop the processor (400) from accessing the memory (200) and grant the stream input buffer (100) access to the memory (200). Then, if the stream input buffer (100) completes access to the memory (200) (e.g., if the input stream data converted into the processor data format is transmitted to the memory (200), the controller (500) can then grant the stream output buffer (300) access to the memory (200). When the stream output buffer (300) completes access to the memory (200) (e.g., when receiving the result data of the computation in the processor data format from the memory (200), the controller (500) can grant the processor (400) access to the memory (200) again.

[0150] Referring to Figure 12, let us examine the access priority for memory (200) in more detail.

[0151] FIG. 12 is a diagram for explaining access priorities for a memory (200) according to one embodiment of the present disclosure.

[0152] According to one embodiment of the present disclosure, since the processor (400), the stream input buffer (100), and the stream output buffer (300) cannot access the memory (200) simultaneously, the controller (500) can manage the access of the processor (400), the stream input buffer (100), and the stream output buffer (300) to the memory (200) based on a request signal or status. In FIG. 12, the access request signal to the memory (200) can include status information.

[0153] In case 1, if the controller (500) does not receive an access request signal for the memory (200) from the processor (400), the stream input buffer (100), and the stream output buffer (300), it may not grant access to the memory (200) to any of the processor (400), the stream input buffer (100), and the stream output buffer (300).

[0154] In case 2, the controller (500) may grant access to the memory (200) to the stream input buffer (100) if it receives an access request signal to the memory (200) from the stream output buffer (300) and does not receive an access request signal to the memory (200) from the processor (400).

[0155] In case 3, the controller (500) may receive an access request signal for the memory (200) from the stream output buffer (300), and if it does not receive an access request signal for the memory (200) from the stream input buffer (100) and the processor (400), it may grant the stream output buffer (300) access to the memory (200).

[0156] In case 4, if the controller (500) does not receive an access request signal for the memory (200) from the processor (400), but simultaneously receives an access request signal for the memory (200) from the stream input buffer (100) and the stream output buffer (300), the controller (500) may first grant access to the memory (200) to a buffer with a smaller size among the stream input buffer (100) and the stream output buffer (300). For example, if the stream input buffer (100) is smaller in size than the stream output buffer (300), the controller (500) may first grant access to the memory (200) to the stream input buffer (100). Thereafter, if the access of the stream input buffer (100) to the memory (200) is terminated, the controller (500) may grant access to the memory (200) to the stream output buffer (300).

[0157] In case 5, the controller (500) may receive an access request signal for the memory (200) from the processor (400), and if it does not receive an access request signal for the memory (200) from the stream input buffer (100) and the stream output buffer (300), it may grant the processor (400) access to the memory (200).

[0158] In case 6, the controller (500) may receive an access request signal for the memory (200) from the processor (400) and the stream input buffer (100). At this time, the stream input buffer (100) may be in a normal state. That is, the stream input buffer (100) may be in a state where the minimum unit buffer area (401) is full, but there is free space to store data in the additional buffer area (402). The controller (500) may determine that the access priority for the memory (200) of the processor (400) is higher than that of the stream input buffer (100), and may grant the processor (400) access to the memory (200). Thereafter, when the processor (400) terminates access to the memory (200), the controller (500) may grant the stream input buffer (100) access to the memory (200).

[0159] In case 7, the controller (500) may receive an access request signal for the memory (200) from the processor (400) and the stream output buffer (300). At this time, the stream output buffer (300) may be in a normal state. That is, the stream output buffer (300) may have an empty minimum unit buffer area (501), but may have data remaining in the additional buffer area (502) to be output to the outside. The controller (500) may determine that the access priority for the memory (200) of the processor (400) is higher than that of the stream output buffer (300), and may grant the processor (400) access to the memory (200). Thereafter, when the processor (400) terminates access to the memory (200), the controller (500) may grant the stream output buffer (300) access to the memory (200).

[0160] In case 8, the controller (500) may receive an access request signal for the memory (200) from the processor (400), the stream input buffer (100), and the stream output buffer (300). At this time, the stream input buffer (100) and the stream output buffer (300) may be in a normal state. For example, the stream input buffer (100) may be in a state where the minimum unit buffer area (401) is full, but there is free space to store data in the additional buffer area (402). The stream output buffer (300) may be in a state where the minimum unit buffer area (501) is empty, but there is data remaining in the additional buffer area (502) to be output to the outside. In this case, the controller (500) may determine that the access priority for the memory (200) of the processor (400) is higher than that of the stream input buffer (100) and the stream output buffer (300), and grant the processor (400) access to the memory (200).

[0161] In case 9, the controller (500) may not receive an access request signal for the memory (200) from the processor (400), but may receive an access request signal for the memory (200) from the stream input buffer (100) which is in a critical state. In addition, the controller (500) may receive an access request signal for the memory (200) from the stream output buffer (300) which is in a normal state, or may not receive an access request signal for the memory (200) from the stream output buffer (300). Since the stream input buffer (100) is in a critical state (e.g., a state in which the minimum unit buffer area (401) and the additional buffer area (402) are about to be full), the controller (500) may determine the highest access priority for the memory (200) of the stream input buffer (100) and grant the stream input buffer (100) access rights to the memory (200).

[0162] In case 10, the controller (500) may not receive an access request signal for the memory (200) from the processor (400), but may receive an access request signal for the memory (200) from the stream output buffer (300) in a critical state. In addition, the controller (500) may receive an access request signal for the memory (200) from the stream input buffer (100) in a normal state, or may not receive an access request signal for the memory (200) from the stream input buffer (100). Since the stream output buffer (300) is in a critical state (e.g., a state in which data to be exhausted as output stream data is about to run out), the controller (500) may determine the highest priority for access to the memory (200) of the stream output buffer (300) and grant the stream output buffer (300) access to the memory (200).

[0163] In case 11, the controller (500) may receive an access request signal for the memory (200) from the stream input buffer (100) in a critical state and the stream output buffer (300) in a critical state, without receiving an access request signal for the memory (200) from the processor (400). At this time, the controller (500) may first grant access to the memory (200) to a buffer with a smaller size among the stream input buffer (100) and the stream output buffer (300). For example, when the stream input buffer (100) is smaller in size than the stream output buffer (300), the controller (500) may first grant access to the memory (200) to the stream input buffer (100). Thereafter, when the access of the stream input buffer (100) to the memory (200) is terminated, the controller (500) may grant access to the memory (200) to the stream output buffer (300).

[0164] In case 12, the controller (500) may receive an access request signal for the memory (200) from the processor (400) and the stream input buffer (100) which is in a critical state. In addition, the controller (500) may receive an access request signal for the memory (200) from the stream output buffer (300) which is in a normal state, or may not receive an access request signal for the memory (200) from the stream output buffer (300). Since the stream input buffer (100) is in a critical state (e.g., a state in which the minimum unit buffer area (401) and the additional buffer area (402) are about to be full), the controller (500) may determine the highest access priority for the memory (200) of the stream input buffer (100) and grant the stream input buffer (100) access rights to the memory (200).

[0165] In case 13, the controller (500) may receive an access request signal for the memory (200) from the processor (400) and the stream output buffer (300) which is in a critical state. In addition, the controller (500) may receive an access request signal for the memory (200) from the stream input buffer (100) which is in a normal state, or may not receive an access request signal for the memory (200) from the stream input buffer (100). Since the stream output buffer (300) is in a critical state (e.g., a state in which data to be exhausted as output stream data is about to run out), the controller (500) may determine the highest access priority for the memory (200) of the stream output buffer (300) and grant the stream output buffer (300) access rights to the memory (200).

[0166] In case 14, the controller (500) may receive an access request signal for the memory (200) from all of the processor (400), the stream input buffer (100) in a critical state, and the stream output buffer (300) in a critical state. At this time, since the stream input buffer (100) and the stream output buffer (300) are in critical states, the controller (500) may first grant access to the memory (200) to a buffer with a smaller size among the stream input buffer (100) and the stream output buffer (300). For example, if the stream input buffer (100) is smaller in size than the stream output buffer (300), the controller (500) may first grant access to the memory (200) to the stream input buffer (100). Thereafter, the controller (500) can grant access to the memory (200) to the stream output buffer (300) when access to the memory (200) of the stream input buffer (100) is terminated.

[0167] According to one embodiment of the present disclosure, the controller (500) can minimize performance degradation of the processor (400) by managing access to the memory (200) of the processor (400), the stream input buffer (100), and the stream output buffer (300) based on access priorities to the memory (200).

[0168] FIG. 13 is a diagram illustrating the processing time of a processor (400) according to one embodiment of the present disclosure. In FIG. 13, it is assumed that the memory (200) access time of an operation performed by the processor (400) changes at every moment.

[0169] According to the line buffer method (a), after the processor (400) completes all operations, an interface (e.g., a buffer) accesses the memory (200) to transmit data. Therefore, according to the line buffer method (a), the operation processing time for the input stream data of the processor (400) may be prolonged, and the chip size may increase.

[0170] According to the method (b) of giving priority to the processor (400), the interface (e.g., buffer) can access the memory (200) and transfer data at the moment when the processor (400) does not access the memory (200). Therefore, in the case of the method (b) of giving priority to the processor (400), the processing time of the processor (400) may be the shortest, but since it is possible to predict when the processor (400) will not access the memory (200), the buffer size must be designed to be large in preparation for the worst case.

[0171] In the Priority to Interface Method (d), the buffer can access the memory (200) whenever it is full. That is, if the processor (400) was accessing the memory (200) at that moment, the buffer can block the processor's (400) access and transfer the data to the memory (200). Therefore, according to the Priority to Interface Method (d), the processor (400) may be frequently interrupted by the buffer during operation processing, which may increase the processing time of the processor (400).

[0172] The dynamic priority method (c) according to one embodiment of the present disclosure interrupts the processor's (400) access to the memory (200) at a moment when the processor (400) does not access the memory (200) and at a moment when the interface (e.g., the stream input buffer (100)) can no longer maintain data (e.g., when the additional buffer area (402) is in a critical state), and allows the interface (e.g., the stream input buffer (100)) to access the memory (200) and transmit data. Therefore, the dynamic priority method (c) has the advantage of shortening the processing time of the processor (400) compared to the method (d) of giving priority to the interface, and minimizing the size of the buffer compared to the method (b) of giving priority to the processor (400). In addition, the dynamic priority method (c) can appropriately manage the time of stream data transmission by combining the additional buffer area (402) and the minimum unit buffer area (401).

[0173] According to one embodiment of the present disclosure, an electronic device (1000) can be provided that minimizes performance degradation of a processor (400) due to access to the memory (200) of an interface by separately configuring a controller (500) that manages access to the memory (200) of a processor (400) and an interface (buffer) while optimizing the chip area by combining an additional buffer area and a minimum unit buffer area.

[0174] An electronic device (1000) according to one embodiment of the present disclosure comprises: a memory (200) for storing data for operation processing of a processor (400); a stream input buffer (100) for sequentially storing input stream data in a stream data format input from the outside, converting the input stream data into a processor data format that can be processed by the processor (400), and transmitting the input stream data converted into the processor data format to the memory (200); a processor (400) for obtaining input stream data converted into the processor data format from the memory (200), and storing operation processing result data for the input stream data converted into the processor data format in the memory (200) in the processor data format; a stream output buffer (300) for storing operation processing result data in the processor data format received from the memory (200), converting the operation processing result data into the stream data format, and sequentially outputting the operation processing result data to the outside; And it may include a controller (500) that determines access rights to the memory (200) of the processor (400), the stream input buffer (100), and the stream output buffer (300) based on the state of the processor (400), the state of the stream input buffer (100), and the state of the stream output buffer (300).

[0175] A processor data format according to one embodiment of the present disclosure may include a word format, which is the smallest unit that the processor (400) can process. A word may be created by stacking unit data (e.g., pixels) of stream data into a single block.

[0176] Each of the stream input buffer (100) and the stream output buffer (300) according to one embodiment of the present disclosure may include a minimum unit buffer area for conversion between a stream data format and a processor data format, and an extra buffer area.

[0177] The size of the minimum unit buffer area according to one embodiment of the present disclosure may be an integer multiple of the size of a word, which is the smallest unit that the processor (400) can process.

[0178] The stream input buffer (100) according to one embodiment of the present disclosure can transmit information that the minimum unit buffer area of ​​the stream input buffer (100) is full to the controller (500). If the stream input buffer (100) is granted access to the memory (200) from the controller (500), the stream input buffer (100) can convert input stream data stored in the minimum unit buffer area of ​​the stream input buffer (100) into a processor data format and transmit the converted input stream data to the memory (200). If the stream input buffer (100) is not granted access to the memory (200) from the controller (500), the stream input buffer (100) can sequentially store input stream data in an additional buffer area of ​​the stream input buffer (100).

[0179] According to one embodiment of the present disclosure, the stream input buffer (100) can transmit a signal to the controller (500) requesting to increase the access priority to the memory (200) when the minimum unit buffer area of ​​the stream input buffer (100) is full and the additional buffer area of ​​the stream input buffer (100) reaches a critical state. Based on being granted access rights to the memory (200) from the controller (500), the stream input buffer (100) can convert input stream data stored in the minimum unit buffer area to the memory (200) and transmit the converted input stream data to the processor data format.

[0180] According to one embodiment of the present disclosure, when the stream input buffer (100) converts input stream data stored in the minimum unit buffer area of ​​the stream input buffer (100) into a processor data format and transmits it to the memory (200), the input stream data stored in the additional buffer area of ​​the stream input buffer (100) can be moved to the minimum unit buffer area of ​​the stream input buffer (100).

[0181] The stream output buffer (300) according to one embodiment of the present disclosure can transmit information that the minimum unit buffer area and the additional buffer area of ​​the stream output buffer (300) are empty to the controller (500). The stream output buffer (300) can receive the operation processing result data in the processor data format from the memory (200) by being granted access to the memory (200) by the controller (500). The stream output buffer (300) can store the operation processing result data in the minimum unit buffer area of ​​the stream output buffer (300).

[0182] The stream output buffer (300) according to one embodiment of the present disclosure can store the operation processing result data stored in the minimum unit buffer area of ​​the stream output buffer (300) in the additional buffer area of ​​the stream output buffer (300) by pushing the operation processing result data stored in the minimum unit buffer area of ​​the stream output buffer (300) as much as possible into the additional buffer area of ​​the stream output buffer (300) and store it in the minimum unit buffer area and the additional buffer area of ​​the stream output buffer (300). The stream output buffer (300) can convert the operation processing result data stored in the minimum unit buffer area of ​​the stream output buffer (300) into a stream data format and sequentially output it to the outside.

[0183] The stream output buffer (300) according to one embodiment of the present disclosure can transmit information that the minimum unit buffer area is empty to the controller (500). If the stream output buffer (300) is granted access to the memory (200) from the controller (500), the stream output buffer (300) can receive the computational processing result data in a processor data format from the memory (200) and store it in the minimum unit buffer area of ​​the stream output buffer (300). If the stream output buffer (300) is not granted access to the memory (200) from the controller (500), the stream output buffer (300) can convert the computational processing result data stored in the additional buffer area of ​​the stream output buffer (300) into a stream data format and sequentially output it to the outside.

[0184] According to one embodiment of the present disclosure, the stream output buffer (300) may transmit a signal to the controller (500) requesting to increase the access priority to the memory (200) when the minimum unit buffer area of ​​the stream output buffer (300) is empty and the additional buffer area of ​​the stream output buffer (300) reaches a critical state. The stream output buffer (300) may receive the computational processing result data in the processor data format from the memory (200) based on being granted access rights to the memory (200) from the controller (500). The stream output buffer (300) may store the computational processing result data in the minimum unit buffer area of ​​the stream output buffer (300).

[0185] When a controller (500) according to one embodiment of the present disclosure receives a request signal requesting access to memory (200) from one of the modules of the processor (400), the stream input buffer (100), and the stream output buffer (300), the controller may grant access to memory (200) to the module that transmitted the request signal.

[0186] When a controller (500) according to one embodiment of the present disclosure receives a request signal requesting access to memory (200) from two or more modules among a processor (400), a stream input buffer (100), and a stream output buffer (300), the controller may grant access to memory (200) to a module having a higher access priority to memory (200) among the modules that transmitted the request signal.

[0187] According to one embodiment of the present disclosure, the controller (500) may determine a higher access priority for the memory (200) of the processor (400) than the stream input buffer (100) and the stream output buffer (300) when the stream input buffer (100) and the stream output buffer (300) are in a normal state.

[0188] According to one embodiment of the present disclosure, the controller (500) may determine a higher access priority to the memory (200) of a buffer with a smaller size among the stream input buffer (100) and the stream output buffer (300) when both the stream input buffer (100) and the stream output buffer (300) are in a threshold state.

[0189] According to one embodiment of the present disclosure, the controller (500) can determine the access priority for the memory (200) in the order of the stream input buffer (100), the processor (400), and the stream output buffer (300) when the stream input buffer (100) is in a critical state.

[0190] A controller (500) according to one embodiment of the present disclosure can receive status information from each of a processor (400), a stream input buffer (100), and a stream output buffer (300).

[0191] According to one embodiment of the present disclosure, when the controller (500) receives information from the stream input buffer (100) that the minimum unit buffer area of ​​the stream input buffer (100) is full, the controller (500) can determine access rights to the memory (200) of the stream input buffer (100) based on the state of the processor (400).

[0192] According to one embodiment of the present disclosure, the controller (500) may not grant access to the memory (200) of the stream input buffer (100) when the processor (400) occupies the memory (200), and may grant access to the memory (200) of the stream input buffer (100) when the processor (400) does not occupy the memory (200).

[0193] According to one embodiment of the present disclosure, when the controller (500) receives information from the stream output buffer (300) that the additional buffer area of ​​the stream output buffer (300) contains operation processing result data but the minimum unit buffer area of ​​the stream output buffer (300) is empty, the controller (500) may determine access rights to the memory (200) of the stream output buffer (300) based on the state of the processor (400).

[0194] A processor data format according to one embodiment of the present disclosure may include a data word corresponding to the smallest unit that the processor can process.

[0195] Each of the stream input buffer and the stream output buffer according to one embodiment of the present disclosure may include a minimum unit buffer area for conversion between a stream data format and the processor data format, and an extra buffer area.

[0196] The size of the minimum unit buffer area according to one embodiment of the present disclosure may be an integer multiple of the size of a data word corresponding to the smallest unit that the processor can process.

[0197] A stream input buffer according to one embodiment of the present disclosure may transmit information indicating that input stream data stored in a minimum unit buffer area of ​​the stream input buffer has reached an upper limit to a controller. The stream input buffer may convert the input stream data stored in the minimum unit buffer area of ​​the stream input buffer into a processor data format and transmit the converted data to the memory based on whether access to the memory is granted. The stream input buffer may sequentially store the input stream data in an additional buffer area of ​​the stream input buffer based on whether access to the memory is not granted.

[0198] According to one embodiment of the present disclosure, a stream input buffer may transmit a signal to a controller requesting an increase in access priority to a memory (200) when input stream data stored in a minimum unit buffer area of ​​the stream input buffer reaches an upper limit and an additional buffer area of ​​the stream input buffer reaches a threshold state. Based on being granted access rights to the memory, the stream input buffer may convert the input stream data stored in the minimum unit buffer area of ​​the stream input buffer into a processor data format and transmit the converted data to the memory.

[0199] A stream input buffer according to one embodiment of the present disclosure can move input stream data stored in an additional buffer area of ​​the stream input buffer to the minimum unit buffer area of ​​the stream input buffer based on input stream data stored in the minimum unit buffer area of ​​the stream input buffer being converted into a processor data format and transmitted to memory.

[0200] A stream output buffer according to one embodiment of the present disclosure can transmit information to a controller that the minimum unit buffer area and the additional buffer area of ​​the stream output buffer are empty. The stream output buffer can receive computational processing result data in a processor data format from memory based on being granted access to the memory. The stream output buffer can store the computational processing result data in the minimum unit buffer area of ​​the stream output buffer.

[0201] A stream output buffer according to one embodiment of the present disclosure can push the operation processing result data stored in the minimum unit buffer area of ​​the stream output buffer to an additional buffer area of ​​the stream output buffer and store the data in the minimum unit buffer area of ​​the stream output buffer and the additional buffer area. The stream output buffer can convert the operation processing result data stored in the minimum unit buffer area of ​​the stream output buffer into a stream data format and sequentially output it.

[0202] A stream output buffer according to one embodiment of the present disclosure can transmit information to a controller that a minimum unit buffer area of ​​the stream output buffer is empty. The stream output buffer can receive computational processing result data in a processor data format from memory based on being granted access to memory and store the data in the minimum unit buffer area of ​​the stream output buffer. The stream output buffer can convert computational processing result data stored in an additional buffer area of ​​the stream output buffer into a stream data format and sequentially output the converted data, based on not being granted access to memory.

[0203] According to one embodiment of the present disclosure, a stream output buffer may transmit a signal to a controller to increase the priority for accessing memory based on the fact that a minimum unit buffer area of ​​the stream output buffer is empty and an additional buffer area of ​​the stream output buffer reaches a critical state. The stream output buffer may receive the computational processing result data in a processor data format from the memory based on the access right to the memory being granted by the controller. The stream output buffer may store the computational processing result data in the minimum unit buffer area of ​​the stream output buffer.

[0204] A controller according to one embodiment of the present disclosure may grant access to memory to one of the processor, the stream input buffer, or the stream output buffer that transmitted the request signal based on receiving a request signal requesting access to memory from one of the processor, the stream input buffer, or the stream output buffer.

[0205] According to one embodiment of the present disclosure, the controller may, based on receiving a request signal requesting access to memory from two or more of a processor, a stream input buffer, and a stream output buffer, grant access to memory to a processor, a stream input buffer, or a stream output buffer having a higher access priority to memory among two or more of the processor, the stream input buffer, or the stream output buffer that transmitted the request signal.

[0206] A controller according to one embodiment of the present disclosure may determine a higher access priority for a memory of a processor than a stream input buffer and a stream output buffer based on the stream input buffer and the stream output buffer being in a normal state.

[0207] A controller according to one embodiment of the present disclosure can determine a higher access priority to memory of a buffer with a smaller size among the stream input buffer and the stream output buffer, based on the fact that both the stream input buffer and the stream output buffer are in a critical state.

[0208] A controller according to one embodiment of the present disclosure can determine an access priority for memory in the order of stream input buffer, processor, and stream output buffer based on whether the stream input buffer is in a critical state.

[0209] A controller according to one embodiment of the present disclosure may receive status information from each of a processor, a stream input buffer, and a stream output buffer.

[0210] According to one embodiment of the present disclosure, a controller can determine access rights to the memory of a stream input buffer based on a state of a processor when input stream data stored in a minimum unit buffer area of ​​a stream input buffer reaches an upper limit.

[0211] According to one embodiment of the present disclosure, a controller may not grant access to the memory of a stream input buffer when the processor is accessing the memory. The controller may grant access to the memory of a stream input buffer when the processor is not accessing the memory.

[0212] According to one embodiment of the present disclosure, a controller can determine access rights to the memory of a stream output buffer based on a state of a processor when the additional buffer area of ​​the stream output buffer contains data resulting from computational processing but the minimum unit buffer area of ​​the stream output buffer is empty.

[0213] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0214] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0215] The embodiments described above are merely specific examples intended to illustrate the technical content of embodiments of the present disclosure and aid in understanding of the embodiments of the present disclosure, and are not intended to limit the scope of the embodiments of the present disclosure. Therefore, the scope of the various embodiments of the present disclosure should be interpreted to include all modifications or variations derived based on the technical concepts of the various embodiments of the present disclosure, in addition to the embodiments disclosed herein.

Claims

1. Memory (200) that stores data for the processor's computational processing; A stream input buffer (100) that sequentially stores input stream data input in a stream data format, converts the input stream data into a processor data format that can be processed by the processor, and transmits the input stream data in the processor data format to the memory; A processor (400) that obtains input stream data in the processor data format from the memory and stores data resulting from the operation processing for the input stream data in the processor data format in the memory; A stream output buffer (300) that stores the operation processing result data received from the memory in the processor data format and sequentially outputs the operation processing result data by converting it into the stream data format; and An electronic device comprising a controller (500) that determines access rights to the memory of the processor, the stream input buffer, and the stream output buffer based on the state of the processor, the state of the stream input buffer, and the state of the stream output buffer.

2. In the first paragraph, the processor data format is: An electronic device including a data word corresponding to the smallest unit that the processor (400) can process.

3. In the first or second paragraph, each of the stream input buffer (100) and the stream output buffer (300) An electronic device comprising a minimum unit buffer area for conversion between the stream data format and the processor data format, and an extra buffer area.

4. In the third paragraph, the size of the minimum unit buffer area is: An electronic device whose size is an integer multiple of the size of a data word corresponding to the smallest unit that the processor (400) can process.

5. In any one of the first to fourth clauses, the stream input buffer (100) Information indicating that the input stream data stored in the minimum unit buffer area of ​​the above stream input buffer (100) has reached the upper limit is transmitted to the controller (500), Based on the access right granted to the above memory (200), the input stream data stored in the minimum unit buffer area of ​​the stream input buffer (100) is converted into the processor data format and transmitted to the memory (200). An electronic device that sequentially stores the input stream data in an additional buffer area of ​​the stream input buffer (100) based on the fact that access to the memory (200) is not granted.

6. In the fifth paragraph, the stream input buffer (100) When the input stream data stored in the minimum unit buffer area of ​​the stream input buffer (100) reaches the upper limit and the additional buffer area of ​​the stream input buffer (100) reaches a threshold state, a signal requesting an increase in access priority to the memory (200) is transmitted to the controller (500). An electronic device that converts input stream data stored in the minimum unit buffer area of ​​the stream input buffer (100) into the processor data format and transmits the converted data to the memory (200) based on being granted access to the memory (200).

7. In the 6th paragraph, the stream input buffer (100) An electronic device that moves input stream data stored in an additional buffer area of ​​the stream input buffer (100) to the minimum unit buffer area of ​​the stream input buffer based on the input stream data stored in the minimum unit buffer area of ​​the stream input buffer (100) being converted into the processor data format and transmitted to the memory (200).

8. In any one of the first to seventh paragraphs, the stream output buffer (300) Information that the minimum unit buffer area and additional buffer area of ​​the stream output buffer (300) are empty is transmitted to the controller (500), Based on being granted access to the memory (200), receiving the operation processing result data in the processor data format from the memory (200), An electronic device that stores the above operation processing result data in the minimum unit buffer area of ​​the stream output buffer (300).

9. In the 8th paragraph, the stream output buffer (300) The operation processing result data stored in the minimum unit buffer area of ​​the above stream output buffer (300) is pushed to the additional buffer area of ​​the above stream output buffer (300) and stored in the minimum unit buffer area of ​​the above stream output buffer (300) and the additional buffer area, An electronic device that sequentially outputs the operation processing result data stored in the minimum unit buffer area of ​​the above stream output buffer (300) by converting it into the above stream data format.

10. In the 9th paragraph, the stream output buffer (300) Information that the minimum unit buffer area of ​​the above stream output buffer (300) is empty is transmitted to the controller (500), Based on being granted access to the memory (200), the operation processing result data in the processor data format is received from the memory (200) and stored in the minimum unit buffer area of ​​the stream output buffer (300). An electronic device that sequentially outputs the operation processing result data stored in the additional buffer area of ​​the stream output buffer (300) by converting it into the stream data format based on the failure to be granted access to the memory (200).

11. In the 10th paragraph, the stream output buffer (300) Based on the fact that the minimum unit buffer area of ​​the stream output buffer (300) is empty and the additional buffer area of ​​the stream output buffer (300) reaches a critical state, a signal is transmitted to the controller (500) to increase the priority for accessing the memory (200). Based on being granted access to the memory (200) from the controller (500), receiving the operation processing result data in the processor data format from the memory (200), An electronic device that stores the above operation processing result data in the minimum unit buffer area of ​​the stream output buffer (300).

12. In any one of the first to eleventh paragraphs, the controller (500) An electronic device that grants access to the memory (200) to one of the processor (400), the stream input buffer (100), or the stream output buffer (300) based on receiving a request signal requesting access to the memory (200) from the processor that transmitted the request signal, the stream input buffer, or the stream output buffer.

13. In any one of the first to 12th clauses, the controller (500) An electronic device, wherein, based on receiving a request signal requesting access to the memory (200) from two or more of the processor (400), the stream input buffer (100), and the stream output buffer (300), the electronic device grants access to the memory (200) to the processor, the stream input buffer, or the stream output buffer having a higher access priority to the memory (200) among two or more of the processor, the stream input buffer, or the stream output buffer that transmitted the request signal.

14. In any one of the first to 13th clauses, the controller (500) An electronic device that determines that the processor (400) has a higher access priority to the memory (200) than the stream input buffer (100) and the stream output buffer (300), based on the stream input buffer (100) and the stream output buffer (300) being in a normal state.

15. In any one of the first to 14th clauses, the controller (500) Based on the fact that both the stream input buffer (100) and the stream output buffer (300) are in a critical state, the access priority of the smaller buffer among the stream input buffer (100) and the stream output buffer (300) to the memory (200) is determined to be higher, An electronic device that determines the access priority for the memory (200) in the order of the stream input buffer (100), the processor (400), and the stream output buffer (300) based on the stream input buffer (100) being in a critical state.

Images