Method and Apparatus for Dataflow-Controlled Matrix Operation for Block-Wise Scale Application

Abstract

A data flow-controlled matrix operation method for applying block-unit scale according to one technical aspect of the present invention is performed in a matrix operation device in which a plurality of operation units are connected along a data transmission path, and comprises the steps of: transmitting a scale vector through the data transmission path and providing identification information indicating that the scale vector is being transmitted; each of the plurality of operation units storing the scale vector based on the identification information; transmitting operation data through the data transmission path and providing transition information corresponding to the block boundary of the operation data; each of the plurality of operation units updating an internal scale selection state in response to the transition information; and each of the plurality of operation units performing a matrix operation by applying a scale value selected from the stored scale vector to the operation data according to the internal scale selection state.

Description

Technology Field

[0001] The present invention relates to a matrix operation technology in which block-unit scale is applied, and more specifically, to a data flow-controlled matrix operation method and apparatus capable of performing block-unit scale application without a separate complex global control network by transmitting a scale vector and operation data through a data transmission path and having each operation unit update an internal scale selection state according to control information corresponding to a block boundary. Background Technology

[0002] Recently, low-precision data representation and block quantization techniques are widely utilized in matrix operations for artificial intelligence computation, particularly for large-scale neural network inference and training, to increase throughput and power efficiency. In these block quantization methods, multiple data elements are grouped into a single block, and numerical ranges are corrected using a scale value corresponding to each block, thereby reducing memory usage and data transfer while maintaining computational accuracy. Furthermore, matrix operation structures based on orthogonal arrays or systolic arrays are widely used as high-speed matrix operation structures because they can secure high parallelism and reusability through data transfer between adjacent operation units.

[0003] However, in a matrix operation environment where block quantization is applied, it is necessary to apply the scale value corresponding to each block to each operation unit at an appropriate time. To this end, conventional methods may involve establishing a dedicated path for supplying scale values ​​or control signals separate from the transmission path of operation data, or a central control unit managing the operation timing of each operation unit. However, with such structures, as the number of operation units increases, the complexity of control wiring and routing burden may grow, and additional design may be required to ensure timing alignment between data transmission and control transmission. In particular, the flexibility of the control structure may be reduced when the block size or the scale application cycle changes.

[0004] Furthermore, as matrix operation units become more highly integrated and array sizes expand, a structure that distributes individual control information for each operation unit over a wide range can increase implementation costs and design complexity. Moreover, if the flow of operation data and the timing of scale application are misaligned, there is a risk that the intended scale value may not be accurately reflected at block boundaries. Therefore, in matrix operation environments where block-unit scaling is applied, there is a demand for matrix operation technologies that can reduce structural complexity while ensuring consistency between data transmission and scale control. Prior art literature

[0005] Republic of Korea Published Patent Application No. 10-2024-0051715 (Date of publication: April 22, 2024) The problem to be solved

[0006] One objective of the present invention is to solve the aforementioned problems and to provide a matrix operation method and apparatus capable of performing block-unit scale application by utilizing a data transmission structure, thereby mitigating the problem of increased wiring complexity, routing burden, and timing alignment burden as the number of operation units increases when a dedicated control path or global control network is used to supply scale values ​​and control information separately from operation data in a matrix operation environment where block-unit scale is applied.

[0007] In addition, in conventional structures, a method may be considered in which a central control unit directly manages the block boundary time and the scale application time for each operation unit or distributes them through a separate control signal network. Consequently, when expanding the array size, the control structure becomes complex, and problems may arise where it is difficult to maintain alignment between the block boundary and the actual operation time. Accordingly, another objective of the present invention is to enable each operation unit to autonomously update its internal scale selection state based on information traveling along a data transmission path, thereby allowing scale transitions corresponding to block boundaries to be performed without separate, complex global control.

[0008] In addition, block quantization methods may vary in block size or scale application cycle depending on computation conditions; however, in conventional structures, flexibility may be reduced as hardware control structures must be modified or fixed control rules must be relied upon to respond to such changes. Accordingly, another objective of the present invention is to provide a data flow control structure capable of flexibly responding to various block-unit scale application conditions by transmitting scale vectors and computation data through a data transmission path and adjusting the timing of scale application according to control information corresponding to block boundaries.

[0009] In addition, if the flow of computational data and the timing of scale application are misaligned, an unintended scale value may be applied, which may lead to a decrease in computational accuracy and processing stability. Accordingly, another objective of the present invention is to improve the accuracy and reliability of block-unit scale application by ensuring that control information corresponding to computational data and block boundaries is transmitted in alignment between computational units, thereby enabling each computational unit to select and apply an appropriate scale value in response to the arrival time of the computational data.

[0010] In addition, another objective of the present invention is to provide a matrix operation technology that can improve the structural simplicity, scalability, and implementation efficiency of a matrix operation device while efficiently utilizing the same data transfer resources through a structure that stores a scale vector in advance and applies a scale value selected from the stored scale vector in block units.

[0011] However, the problems to be solved in this disclosure are not limited to those mentioned above, and may be expanded in various ways without departing from the spirit and scope of this disclosure. means of solving the problem

[0012] One technical aspect of the present invention provides a data flow-controlled matrix operation method for applying block-unit scales. The method is performed in a matrix operation device in which a plurality of operation units are connected along a data transmission path, and comprises the steps of: transmitting a scale vector through the data transmission path and providing identification information indicating that the scale vector is being transmitted; each of the plurality of operation units storing the scale vector based on the identification information; transmitting operation data through the data transmission path and providing transition information corresponding to the block boundary of the operation data; each of the plurality of operation units updating an internal scale selection state in response to the transition information; and each of the plurality of operation units performing a matrix operation by applying a scale value selected from the stored scale vector to the operation data according to the internal scale selection state.

[0013] In one embodiment, the step of transmitting a scale vector through the data transmission path and providing identification information indicating that the scale vector is being transmitted together may include the step of providing the identification information in a first state while transmitting the scale vector through the data transmission path, and the step of transmitting the computation data may include the step of providing the identification information in a second state distinct from the first state while transmitting the computation data through the data transmission path.

[0014] In one embodiment, the step of having each of the plurality of operation units store the scale vector based on the identification information may include the step of storing data input through the data transmission path as the scale vector when the identification information corresponds to the scale vector, and the step of transmitting the same data stored as the scale vector to a subsequent operation unit along the data transmission path.

[0015] In one embodiment, after the step of each of the plurality of operation units storing the scale vector based on the identification information, the method may further include the step of setting the internal scale selection state of each of the plurality of operation units to an initial state corresponding to a first scale value of the stored scale vector, and the step of performing a matrix operation by applying the first scale value corresponding to the initial state to the operation data of the first block.

[0016] In one embodiment, the step of providing transition information corresponding to the block boundary of the operation data together may include the step of providing the transition information by activating it when the operation data corresponds to the block boundary, and the step of providing the transition information by deactivating it when the operation data does not correspond to the block boundary, and the step of updating the internal scale selection state may include the step of updating the internal scale selection state to a state corresponding to the next scale value when the transition information is activated, and the step of maintaining the internal scale selection state when the transition information is deactivated.

[0017] In one embodiment, the step of performing the matrix operation may include selecting a scale value corresponding to the internal scale selection state from the stored scale vector, and applying the selected scale value to the operation data of the current block to perform the matrix operation.

[0018] In one embodiment, the step of transmitting a scale vector through the data transmission path and providing identification information indicating that the scale vector is being transmitted together includes the step of configuring a plurality of scale values ​​corresponding to each of a plurality of blocks into a single scale vector packet and transmitting it through the data transmission path, and the step of causing each of the plurality of computational units to store the scale vector may include the step of storing the plurality of scale values ​​included in the scale vector packet in parallel.

[0019] In one embodiment, the step of updating the internal scale selection state may include a step of performing an update based on the transition information in response to the time when the operation data reaches each operation unit.

[0020] Another technical aspect of the present invention provides a data flow-controlled matrix operation device for applying block-unit scales. The device comprises: an input control unit that receives matrix data and a scale vector and transmits identification information indicating that the scale vector is being transmitted and transition information corresponding to the block boundary of the matrix data through a data transmission path; a scale storage unit that stores the scale vector transmitted through the data transmission path based on the identification information; a state control unit that updates an internal scale selection state in response to the transition information; a scale selection unit that selects a scale value from the scale vector stored in the scale storage unit according to the internal scale selection state; and a matrix operation unit that performs matrix operations by applying the scale value selected by the scale selection unit to the matrix data. Effects of the invention

[0021] According to various embodiments of the present invention, scale vectors and computation data are transmitted through the same data transmission path, and each computation unit is configured to perform storage of the scale vector and update of the internal scale selection state based on identification information and transition information transmitted along the data transmission path. This provides the effect of alleviating the burden of having to configure a separate dedicated control path or a complex global control network for scale application. Accordingly, the wiring structure of the matrix computation device can be simplified, and routing complexity and implementation burden associated with an increase in the number of computation units can be reduced.

[0022] In addition, according to various embodiments of the present invention, since each operation unit is configured to locally update its internal scale selection state in response to transition information corresponding to a block boundary, it provides the effect of enabling block-unit scale transitions without the central control unit directly distributing or managing the timing of scale application for each operation unit. Accordingly, the scalability of the control structure is improved even when the array size is expanded, and control operations for multiple operation units can be performed more efficiently.

[0023] In addition, according to various embodiments of the present invention, since the scale vector is stored in advance and a scale value corresponding to an internal scale selection state is selected and applied to the computation data, it provides the effect of reducing separate repetitive scale loading or external control intervention during computation while reflecting block-unit scale values ​​at an appropriate time. Accordingly, data movement overhead can be reduced and the efficiency of matrix operation processing can be improved.

[0024] In addition, according to various embodiments of the present invention, transition information corresponding to computation data and block boundaries is configured to be transmitted in alignment along a data transmission path, thereby providing the effect of enabling each computation unit to select and apply an appropriate scale value in response to the arrival time of the computation data. Accordingly, the problem of unintended scale values ​​being applied at block boundaries can be mitigated, and the accuracy and reliability of block-unit scale application can be improved.

[0025] In addition, according to various embodiments of the present invention, by configuring and storing a plurality of scale values ​​corresponding to each of a plurality of blocks into a single scale vector packet, the effect of efficiently managing scale information for a plurality of blocks while utilizing the same data transmission resources is provided. Accordingly, the efficiency of scale information transmission can be increased, and resource utilization in a data flow-based matrix operation structure can be improved.

[0026] Furthermore, according to various embodiments of the present invention, since the block-unit scale application structure is configured to operate based on the provision of control information through a data transmission path, it provides the effect of being able to flexibly respond to various operation conditions in which the block size or scale application cycle varies. Accordingly, the application range of the matrix operation device can be expanded, and usability in various quantization environments can be improved.

[0027] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be obvious to those skilled in the art to which the present invention belongs from the description below. Brief explanation of the drawing

[0028] FIG. 1 is an overall block diagram of a data flow-controlled matrix operation system for applying block unit scale according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a data flow-controlled matrix operation method for applying block unit scale according to an embodiment of the present invention. FIG. 3 is a block diagram illustrating the detailed configuration of a matrix operation device according to one embodiment of the present invention. FIG. 4 is a diagram illustrating the hardware block configuration of a data flow-controlled matrix operation device for applying block-unit scale according to an embodiment of the present invention. FIG. 5 is a timing diagram illustrating how a scale transition operation corresponding to the preloading of a scale vector and a block boundary is performed on the time axis in a data flow-controlled matrix operation device for applying a block unit scale according to an embodiment of the present invention. FIG. 6 is a diagram illustrating the concepts of scale vector packing and data block mapping according to one embodiment of the present invention. FIG. 7 is a drawing illustrating an example of scale application in a first block size according to an embodiment of the present invention. FIG. 8 is a drawing illustrating an example of scale application in a second block size according to an embodiment of the present invention. FIG. 9 is a diagram illustrating an internal processing structure of an operation unit and a structure in which a plurality of operation units are connected at an array level according to an embodiment of the present invention. FIG. 10 is a flowchart illustrating a selective scale maintenance and conditional transition operation procedure according to one embodiment of the present invention. FIG. 11 is a flowchart illustrating an axis-by-axis asynchronous scale synthesis-based matrix operation procedure according to an embodiment of the present invention. FIG. 12 is a flowchart illustrating a block unit matrix operation control procedure according to the application of asynchronous scale by axis according to an embodiment of the present invention. Specific details for implementing the invention

[0029] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0030] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, each of phrases such as A or B, at least one of A and B, at least one of A or B, A, B or C, at least one of A, B and C, and at least one of A, B, or C may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as first, second, or first or second may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as coupled or connected to another (e.g., second) component, with or without the terms functionally or communicationally, it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0031] As used in the various embodiments of this document, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0032] Various embodiments of this document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., memory) readable by a machine or device. For example, the processor of the machine or device may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by a machine may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0033] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0034] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0036] The meanings of the key terms used in this specification are as follows, and each term is defined from a functional perspective to aid in understanding the invention. The following definitions of terms are intended to describe embodiments of the invention and should not be interpreted as unduly limiting the scope of the invention.

[0037] "Dataflow-Controlled Matrix Operation Apparatus" refers to a matrix operation apparatus configured such that operation data used for matrix operations and scale-related control information are continuously moved along the same data transmission path, and each operation unit updates its internal state and performs operations in response to identification information and transition information included in the transmitted information without an external controller. The dataflow-controlled matrix operation apparatus in the present invention can be implemented in various forms, such as a systolic array structure, a pipelined operation array structure, or a matrix operation structure in which a plurality of processing elements are arranged in the row and column directions.

[0038] A "scale vector" refers to a set of scale values ​​to be applied to each of a plurality of operation data blocks. In the present invention, the scale vector may be composed of a row-direction scale vector, a column-direction scale vector, or a combination thereof, and a plurality of scale values ​​may be transmitted through a data transmission path in a packed state. Additionally, the scale vector may be expressed in the form of a fixed-point value, a floating-point value, or a quantization factor.

[0039] "Identifier information" refers to control information provided to enable multiple computation units to determine that information transmitted along a data transmission path is not general computation data but a scale vector. The identifier information in the present invention may be implemented in the form of a separate tag bit, header field, slot type code, or reserved bit on a protocol, and each computation unit may recognize input data as a scale vector storage target based on the identifier information.

[0040] "Transition information" refers to control information provided to indicate the block boundaries of operation data, the point of block transition, or the point of change of the scale application interval. In the present invention, the transition information may be implemented as bits, flags, or codes indicating the end point of a specific data block, the start point of the next block, or the point of transition for scale application by axis, and each operation unit may update the internal scale selection state in response to the transition information.

[0041] "Internal scale selection state" refers to internal state information indicating which position among the stored scale vectors to apply to the matrix operation block currently being performed. In the present invention, the internal scale selection state can be managed as a row-direction state, a column-direction state, or an axis-independent state, and can be maintained or updated depending on whether block boundaries or transition information occur.

[0042] A "scale selector" refers to a functional block that selects a scale value to be applied to the current operation data from a stored scale vector according to an internal scale selection state. The scale selector in the present invention may be implemented in the form of a multiplexer, an index selection circuit, a selection register, or an address-based reference circuit, and may be configured independently for the row direction and the column direction, respectively.

[0043] "Axis-specific state" refers to an internal scale selection state maintained independently for the row direction and the column direction, respectively. In the present invention, since row-direction data block transitions and column-direction data block transitions may occur at different times, each axis-specific state can be updated or maintained independently of the others. Accordingly, asynchronous scale application is possible, in which the scale selection state is updated on one axis while the existing state is maintained on the other axis.

[0044] "Composite scale value" refers to a final scale value applied to actual computation data by combining a scale value selected in the row direction and a scale value selected in the column direction. The composite scale value in the present invention can be generated according to multiplication, addition, exponential synthesis, or predefined combination rules, and can be directly used for quantization restoration or precision correction operations.

[0045] "Block boundary" refers to the boundary point where matrix operation data is separated by scale application units. In the present invention, the same scale selection state may be maintained within the same block, and the scale selection state may be updated only when transition information is provided at the block boundary. Block boundaries may be defined based on data length, tile units, or operation schedule units.

[0046] "In-band control" refers to a method of transmitting control information together within the same data transmission path as computation data without using a separate control channel. In the present invention, since identification information and transition information are provided on the same path as the computation data, each computation unit can autonomously control its state without separate external synchronization control or a control bus.

[0048] FIG. 1 is an overall block diagram of a data flow-controlled matrix operation system for applying block unit scale according to an embodiment of the present invention.

[0049] The system according to FIG. 1 may be structured such that a host device (10) provides input data and scale-related information required for matrix operations to a matrix operation device (100), and the matrix operation device (100) performs matrix operations reflecting block unit scale application on the input data, and then records the operation result in a result storage device (20) or transmits it back to the host device (10).

[0050] Referring to FIG. 1, the host device (10) can generate or prepare input matrix data to be subject to matrix operation, operand matrix data, and scale vector required for block unit scale application.

[0051] In one embodiment, the host device (10) may pack matrix data and scale vectors in a pre-sorted order and transmit them to the matrix operation device (100), and may also provide identification information or block boundary information corresponding to each data block. In this case, the host device (10) may be implemented as a central processing unit, a graphics processing unit, an artificial intelligence operation control processor, or other electronic device capable of operation control.

[0052] The matrix operation device (100) can receive matrix data and scale vectors transmitted from the host device (10) and process them along an internal data transmission path.

[0053] In one embodiment, the matrix operation device (100) loads a scale vector into an internal storage structure in advance before performing the operation, and then, when operation data is input in block units, selects a scale value corresponding to the current block and applies it to the operation data. Additionally, the matrix operation device (100) can interpret transition information corresponding to block boundaries to update or maintain the internal scale selection state, and accordingly, the selected scale value can be maintained between consecutive data blocks without separate scale retransmission.

[0054] In one embodiment, the matrix operation device (100) may include a plurality of operation units, and each operation unit may share the same set of scale vectors or store different scale vectors independently. Additionally, the matrix operation device (100) may select an appropriate scale vector based on the location or identification information of the data block currently being processed, and perform multiplication, accumulation, or other matrix operations based on the selected scale value. Due to this structure, continuous matrix operations can be performed without interrupting the flow of operation data, even in a quantized data environment where block-unit scale application is required.

[0055] The result storage device (20) can store the operation results generated by the matrix operation device (100). The result storage device (20) can be implemented as an internal memory, buffer memory, external storage device, or shared memory area, and can be used to temporarily maintain the operation results and then for subsequent operations or transfer to an external system. Additionally, the result storage device (20) can store multiple operation result blocks sequentially or record them corresponding to addresses per block.

[0056] In the structure according to FIG. 1, by performing the loading of scale vectors and block-unit state control within the matrix operation device (100), the operation data and scale application flow can be separated and controlled without repeatedly supplying scale information from the outside during each operation. Accordingly, the efficiency of data transmission bandwidth usage can be improved, and operation delay can be mitigated in a matrix operation environment where block-unit quantization is applied.

[0058] FIG. 2 is a flowchart illustrating a data flow-controlled matrix operation method for applying block unit scale according to an embodiment of the present invention.

[0059] The procedure according to FIG. 2 may be a procedure in which, after preparing matrix data and a scale vector, the scale vector is transmitted and stored, operation data and identification information are transmitted while the internal scale selection state is initialized, and the internal scale selection state is updated or maintained according to transition information corresponding to the block boundary, and a matrix operation is performed by applying the selected scale value to the operation data. Through such a procedure, even if the scale vector and the operation data are transmitted through the same data transmission path, the scale value corresponding to each block can be selected and applied at an appropriate time.

[0060] Referring to FIG. 2, in step S210, the matrix operation device (100) may prepare matrix data and a scale vector. In one embodiment, the matrix operation device (100) may receive input matrix data, operand matrix data, and a scale vector including a plurality of scale values ​​corresponding to a plurality of blocks from a host device (10). At this time, the matrix data may be arranged so as to be processed by dividing it into blocks, and the scale vector may be arranged in an order corresponding to each block. For example, if 512-dimensional input data is divided into blocks of 16 dimensions, the matrix operation device (100) may prepare a scale vector including a scale value corresponding to each 16-dimensional block.

[0061] In one embodiment, the matrix operation device (100) may configure a plurality of scale values ​​corresponding to a plurality of blocks into a single scale vector packet. In this case, the scale vector packet may be provided in a packed form to fit the bandwidth of the data transmission path. For example, a plurality of 8-bit scale values ​​may be arranged to be included in a single transmission unit.

[0062] In step S220, the matrix operation device (100) can transmit a scale vector and store it in a plurality of operation units. In one embodiment, the matrix operation device (100) can transmit the scale vector through a data transmission path and can also provide identification information indicating that the scale vector is being transmitted. Each of the plurality of operation units determines that the input data corresponds to the scale vector based on the identification information and can store the scale vector in an internal storage structure.

[0063] In one embodiment, the matrix operation device (100) may provide identification information in a first state while transmitting a scale vector. This first state may be a state indicating that the data currently transmitted through the data transmission path is a scale vector. Each of the plurality of operation units may store the data input through the data transmission path as a scale vector in response to the first state.

[0064] In one embodiment, each operation unit can transmit the same scale vector to a subsequent operation unit along a data transfer path while maintaining the stored scale vector internally. Accordingly, the same scale vector used for storage in the preceding operation unit is sequentially transmitted to the subsequent operation unit, so that multiple operation units can share the same scale vector or each store their corresponding scale vector.

[0065] In one embodiment, the matrix operation device (100) can store a plurality of scale values ​​included in a scale vector packet in parallel. In this case, each operation unit can maintain a plurality of candidate scale values ​​from the scale vector packet in an internal register or buffer.

[0066] In step S230, the matrix operation unit (100) can initialize the internal scale selection state. In one embodiment, each operation unit can set the internal scale selection state to an initial state corresponding to a first scale value of the stored scale vector after the scale vector is stored. This initial state may be, for example, a state in which the scale index points to a zero position.

[0067] In one embodiment, the matrix operation device (100) may be prepared to apply a first scale value corresponding to the initial state to the operation data of the first block. Accordingly, when the operation data of the first block is input, the first scale value may be applied immediately without separate additional scale loading. For example, if the scale vector includes multiple scale values ​​such as S0, S1, and S2, the matrix operation device (100) may initialize the internal scale selection state to a state corresponding to S0.

[0068] In step S240, the matrix operation device (100) can transmit operation data and identification information. In one embodiment, the matrix operation device (100) can transmit operation data to a plurality of operation units through a data transmission path, and at this time, can also provide identification information indicating that the operation data is being transmitted.

[0069] In one embodiment, the matrix operation device (100) may provide identification information in a second state distinct from the first state while transmitting operation data. This second state may be a state indicating that the currently transmitted data is data to be operated on. Each of the plurality of operation units may process the input data as data to be operated on rather than as a scale vector storage target in response to the second state. For example, it may be configured to recognize it as a scale vector when the identification information is 1, and to recognize it as operation data when the identification information is 0. In this case, even if the same data transmission path transmits scale vectors and operation data in a time-sharing manner, each operation unit may distinguish and process the type of the current input.

[0070] In step S250, the matrix operation device (100) may provide transition information corresponding to the block boundary of the operation data. In one embodiment, the matrix operation device (100) may provide transition information by enabling it when the current operation data corresponds to the block boundary, and provide transition information by deactivating it when it does not correspond to the block boundary. This transition information may be control information for determining whether each operation unit changes the internal scale selection state. For example, if a specific chunk among consecutive data chunks corresponds to the beginning of a new block, the matrix operation device (100) may enable transition information associated with said chunk. On the other hand, if the current chunk continues to belong to an existing block, the matrix operation device (100) may deactivate transition information.

[0071] In one embodiment, the operation data and the transition information may be sequentially transmitted to a plurality of operation units along a data transmission path with the same transmission delay characteristics. Accordingly, each operation unit may receive appropriate transition information together in response to the time when the operation data arrives.

[0072] In step S260, the matrix operation unit (100) may update or maintain the internal scale selection state. In one embodiment, each operation unit may update the internal scale selection state to a state corresponding to the next scale value when the transition information is enabled. On the other hand, when the transition information is disabled, the current internal scale selection state may be maintained.

[0073] In one embodiment, the update of the internal scale selection state may be implemented by increasing the scale index, moving a pointer that specifies the next scale value, or transitioning a state bit indicating the next selection state. For example, when S0 is applied to a first block and transition information corresponding to the start of a second block is activated, the matrix operation device (100) may update the internal scale selection state to a state corresponding to S1. When the transition information is deactivated, the matrix operation device (100) may maintain the current S0 corresponding state as is and apply the same scale value to subsequent data within the same block.

[0074] In one embodiment, each operation unit may operate such that an update based on transition information is performed in response to the time when the operation data reaches each operation unit. Accordingly, even if there is a pipeline delay between a preceding operation unit and a succeeding operation unit, the timing of the application of data and scale state at each operation unit can be maintained in alignment. In this case, the transition information may be applied to each operation unit in response to the time when the operation data reaches each operation unit.

[0075] In step S270, the matrix operation unit (100) can perform a scale-applied matrix operation. In one embodiment, each operation unit can select a scale value corresponding to the current internal scale selection state from a stored scale vector and perform a matrix operation by applying the selected scale value to the operation data of the current block. At this time, each operation unit can select one scale value specified by the current internal scale selection state from among a plurality of stored scale values. This matrix operation may be a multiplication, accumulation, inner product operation, or a partial operation of matrix multiplication.

[0076] In one embodiment, each operation unit may select one of a plurality of scale values ​​stored in a scale vector storage unit and use it as an active scale. The selected active scale may be applied to the currently input operation data or to the data corresponding to the block currently being processed. For example, a first scale value may be applied to the first block, and a second scale value corresponding to the updated internal scale selection state may be applied to the second block after the block boundary. The scale application method may be multiplication correction, decoding correction, or normalization coefficient reflection for the operation data.

[0077] In one embodiment, when multiple scale values ​​corresponding to a plurality of blocks are included in a single scale vector packet and stored in parallel, each operation unit can select the scale value at the location indicated by the current internal scale selection state and apply it to the operation data of the current block. Accordingly, the same scale vector packet can be utilized sequentially across multiple blocks.

[0078] In step S280, the matrix operation device (100) can determine whether there is residual operation data. In one embodiment, the matrix operation device (100) can determine whether there is an unprocessed subsequent block or subsequent data chunk, and if there is residual operation data, the procedure of steps S240 to S270 can be repeated. If there is no residual operation data, the matrix operation procedure can be terminated and the final operation result can be output or recorded in the result storage device (20). For example, if the entire input matrix data is divided into multiple blocks, the matrix operation device (100) can determine whether there is data for the next block remaining after the operation for the current block is completed. If there is a next block, the operation data of the next block and the corresponding identification information and transition information are transmitted again, and matrix operations for the subsequent block can be continued according to the updated or maintained internal scale selection state.

[0079] In the procedure according to FIG. 2, the transmission and storage of a scale vector, initialization of an internal scale selection state, transmission of computational data and identification information, provision of transition information corresponding to block boundaries, updating or maintaining an internal scale selection state, and matrix operations applying a selected scale value can be connected as a single continuous data flow control procedure. Accordingly, appropriate scale values ​​can be applied to each block in a consistent manner while utilizing the same data transmission path, and block-unit quantization operations can be efficiently performed without repeatedly supplying scale vectors from the outside.

[0081] FIG. 3 is a block diagram illustrating the detailed configuration of a matrix operation device according to one embodiment of the present invention.

[0082] The configuration illustrated in FIG. 3 can be configured to store matrix data and scale vectors transmitted through the same data transmission path separately, synchronously update an internal selection state according to transition information corresponding to block boundaries, and select a value corresponding to the current block from among a plurality of stored scale values ​​and provide it to a matrix operation unit. In particular, since the order of scale application for multiple blocks can be maintained using internal state transitions without repeatedly supplying separate scale values ​​for each block from the outside, it can have structural features in that block-unit precision correction is possible without increasing transmission bandwidth.

[0083] Referring to FIG. 3, the matrix operation device (100) may include an input control unit (110), a scale storage unit (120), a state control unit (130), a scale selection unit (140), and a matrix operation unit (150).

[0084] The input control unit (110) can receive matrix data and scale vectors input from an external device and transmit them to an internal processing path.

[0085] In one embodiment, the input control unit (110) may include a matrix data receiving function and a scale vector receiving function, and may transmit internally in conjunction with identification information to distinguish whether the current input is matrix data or a scale vector for data input in a time-division manner through the same data transmission channel. For example, the input control unit (110) may control the transmission of a scale vector in the first section and matrix data to be operated on in the second section and thereafter, and may provide an identification bit indicating the current data type at each transmission point. Accordingly, the scale storage step and the operation step can be separated in the same transmission structure without a separate dedicated bus.

[0086] In one embodiment, the matrix data and scale vector are transmitted via time division through a main data path having a wide bandwidth, and the identification information (Scale ID) and transition information (Step Signal) can be transmitted through a scale sideband that is synchronized with the main data path and wired side-by-side. This enables control using only a combination of 1-bit signal lines without a separate complex instruction decoder or global wiring, thereby minimizing the unit area of ​​individual computation units and significantly reducing routing congestion in a systolic array structure where multiple computation units (e.g., Dot Product Engines (DPE)) are densely packed.

[0087] Additionally, the input control unit (110) can arrange the transmission order of matrix data divided into multiple blocks in block units so that subsequent state control and scale selection align with the block boundaries.

[0088] The scale storage unit (120) can store a scale vector transmitted from the input control unit (110). In one embodiment, the scale storage unit (120) may include a scale vector storage function and an identification information correspondence function, and may recognize input data as a storage target only when the identification information represents a scale vector.

[0089] For example, when the identification information is in a first state, the scale storage unit (120) can decompose the input data into multiple scale values ​​and sequentially store them in an internal storage register. At this time, the stored multiple scale values ​​can be arranged to correspond to multiple blocks.

[0090] In one embodiment, the scale storage unit (120) may maintain the stored scale values ​​in a sequential index structure and may have an address correspondence structure so that the scale value at a specific location can be selected according to the selection state of the state control unit (130).

[0091] In addition, if a single scale vector packet contains multiple scale values ​​corresponding to multiple blocks, the scale storage unit (120) can expand the packet-unit input into multiple internal entries and store them. Accordingly, the entire subsequent block processing can be supported solely by internal storage without repeatedly transmitting scale values ​​from the outside for each block.

[0092] The state control unit (130) can interpret transition information corresponding to block boundaries and update or maintain the currently applied scale selection state. In one embodiment, the state control unit (130) may include a transition information interpretation function and a selection state update function. For example, if the currently input matrix data is data within the same block, the state control unit (130) can maintain the selection state, and if transition information corresponding to the start of a new block is input, it can switch to a state corresponding to the next scale value.

[0093] In one embodiment, the selection state may be maintained in the form of a scale index value, a status bit, or a pointer. For example, the first index may be maintained during the first block processing, and the second index may be incremented upon block boundary detection. Additionally, the state control unit (130) may adjust the timing of the application of transition information by taking into account the operation pipeline delay. That is, delay alignment may be performed so that the selection state is valid on the same time axis as the time when the actual operation data reaches the matrix operation unit (150). Accordingly, even if block boundary detection occurs at a preceding time, the scale value that exactly corresponds to it can be selected at the time of actual scale application. Through such a state-based control structure, the block-unit scale transition order can be maintained internally without an external control signal.

[0094] The scale selection unit (140) can select a scale value corresponding to the current selected state from the scale storage unit (120) and provide it to the matrix operation unit (150). In one embodiment, the scale selection unit (140) may include a scale value selection function and a block correspondence application function. For example, when the state control unit (130) maintains a third state, the scale selection unit (140) may select a value at a third position among the stored multiple scale values ​​and correspond it to the currently input block data.

[0095] In one embodiment, since the selection state is maintained within the same block, the same scale value can be applied repeatedly, and after the block boundary, the next scale value can be automatically selected according to the updated state. Additionally, the scale selection unit (140) may not simply transmit the selected scale value, but may align or expand it to fit the current data length, block size, or internal operation width. For example, an 8-bit scale value can be expanded to fit the internal multiplier input width and provided to the matrix operation unit (150). By this structure, a single scale vector stored can be utilized sequentially for the entirety of multiple blocks.

[0096] The matrix operation unit (150) can perform matrix operations by applying a selected scale value to the currently input matrix data. In one embodiment, the matrix operation unit (150) may include a scale application operation function and an operation result output function. For example, the selected scale value can be multiplied to the matrix data of the current block to restore it to a normalized value, and then an accumulation operation or matrix multiplication can be performed.

[0097] In one embodiment, the matrix operation unit (150) can maintain continuous operations within the same pipeline even if different scale values ​​are applied to each block. That is, at each block transition point, a new scale value can be reflected solely through the internal operations of the state control unit (130) and the scale selection unit (140) without interrupting external operations. Additionally, the matrix operation unit (150) can transmit the result generated after the scale application to an accumulator or output buffer, and can maintain pipeline connections to ensure continuity with the next block processing if necessary.

[0098] In the structure according to FIG. 3, the application of scale required for block-unit quantization operations can be realized solely through internal state transitions without external resupply. In particular, since the order of scale application for multiple blocks can be maintained without repeatedly inserting separate control packets at each block boundary or retransmitting scale values, precision and computational efficiency can be simultaneously secured without increasing data bandwidth. Furthermore, even if the block size changes, the same hardware structure can be reused by adjusting only the state transition rules and the corresponding structure of the stored scale vectors, thereby ensuring structural scalability for various quantization units.

[0100] FIG. 4 is a diagram illustrating the hardware block configuration of a data flow-controlled matrix operation device for applying block-unit scale according to an embodiment of the present invention.

[0101] The configuration illustrated in FIG. 4 can exemplify how the input control unit (110), scale storage unit (120), state control unit (130), scale selection unit (140), and matrix operation unit (150) described in FIG. 3 can be subdivided and implemented on an actual hardware path. That is, FIG. 4 can illustrate an internal implementation structure for processing scale vectors and input data transmitted through a first path, and data of another axis or input data of other characteristics transmitted through a second path, in a consistent manner within a single operation unit. In particular, the drawing may reflect the core data flow control structure of the present invention in that the block-unit scale value is not distributed through a separate large-scale global control network separated from the data, but rather the internal scale control unit is configured to control the selection state of the stored scale value based on identification information and transition information associated with the input data.

[0102] Referring to FIG. 4, the matrix operation device (100) may include a first path control unit and a second path control unit as specific implementations of the input control unit (110), and the operation unit (e.g., Dot Product Engine (DPE)) may include a scale buffer corresponding to a scale storage unit (120), a scale control unit corresponding to a state control unit (130), a selection structure corresponding to a scale selection unit (140), and an operation block including a multiplier and an accumulator corresponding to a matrix operation unit (150).

[0103] The first path control unit can control the transmission of block-shaped scale values ​​and input data of the first path by time division. In the example of FIG. 4, a tag corresponding to a scale identifier is set in the section where the scale vector is transmitted, and a tag indicating whether the data is valid and whether it corresponds to a block boundary is set in the section where the computation data is transmitted. In this way, the first path control unit, as an embodiment of the input control unit (110) of FIG. 3, can combine data and control information into a single transmission flow and provide it to a computation unit.

[0104] In one embodiment, the first path control unit may provide a step signal along with input data in accordance with the block size. This step signal may indicate that specific input data corresponds to the start of a new block and may imply that the next scale value should be used starting from that input data. Accordingly, which of the multiple scale values ​​stored in the scale buffer should be applied to the current block can be determined without external redirection during the operation. For example, the first path control unit may first transmit a single scale value or a group of multiple scale values, and then continuously transmit operation data through the same physical data path. In this case, the scale transmission section and the operation data transmission section may be distinguished by different tag combinations, and the operation unit may interpret this to perform the storage operation and the operation operation separately.

[0105] The second path control unit is an example of an implementation for another axis input path of the input control unit (110), and can control the block scale according to the characteristics of the data input through the second path. In the example of FIG. 4, the second path control unit is configured separately from the first path and is illustrated so that independent scale control is possible for the second path data as well. This structure may be useful when row direction data and column direction data have different characteristics or are subject to different scale policies.

[0106] In one embodiment, the second path control unit may generate control information for the second path according to the block unit, precision, or arrangement method of the data input through the second path. For example, the first path data may have a scale transition at regular block sizes, and the second path data may have different block lengths or different transition cycles. In this case, the second path control unit may provide the data in an aligned manner so that a subsequent scale control unit can interpret it. Such a second path control structure may serve as the basis for an axis-specific asynchronous scale synthesis structure described in subsequent embodiments.

[0107] The operation unit receives data and control information transmitted from the first path control unit and the second path control unit, and can perform operations reflecting the application of block-unit scales. In the example of FIG. 4, the operation unit includes a scale control unit, a scale buffer, and multiplier and accumulator blocks. This structure shows that the input data is not supplied directly to the operation block, but rather the operation is performed after undergoing a control process that determines the currently active scale by reflecting the type of input data and whether it is a block boundary.

[0108] The scale control unit inside the operation unit is a specific implementation of the state control unit (130) of FIG. 3, and interprets the input tags and determines whether the current input is scale data or operation data, and can update or maintain the internal scale selection state according to the block boundary. In the example of FIG. 4, control information such as step, svid, and dvid is included in the tag field labeled as scale tags, and it is illustrated that the current input is determined whether to be loaded into the scale buffer through a determination block corresponding to is_scale.

[0109] In one embodiment, the scale control unit may control the storage of input data in the scale buffer when a value corresponding to the scale identifier is activated. For example, the current input may be processed as a storage target in the form of if(vid) scale_buffer = data. On the other hand, during the operation data input section, the storage operation may not be performed, and the currently selected scale value may be controlled to be supplied to the operation path.

[0110] In the example of FIG. 4, a scale counter is included, and the internal counter is expressed to operate in the form of if(vid) cnt = 0, elif(step) cnt++. This implies that the counter is initialized when the scale buffer is loaded, and the counter is incremented whenever step information corresponding to the block boundary is input to select the next scale value. This internal counter structure may correspond to an implementation example of the internal scale selection state defined in the claims of the present invention.

[0111] The scale buffer is an example of an implementation of the scale storage unit (120) of FIG. 3 and can store multiple scale values ​​transmitted through the first path. In the example of FIG. 4, the scale buffer is shown in a form having multiple entries, and a scale value corresponding to the current cnt value is selected through the selection structure at the bottom and supplied to the operation block.

[0112] In one embodiment, the scale buffer can store multiple scale values ​​included in a single scale vector packet in parallel. For example, if S0, S1, S2, and S3 are stored, S0 may be selected if the cnt value is 0, and S1 may be selected if the cnt value is 1.

[0113] The above selection structure is an example of an implementation of the scale selection unit (140) of FIG. 3, and can select the current target scale value according to the internal state.

[0114] An operation block including a multiplier and an accumulator is an example of an implementation of the matrix operation unit (150) of FIG. 3, and can perform actual matrix operations using the currently input operation data and an active scale value selected from the scale buffer.

[0115] In the hardware structure according to FIG. 4, as an implementation of the input control unit (110), the first path control unit transmits the scale vector and operation data in a time-division manner; as an implementation of the state control unit (130), the scale control unit interprets the tag and transition information to update the state; as an implementation of the scale storage unit (120), the scale buffer stores multiple scale values; the scale selection unit (140) selects the current scale value; and the matrix operation unit (150) is configured to perform an operation by applying the selected scale, thereby enabling efficient matrix operation with block-unit quantization applied.

[0117] FIG. 5 is a timing diagram illustrating how a scale transition operation corresponding to the preloading of a scale vector and a block boundary is performed on the time axis in a data flow-controlled matrix operation device for applying a block unit scale according to an embodiment of the present invention.

[0118] The operation according to FIG. 5 may be an example in which the input control unit (110), scale storage unit (120), state control unit (130), scale selection unit (140), and matrix operation unit (150) described in FIG. 3 operate sequentially in cooperation along the time axis. That is, a scale vector is first transmitted through the main data bus, and in the section where the scale vector is transmitted, the input control unit (110) activates scale identification information so that it is stored in the scale storage unit (120); in the section where the operation data is transmitted, the state control unit (130) maintains the current internal scale selection state, and the scale selection unit (140) supplies the currently selected scale value to the matrix operation unit (150); and at a point corresponding to the block boundary, a step signal corresponding to the transition information is activated so that the internal scale selection state of the state control unit (130) is updated, thereby applying a new scale value from the next block onwards. Through such time axis-based control, even if the scale vector and the operation data share the same data transmission path, the scale value corresponding to each block can be applied in alignment.

[0119] Referring to FIG. 5, the clock cycle, main data bus, scale identification information, step signal, and internal state of the operation unit are arranged in chronological order from left to right. This configuration visually shows how externally provided data and control information correspond to the internal state of each function block of FIG. 3 along the time axis. In one embodiment, the main data bus may be a data transmission path having a width of 128 bits, the scale identification information may be 1 bit of control information, and the step signal may also be 1 bit of transition information.

[0120] The first section illustrated in FIG. 5 may be a preloading section of a scale vector. In this section, the input control unit (110) may transmit a scale vector including a plurality of scale values ​​S0 to S15 through the main data bus. At this time, the scale identification information may be provided in an active state, and the step signal may be maintained in an inactive state. The scale storage unit (120) interprets that the scale identification information is activated, determines that the current input data is a scale vector rather than computational data, and may store the transmitted scale vector in an internal storage structure.

[0121] For example, if 16 scale values ​​of 8 bits each are included in a single 128-bit transmission unit, the scale storage unit (120) can receive these multiple scale values ​​simultaneously and load them into an internal storage structure. By this structure of pre-loading scale vectors before the start of operation, in the subsequent block processing section, the necessary values ​​among the multiple scale values ​​stored internally can be sequentially selected and used without retrieving scale values ​​from the outside for every block. The state control unit (130) can set the internal scale selection state to an initial state during this pre-loading section. For example, the scale index can be set to 0 so that the first scale value S0 among the stored scale vectors corresponds to the currently active scale value.

[0122] The second section illustrated in FIG. 5 may be a section for performing operations on the first block. In this section, the input control unit (110) may transmit a chunk of operation data through the main data bus. At this time, the scale identification information may be provided in an inactive state, and the step signal may also be maintained in an inactive state. The state control unit (130) may maintain an internal scale selection state, and the scale selection unit (140) may select one scale value specified by the current state among a plurality of stored scale values.

[0123] For example, if the internal scale selection state maintains the initial state, the scale selection unit (140) can select S0 and supply it to the matrix operation unit (150). The matrix operation unit (150) can perform operations using the input data chunk and the selected S0. In the example of FIG. 5, the part expressed as "Compute(Data * S0)" can represent this operation.

[0124] As such, while multiple data chunks belonging to one block are input continuously, the step signal is kept in an inactive state, so the internal state of the state control unit (130) does not change, and the same scale value can be repeatedly applied to the data belonging to the same block.

[0125] The third section illustrated in FIG. 5 may be a scale transition section corresponding to a block boundary. In this section, the input control unit (110) may transmit a chunk of operation data belonging to a new block. At this time, the scale identification information may be in an inactive state, but the step signal may be provided in an active state. The state control unit (130) may update the internal scale selection state to a state corresponding to the next scale value in response to the activated step signal.

[0126] For example, a state control unit (130) that previously maintained a state corresponding to S0 can transition to a state corresponding to S1 in response to the activation of a step signal. In the example of FIG. 5, the part indicated as "Index++, S0 -> S1" can represent such a state transition.

[0127] The scale selection unit (140) can select S1 according to the updated state, and the matrix operation unit (150) can perform operations by applying S1 to the new block data. In the example of FIG. 5, the part expressed as "Compute(Data * S1)" may indicate that a new scale value is applied to the new block data immediately after the state transition. In this process, the phenomenon of a pipeline stall occurring while waiting for a global control command for scale change is prevented, and seamless, immediate operations can be performed by being perfectly synchronized with the data flow.

[0128] In the structure according to FIG. 5, a scale vector is preloaded before the start of the operation, and while operation data is input, the scale value pointed to by the current internal state among the stored multiple scale values ​​is repeatedly applied, and at the block boundary point, the internal state is updated according to the transition information and the next scale value is applied. Accordingly, scale switching can be accurately performed while maintaining the data transmission flow even in a data environment where block-unit quantization is applied.

[0129] According to this embodiment, the scale application order of multiple blocks can be maintained with only a single scale vector preloading operation without the need to repeatedly transmit scale values ​​in block units, thereby improving the efficiency of data transmission bandwidth usage. In addition, since block boundary corresponding transition information is transmitted in a timely alignment with the computation data, the scale value corresponding to each block is applied at the correct time, thereby improving the accuracy and reliability of block-unit quantization operations.

[0131] FIG. 6 is a diagram illustrating the concepts of scale vector packing and data block mapping according to one embodiment of the present invention.

[0132] The concept illustrated in FIG. 6 can represent a structure in which the input control unit (110), scale storage unit (120), state control unit (130), and scale selection unit (140) described in FIG. 3 cooperate to form a single scale vector packet with multiple scale values ​​corresponding to each of the multiple blocks, store this internally, and then sequentially apply it to multiple data blocks input via a time division method. With such a structure, scale application to the entire set of multiple blocks can be supported using only a single scale vector packet without individually and repeatedly transmitting scale values ​​for each block.

[0133] Referring to FIG. 6, a set of logical scale values ​​is shown at the top. In one embodiment, the input control unit (110) may set a plurality of scale values ​​S0 to S15, each corresponding to a plurality of blocks of input data, into a single scale group. These plurality of scale values ​​may each have a predetermined number of bits, such as 8 bits, and each value may represent a scale value to be applied to a specific data block. For example, S0 may correspond to the 0th data block, S1 to the 1st data block, and S15 to the 15th data block.

[0134] The part labeled "Scale Group: S0 ~ S15" at the top of Fig. 6 may represent an example where multiple logical scale values ​​are prepared as a single group. In this case, the scale values ​​are individual values ​​that exist independently, but they can be combined into a single scale vector packet on the actual data transmission path.

[0135] A physical bus transmission unit is illustrated in the upper right corner of FIG. 6. In one embodiment, the input control unit (110) may vectorize and pack the set of logical scale values ​​to form a single scale vector packet. In the example of FIG. 6, it is labeled "Scale Vector Packet 128-bit Single Flit," which indicates an example where multiple scale values ​​are included in and transmitted within a single 128-bit transmission unit.

[0136] For example, if there are 16 8-bit scale values, the input control unit (110) can pack these 16 scale values ​​into a single 128-bit packet and transmit them in a single physical transmission. At this time, the input control unit (110) can activate identification information in the section where the scale vector packet is transmitted so that the scale storage unit (120) recognizes it as a scale vector to be stored rather than as computational data in a subsequent step.

[0137] The scale storage unit (120) can load the transmitted scale vector packet into an internal storage structure based on the identification information. The part labeled "Stored in Register & Applied Sequentially" at the bottom of FIG. 6 may indicate the process of loading a scale vector packet, which has been physically transmitted once, into an internal storage structure.

[0138] For example, the scale storage unit (120) can store S0 to S15 by corresponding them to internal entries. By storing multiple scale values ​​at once in this way, the number of transmissions can be reduced compared to a method of transmitting scale values ​​corresponding to the number of blocks separately for each block.

[0139] A time-divided data stream is illustrated in the lower part of FIG. 6. In one embodiment, the state control unit (130) may maintain or update the internal scale selection state, and the scale selection unit (140) may select one of the stored scale values ​​according to the internal state.

[0140] For example, when a zero data block is input immediately after storing a scale vector packet, the state control unit (130) can maintain the initial state, and the scale selection unit (140) can select S0 among the stored multiple scale values ​​and correspond it to the zero data block. Subsequently, when transition information corresponding to the block boundary is input, the state control unit (130) can update the internal scale selection state to the next position, and the scale selection unit (140) can select S1 and correspond it to the first data block.

[0141] In the same way, S2, S3, ..., S15 can be sequentially corresponded to subsequent data blocks. The structure shown in FIG. 6, where S0 is applied to Data Block 0, S1 to Data Block 1, and S15 to Data Block 15 using dotted arrows, can represent this sequential correspondence relationship.

[0142] For example, in the initial state, a first position of the stored scale vector is selected, and whenever transition information corresponding to the block boundary is activated, the state control unit (130) can update the internal scale selection state to the next position. Accordingly, the scale selection unit (140) can select an appropriate scale value on a block-by-block basis by utilizing the stored scale vector packet as if it were an internal scale value table.

[0143] The structure illustrated in FIG. 6 may also be meaningful from the perspective of combining a data transmission path and a control structure. The input control unit (110) may induce a scale storage operation using identification information at the time of scale vector packet transmission, and then provide computation data through the same data transmission path at the time of data block transmission. In this case, even if the same data transmission path is used commonly for scale vector transmission and computation data transmission, the storage operation and the computation operation can be performed separately internally.

[0144] In the structure according to FIG. 6, multiple logical scale values ​​are vectorized and packed into a single scale vector packet and physically transmitted in batches. After being pre-loaded into the scale storage unit (120), they can be sequentially applied to multiple data blocks input in a time-division manner by the state control unit (130) and the scale selection unit (140). Accordingly, since scale application to the entire block can be supported without repeatedly transmitting scale values ​​for each block, the efficiency of data transmission bandwidth usage can be improved. In addition, since a single stored scale vector packet is sequentially utilized for multiple blocks, consistency of scale application and continuity of computation processing can be simultaneously secured in a block-unit quantization operation environment.

[0146] FIG. 7 is a drawing illustrating an example of scale application in a first block size according to an embodiment of the present invention.

[0147] The example according to FIG. 7 can show a process in which, when the block size is set to 16, multiple scale values ​​included in a single scale vector packet or scale group are applied in a one-to-one correspondence to multiple data blocks that are input consecutively. This example may be intended to explain that block-unit scale application can be accurately achieved while using the same data transmission path by first storing the scale vector and then updating the internal scale selection state according to transition information corresponding to the block boundary.

[0148] Referring to FIG. 7, the data to be used as an example may consist of 512 1-byte elements, and the basic operation unit may be 16 bytes. Accordingly, the entire data may be divided into B0 to B31 or multiple corresponding blocks, and when the block size is set to 16, one scale value may be configured to correspond to one data block. In the example of FIG. 7, the set of scale values ​​may be provided as 32 1-byte scale values, and these scale values ​​may be arranged in order corresponding to the block sequence, such as S0, S1, ..., S31. In this case, S0 may correspond to the 0th block, S1 to the 1st block, and S15 to the 15th block, and subsequent scale values ​​may be sequentially corresponded to the subsequent blocks.

[0149] In one embodiment, at time t = 0, the input control unit (110) may provide an input including a scale value S0 through a data transmission path. At this time, identification information may be activated to indicate that the current input is a scale vector or a scale value. Accordingly, the scale storage unit (120) may record the input S0 in an internal scale storage structure, and the state control unit (130) may set the internal scale selection state to an initial state. As an embodiment of the internal scale selection state, cnt = 0 may be set. This initial state may indicate that the scale value at a first position among the stored scale values ​​should be used as the currently active scale value.

[0150] At time t = 1, the input control unit (110) may provide the 0th data block B0 through a data transmission path. At this time, identification information may be disabled to indicate that the current input is operation data, and transition information may be disabled to indicate that it does not yet correspond to a block boundary. Accordingly, the state control unit (130) may maintain an internal scale selection state, and the scale selection unit (140) may select S0 indicated by the current internal scale selection state cnt = 0. The matrix operation unit (150) may perform an operation after applying the selected S0 to B0. The indication "scale[0] reflected after operation" in FIG. 7 may indicate the operation in which S0 is applied to the 0th block data in this manner.

[0151] At time t = 2, the input control unit (110) can provide the first data block B1 through the data transmission path. At this time, the identification information may be disabled to correspond to the operation data, and the transition information may be enabled to indicate that it corresponds to the new block boundary. Accordingly, the state control unit (130) can increase the internal scale selection state from cnt = 0 to cnt = 1, and the scale selection unit (140) can select S1 corresponding to the increased state. The matrix operation unit (150) can perform an operation after applying the selected S1 to B1. The indications "cnt++" and "scale[1] reflected after operation" in FIG. 7 may indicate the transition of the internal selection state and the application of the new scale value accordingly.

[0152] Subsequently, based on the same principle, in an input section that does not correspond to a block boundary, transition information is deactivated so that the state control unit (130) can maintain the internal scale selection state, and in an input section corresponding to the next block boundary, transition information is activated so that the state control unit (130) can increase the internal scale selection state again. Accordingly, the scale selection unit (140) can sequentially correspond S2 to B2, S3 to B3, ..., S15 to B15. The "..." mark shown at the bottom left of FIG. 7 may indicate that this repetitive state transition and scale application structure is performed in the same way for intermediate blocks.

[0153] An example of a subsequent scale section is illustrated on the right side of FIG. 7. For example, at time t = 17, the input control unit (110) can transmit the first scale value of the subsequent scale vector packet or the starting scale value of the subsequent scale group, and the scale storage unit (120) can store this, after which the state control unit (130) can set the internal scale selection state back to the initial state, for example, cnt = 0. Subsequently, when B16 is input at time t = 18, the first scale value of the stored section can be applied since the transition information is in a deactivated state, and when B17 is input at time t = 19 and the transition information is activated, cnt is increased so that the next scale value of the section can be selected. This structure demonstrates that the entire data is not processed by only one scale group, but rather that multiple scale groups or scale vector packets are supplied sequentially to cover the entire data range.

[0154] In one embodiment, the notations "scale 0~15" and "scale 16~31" shown in FIG. 7 may indicate that a set of scale values ​​can be divided into multiple sections and stored or transmitted. In this case, the scale vector may be transmitted again for each section, and at each transmission point, identification information is activated so that the scale storage unit (120) stores a new scale value, and then the state control unit (130) can reset the internal scale selection state. This structure may be effective even when the total data length is long and it is difficult to cover all blocks with a single scale vector packet.

[0155] The example in FIG. 7 can explain the most direct application structure in which one scale value corresponds to each block when the block size is 16. In this structure, since the cycle in which transition information is activated coincides with the block transition cycle, the number of increases in the internal scale selection state of the state control unit (130) and the increase in the block number can be coupled in the same order. Therefore, the number of the block currently being processed and the number of the scale value to be selected can correspond with only a simple state increase rule.

[0156] For example, if the block size is 16 and the basic operation unit is also 16 bytes, one data block can correspond to one operation unit. In this case, one block of data is input in each operation cycle, and whether a new scale value is applied starting from the next block can be determined based on whether transition information is active. Therefore, it is possible to control that one scale value corresponds exactly to one block operation result, and block-by-block precision correction can be clearly maintained.

[0157] In the example according to FIG. 7, a scale vector is stored first, and then when operation data is input in block units, the state control unit (130) increases the internal scale selection state according to the transition information, and the scale selection unit (140) selects a scale value corresponding to the increased state, and the matrix operation unit (150) can sequentially apply it to each block. Accordingly, in an environment where the block size is 16, the one-to-one correspondence of scale values ​​per block can be maintained through simple internal state transitions, and an appropriate scale can be accurately reflected in each block without repeatedly supplying scale values ​​from the outside for each block. In addition, since such a structure can align the scale transition point and the data block boundary in matrix operations where block-unit quantization is applied, it can simultaneously improve operation accuracy and implementation efficiency.

[0159] FIG. 8 is a drawing illustrating an example of scale application in a second block size according to an embodiment of the present invention.

[0160] The example according to FIG. 8 can show a structure in which, when the block size is set to 32, a single scale value is repeatedly applied to multiple consecutive data blocks, and the internal scale selection state is updated at longer intervals as transition information corresponding to the block boundaries is activated at a predetermined period. This example may be intended to explain that different block sizes can be accommodated by adjusting only the activation period of the transition information without significantly changing the configuration of the stored scale vector itself.

[0161] Referring to FIG. 8, the data to be used as an example may consist of 512 1-byte elements, and the basic operation unit may be 16 bytes. In this case, the entire data may be divided into B0 to B31 or a plurality of corresponding data blocks. In the example of FIG. 8, since the block size is set to 32, one scale value may correspond to two consecutive data blocks. For example, the zero scale value S0 may be applied commonly to B0 and B1, and the first scale value S1 may be applied commonly to B2 and B3, and in the same way, subsequent scale values ​​may be sequentially applied to subsequent data block pairs.

[0162] In one embodiment, at time t = 0, the input control unit (110) may provide an input including a scale value S0 through a data transmission path. At this time, identification information may be activated to indicate that the current input corresponds to a scale vector or a scale value. Accordingly, the scale storage unit (120) may record the input S0 in an internal scale storage structure, and the state control unit (130) may set the internal scale selection state to an initial state. As an embodiment of the internal scale selection state, cnt = 0 may be set. This initial state may indicate that the scale value at a first position among the stored scale values ​​should be used as the currently active scale value.

[0163] At time t = 1, the input control unit (110) may provide the zero data block B0 through a data transmission path. At this time, the identification information may be disabled to indicate that the current input is operation data, and the transition information may be disabled to indicate that it is not yet time for scale transition. Accordingly, the state control unit (130) may maintain an internal scale selection state, and the scale selection unit (140) may select S0 indicated by the internal scale selection state cnt = 0. The matrix operation unit (150) may perform an operation after applying the selected S0 to B0. The indication "scale[0] reflected after operation" in FIG. 8 may indicate such an operation.

[0164] At time t = 2, the input control unit (110) can provide the first data block B1 through the data transfer path. At this time, the identification information may remain in a deactivated state to correspond to the operation data, and the transition information may also remain in a deactivated state. This may be an important difference compared to FIG. 7. In the case where the block size is 16, the transition information could be activated whenever moving to the next data block, but in the current example where the block size is 32, since two data blocks correspond to one scale value, the transition information may not yet be activated at the time of B1 processing. Accordingly, the state control unit (130) can maintain the existing state without updating the internal scale selection state, and the scale selection unit (140) can select S0 again. The matrix operation unit (150) can perform the operation by applying the same S0 to B1. The indication in FIG. 8, "If step = 0, reuse existing scale value without index update," can explain this operating principle.

[0165] At time t = 3 or the corresponding next transition time, the input control unit (110) may provide a second data block B2 through a data transmission path. At this time, identification information may be disabled to correspond to operation data, and transition information may be enabled. The activation of the transition information may indicate that B2 corresponds to the start of a new scale application period. Accordingly, the state control unit (130) may increase the internal scale selection state from cnt = 0 to cnt = 1, and the scale selection unit (140) may select S1 corresponding to the increased state. The matrix operation unit (150) may perform an operation after applying the selected S1 to B2. The indications "cnt++" and "scale[1] reflected after operation" in FIG. 8 may indicate such internal state transition and the application of a new scale value.

[0166] Subsequently, when B3 is input, the transition information can be deactivated again, and the state control unit (130) can maintain the internal scale selection state without updating it. Accordingly, the scale selection unit (140) can select S1 as is, and the matrix operation unit (150) can apply S1 to B3. With this structure, a single scale value can be repeatedly applied to two consecutive blocks.

[0167] The example up to B15 shown in the lower left of FIG. 8 may indicate that the same rule is repeated for the intermediate blocks. For example, S2 corresponds to B4 and B5, S3 corresponds to B6 and B7, and in the same way, S7 corresponds to B14 and B15. In the example of FIG. 8, the indication that scale[7] is reflected as the processing result of B15 at t = 16 may show that a total of 16 data blocks can be divided into 2 blocks and processed by 8 scale values ​​S0 to S7. This reflects the fact that the block size is set to twice the basic operation unit.

[0168] An example of a subsequent data interval is illustrated on the right side of FIG. 8. For example, when transition information is activated when B16 is input at time t = 17, the state control unit (130) can increase the internal scale selection state again to transition to a state corresponding to S8. Accordingly, the scale selection unit (140) can select S8, and the matrix operation unit (150) can apply S8 to B16. Subsequently, when B17 is input, the transition information is deactivated, so the same S8 can be maintained and applied. In the same way, each scale value can be repeatedly applied to subsequent data blocks in pairs of two blocks. In the example of FIG. 8, the indication that scale

[15] is reflected when processing the last block indicates that the entire data range is processed sequentially and correspondingly applied up to the last scale value.

[0169] In one embodiment, the example of FIG. 8 may show a structure in which a scale vector packet contains 16 scale values, and each scale value is repeatedly applied to two data blocks. In this case, even if only one scale vector packet is physically loaded, it may be possible to logically apply scale to all 32 data blocks. That is, even when the block size is 32, the range of data that a single scale value must process is widened, but the storage structure of the scale storage unit (120), the internal scale selection state structure of the state control unit (130), and the data transmission path structure can be maintained as they are. This implies that the structure of the present invention is not limited to a specific block size and can flexibly respond to various quantization units.

[0170] For example, when the block size is 16, transition information is activated for every data block, and the internal scale selection state of the state control unit (130) can be increased, but when the block size is 32, transition information can be set to be activated only once every two data blocks. In this way, by varying only the activation cycle of the transition information, the reuse length of each scale value can be adjusted while maintaining the same internal counter structure and the same scale vector storage structure. This can be advantageous in that it allows for immediate support of algorithms for various block sizes (e.g., MXFP format, etc.) simply by adjusting the timing of the application of transition information (e.g., step signal) at the software or compiler level, without the need to redesign the hardware.

[0171] In the example according to FIG. 8, as the block size is set to 32, a single scale value is repeatedly applied to multiple consecutive data blocks, and transition information is activated only after all of the multiple data blocks have been processed to update the internal scale selection state. Accordingly, scale values ​​can be reused for longer data ranges using a single stored scale vector packet, and a block-unit precision correction structure can be maintained while reducing the frequency of scale value switching. Furthermore, since block size changes can be realized solely by adjusting the internal transition cycle, implementation efficiency and structural scalability can be improved in that the same data flow control structure can be reused for various quantization environments.

[0173] FIG. 9 is a diagram illustrating an internal processing structure of an operation unit and a structure in which a plurality of operation units are connected at an array level according to an embodiment of the present invention.

[0174] The configuration illustrated in FIG. 9 can be configured such that an operation unit receives matrix data, scale identification information, and transition information together through an integrated input bus, performs a scale vector storage operation according to the scale identification information, updates an internal scale selection state according to the transition information, selects a scale value corresponding to the updated state, applies it to the operation data, and then performs an accumulation operation. Due to this internal processing structure, even if multiple operation units are connected in a systolic array or a similar array structure, each operation unit can perform block-unit scale application according to the same data flow rules without a separate, complex external global control network.

[0175] Referring to Fig. 9, a structure is illustrated in which a plurality of operation units are connected along row-direction paths and column-direction paths in the global view area on the right.

[0176] In one embodiment, a row direction driver can provide row direction input data to a first operation unit DPE_00 along an integrated row path, and the first operation unit DPE_00 can transmit an integrated signal after processing to the next operation unit DPE_01. A column direction driver can provide column direction input data to a first operation unit DPE_00 along an integrated column path, and the first operation unit DPE_00 can transmit an integrated signal after processing to the next operation unit DPE_10. By configuring the row direction and column direction paths to intersect at a single operation unit in this manner, each operation unit can perform a corresponding operation at the point where the row direction data and the column direction data meet.

[0177] The above row direction driver, column direction driver, and integrated input bus may be specific implementation examples in which the input control unit (110) of FIG. 3 provides input to a plurality of operation units.

[0178] In one embodiment, each path may not only transmit simple data bits, but may also transmit integrated input bits including scale identification information and transition information along with the data bits. Accordingly, at the array level, each operation unit may have the same input interpretation structure regardless of its position, and scale vector loading and scale transition control may be performed autonomously within each operation unit.

[0179] On the left side of FIG. 9, the internal processing structure of the first operation unit DPE_00 is illustrated in detail. In one embodiment, the first operation unit DPE_00 may include an integrated input bus, a sideband signal check and decode block, an ID check block, a step check block, a write enable block, an index counter, a data path router, a scale vector register, a scale multiplexer, a MAC core, an accumulator, a pipeline register, and an integrated output bus. Such a structure may be a specific implementation of the data transmission path, scale vector storage unit, state control unit, scale selection unit, and matrix operation unit defined in the claims.

[0180] The integrated input bus can receive input bits including data bits, scale identification information, and transition information. The integrated input bus may be an embodiment of the data transmission path defined in the claims. In the example of FIG. 9, it is shown in the form of {Data (128b), Scale ID (1b), Step (1b)}. This structure may mean that scale vectors and computational data can be transmitted through the same physical transmission resource, along with control information indicating what meaning they have.

[0181] For example, if scale identification information is enabled, the current input is data for loading a scale vector, and if scale identification information is disabled, the current input may be data for operation. If step information is enabled, the current input point may correspond to the start of a new block or the point at which a new scale value is applied.

[0182] The sideband signal inspection and decode block can interpret scale identification information and transition information input from the integrated input bus. In one embodiment, the decode block can branch and transmit the input control information to a scale storage operation and a state transition operation. For example, scale identification information can be transmitted to an ID inspection block, and step information can be transmitted to a step inspection block. Due to this structure, multiple control information contained within a single input bus can be interpreted in parallel by different internal function blocks.

[0183] The ID check block can determine whether the current input is a scale vector. In one embodiment, the ID check block can perform a conditional determination such as "Scale ID = 1?", and if the determination result is true, it can activate the write enable block to allow the current input data to be stored in the scale vector register. "Yes (Load Mode)" in FIG. 9 may represent this scale loading mode.

[0184] For example, if Scale ID is set to 1 in the section where the scale vector packet is transmitted, the ID check block can recognize the current data bit as a storage target rather than a computation target, and can allow it to be written to the scale vector register through the write enable block. At this time, multiple transmitted scale values ​​can be stored as candidate scale values ​​inside the scale vector register.

[0185] The step check block can determine whether transition information is activated. In one embodiment, the step check block can perform a conditional determination such as "Step = 1?" and update an index counter if the determination result is true. "Yes (Trigger)" in FIG. 9 may indicate a case where a block boundary corresponding trigger occurs.

[0186] For example, if the current input data corresponds to the start of a new block, Step may be provided as 1, and the step check block may detect this and provide an update signal to the index counter to transition the internal scale selection state to the next position. Conversely, if Step is 0, the current internal scale selection state may be maintained. Such a structure may be a hardware implementation example of the block boundary-based state transition operation described in FIGS. 5 to 8.

[0187] The write enable block can control the storage operation of the scale vector register. In one embodiment, if the ID check block determines that the current input is a scale vector loading section, the write enable block may be activated, and as a result, the current input data bits may be written to the scale vector register. Due to this structure, even if the scale vector and the operation data share the same input bus, the scale loading operation and the operation target data processing operation can be separated without conflict within the operation unit. Accordingly, even if the same integrated input bus is used, the scale vector storage operation is performed in the section where the identification information is enabled, and the same data path can be switched to the operation data transfer path in the section where the identification information is disabled.

[0188] An index counter can operate as a state variable indicating the internal scale selection state. In one embodiment, the index counter may be initialized when a scale vector is loaded and may be incremented when step information corresponding to a block boundary is activated. This counter value may indicate which scale value at the current position among a plurality of stored scale values ​​should be used. For example, immediately after loading the scale vector, the index counter may be set to 0 so that the first scale value is selected, and after the first block boundary, it may be incremented to 1 so that the second scale value is selected. Since the counter value is maintained while no block boundary occurs, the same scale value may be repeatedly applied to multiple data chunks within the same block.

[0189] The data path router can branch data bits received from the integrated input bus to an appropriate internal processing path. In one embodiment, the data path router may vary the output path depending on whether the current input data is a scale storage target or an operation target data. For example, during the scale loading section, the data bits may be provided to the scale vector register side along with a write enable signal, and during the operation data section, the data bits may be provided to the operand path on the MAC core side.

[0190] This structure may mean that even if the same data bit path is physically maintained as a single unit, it can be logically separated into a storage phase and an operation phase.

[0191] A scale vector register can maintain a plurality of stored scale values. In one embodiment, the register can store a plurality of scale values ​​included in a single scale vector packet in parallel, and each storage location may correspond to a specific block or a specific block interval. "Candidate Scales" in FIG. 9 may represent these plurality of candidate scale values.

[0192] For example, the scale vector register can simultaneously hold multiple scale values ​​such as S0, S1, ..., S15, and S0 can be selected when the index counter currently points to 0, and S1 can be selected when the index counter points to 1.

[0193] With this storage structure, the stored scale values ​​can be utilized sequentially according to internal state transitions without the operation of retrieving scale values ​​from the outside for each block.

[0194] A scale multiplexer can select one scale value specified by the current index counter from among multiple scale values ​​stored in a scale vector register. In one embodiment, a selection index output by the index counter can be used as a selection signal for the scale multiplexer. As a result, the scale multiplexer can output one scale value corresponding to the current block among the stored candidate scale values ​​as the active scale.

[0195] For example, if the index counter outputs 3, the scale multiplexer can select the scale value at the fourth position as the active scale. This active scale can be combined with the currently input operation data and provided to the MAC core.

[0196] The MAC core can perform multiplication and accumulation operations using the operation target data transmitted from the data path router and the active scale transmitted from the scale multiplexer. In one embodiment, the MAC core can perform multiplication operations with input data of another axis after restoring or correcting the quantized operation data to a selected scale value. At this time, since the scale value corresponding to the block currently being processed is accurately selected and applied, matrix operations reflecting block-unit precision correction may be possible.

[0197] For example, since the index counter value is maintained within the same block, the MAC core can process consecutive data chunks by repeatedly using the same active scale, and after the block boundary, it can process subsequent blocks by applying a new active scale based on the updated counter value.

[0198] The accumulator can generate a final matrix operation result by accumulating the output results of the MAC core. In one embodiment, the accumulator can sequentially accumulate the multiplication results of row and column inputs and output the accumulated result when a certain operation interval is completed. Due to this structure, the continuous accumulation operation itself can be maintained within the same pipeline even if different scale values ​​are applied to each block.

[0199] A pipeline register can operate as a relay structure for transmitting bits of the integrated input bus to a subsequent operation unit. In one embodiment, the pipeline register can pass both data bits and control bits through so that the subsequent operation unit receives the same input format. "Pass-through all bits" in FIG. 9 may represent the pass-through structure.

[0200] For example, even if the current operation unit DPE_00 stores the input scale vector internally, the same input bit can be passed to the subsequent operation unit DPE_01 or DPE_10 through the pipeline register. Accordingly, multiple operation units can share the same scale vector or perform state transitions according to the same rules.

[0201] The integrated output bus can provide output bits that have passed through the pipeline registers to the next operation unit. In one embodiment, the integrated output bus may include not only data bits but also scale identification information and transition information. By this structure, each operation unit in the array can receive an input having the same semantics as its preceding operation unit, and the procedures of storage, state transition, scale selection, and operation can be repeatedly applied throughout the array.

[0202] In the structure according to Fig. 9, each operation unit can independently interpret data and control information received through the integrated input bus to perform scale vector storage, update internal scale selection state, active scale selection, and matrix operations. Accordingly, even without a separate complex central control network directly distributing the scale application timing for each operation unit across the entire array, each operation unit can autonomously perform block-unit scale application using only the same data flow rules.

[0203] Furthermore, since data bits and control bits are passed directly to the subsequent operation unit through pipeline registers, the timing of operation data arrival and scale transitions at each operation unit can be maintained in alignment, even if there is a positional difference between the preceding and succeeding operation units. Accordingly, the block-by-block scale application order can be maintained solely through internal state transitions after pre-loading the scale vector once, thereby ensuring both structural scalability and computational efficiency simultaneously, even as the array size increases.

[0205] FIG. 10 is a flowchart illustrating a selective scale maintenance and conditional transition operation procedure according to one embodiment of the present invention.

[0206] The procedure according to FIG. 10 may be a procedure that receives current block data and transition information, first determines whether the maintenance condition is satisfied, maintains the current scale selection state as is if the maintenance condition is satisfied, and if the maintenance condition is not satisfied, determines whether the transition condition is satisfied to update to the next scale selection state, and performs matrix operations by applying the scale value determined as a result. Through such a procedure, the scale selection state is not always mechanically changed even if a block boundary exists, but rather the scale transition can be selectively controlled by reflecting the characteristics of the current data interval, a block reuse policy, or a pre-set maintenance condition.

[0207] Referring to FIG. 10, in step S1010, the matrix operation device (100) may input current block data and transition information. In one embodiment, the matrix operation device (100) may receive block data currently being processed and transition information indicating whether the block data corresponds to a block boundary together through a data transmission path. The transition information may indicate whether the current input data corresponds to the start of a new block or whether it is a point in time when a scale selection state can be switched. For example, in a block size 16 environment, transition information may be activated at every block start time, and in a block size 32 environment, if two consecutive blocks are set to share the same scale value, the transition information may be inactive at the start of the first block and may be activated only at a point after the second block. In one embodiment, the current block data may be a single data chunk or may be part of multiple data constituting a single block.

[0208] In step S1020, the matrix operation device (100) can determine whether a maintenance condition is satisfied. In one embodiment, the maintenance condition may be a condition that allows the current scale selection state to be maintained as is. The maintenance condition may be satisfied when the current block and the subsequent block share the same scale value, when the current block is included within the existing scale application range, or when scale conversion is unnecessary within the same precision group.

[0209] For example, if a single scale value is set to be repeatedly applied to two consecutive blocks, the retention condition can be satisfied even if the second block is input immediately after the first block is processed. In this case, the existing scale value can be used as is without increasing the scale selection state.

[0210] In one embodiment, whether the maintenance condition is satisfied can be determined based on a block index, the number of repeated uses, scale reuse range information, quantization group information, or a predefined policy table.

[0211] In step S1030, the matrix operation device (100) may maintain the current scale selection state if the maintenance condition is satisfied. In one embodiment, the matrix operation device (100) may maintain the current value as is without changing the index counter, pointer, or status bit indicating the internal scale selection state. Accordingly, among the stored multiple scale values, the scale value currently selected may continue to be applied to subsequent blocks. For example, if the currently active scale is scale[5] and the maintenance condition is satisfied, the matrix operation device (100) may apply scale[5] as is to the next block without increasing the internal index. Such maintenance operation can prevent unnecessary state updates in sections where scale switching is unnecessary and can enable stable precision correction for consecutive blocks to which the same scale value must be applied.

[0212] In step S1040, the matrix operation device (100) can determine whether the transition condition is satisfied if the maintenance condition is not satisfied. In one embodiment, the transition condition may be a condition that requires updating the current scale selection state to the next state. The transition condition may be established by the start of a new scale application interval, the end of the number of scale reuses, reaching a block boundary, entering a new quantization group, or satisfying a predefined scale transition rule. For example, if a scale value is set to be applied only up to two consecutive blocks, the maintenance condition may not be satisfied at the time when the third block is input, and at this time, the transition condition may be satisfied so that the value changes to the next scale value. In one embodiment, if the transition condition is not satisfied, the current scale selection state may be conservatively maintained. Due to this structure, a transition is not performed unconditionally immediately merely because the maintenance condition is not satisfied, but additional judgment regarding the actual necessity of a transition may be possible.

[0213] In step S1050, the matrix operation device (100) may update to the next scale selection state when a transition condition is satisfied. In one embodiment, an index counter, which is an implementation of the internal scale selection state, may be incremented, and the next scale value corresponding to the incremented value may be designated as the active scale. For example, if the current active scale is scale[7] and the next block corresponds to the starting point of a new scale interval, the matrix operation device (100) may update the internal index counter to 8 so that scale[8] becomes the next active scale. In one embodiment, the update is not limited to a simple increment method, but may jump to a specific different location according to a predefined mapping table or switch to a specific group unit. However, the key point of this embodiment is that the selection state is updated only when a transition is required, and when a transition is not required, the maintenance is selected.

[0214] In step S1060, the matrix operation device (100) may select a scale value according to a maintained or updated scale selection state. In one embodiment, the matrix operation device (100) may select one scale value indicated by the current internal scale selection state among a plurality of scale values ​​stored in the scale storage unit. For example, if the state is maintained in step S1030, the existing scale[5] may continue to be selected, and if the state is updated in step S1050, a new scale[8] may be selected. Through such a selection procedure, the state control result may be converted into a specific scale value to be used in the actual operation step.

[0215] In step S1070, the matrix operation device (100) can perform matrix operations by applying a selected scale value. In one embodiment, the current block data can be combined with the scale value selected in step S1060 and used for MAC operations or other matrix operations. For example, if a maintenance path is selected, the current block operation can be performed with the existing scale[5] applied as is, and if a transition path is selected, the subsequent block operation can be performed with the updated scale[8] applied. This structure allows for selective control of the reuse or transition of scale values ​​according to block characteristics while maintaining the same hardware operation path.

[0216] In step S1080, the matrix operation device (100) may determine whether there is remaining operation data and repeat or terminate the operation. In one embodiment, the matrix operation device (100) may repeat the procedure of steps S1010 through S1070 if there are unprocessed subsequent blocks. On the other hand, if there are no more subsequent blocks, the current matrix operation may be terminated and the result may be output or recorded in the result storage device (20). For example, if the entire data stream is divided into multiple blocks, the maintenance condition and the transition condition may be re-evaluated for each block, and the scale selection state may be maintained or updated according to the result. Through such a repetition structure, an optional scale maintenance policy may be applied across the entire data range.

[0217] In the procedure according to FIG. 10, even if block boundaries or transition information exist, the scale selection state is not always changed; instead, maintenance conditions are determined first, and only if maintenance conditions are not satisfied are transition conditions additionally determined to decide whether to perform a scale transition. Accordingly, a single scale value can be used repeatedly for the required range, and unnecessary scale transitions can be reduced while transitioning to the next scale value precisely at the point where an actual transition is needed. Furthermore, since the transition frequency and scale reuse length can be adjusted according to changes in block size, data characteristics, or precision policies while maintaining the same internal state control structure, the flexibility of scale application, computational continuity, and hardware utilization efficiency can be improved simultaneously.

[0219] FIG. 11 is a flowchart illustrating an axis-by-axis asynchronous scale synthesis-based matrix operation procedure according to an embodiment of the present invention.

[0220] The procedure according to FIG. 11 may be a procedure in which row direction data and row direction scale vectors and column direction data and column direction scale vectors are respectively transmitted, and then the row direction internal scale selection state and the column direction internal scale selection state are independently updated or maintained, and the scale values ​​selected on each axis are combined to generate a final scale value, and then the combined scale value is applied to perform matrix operations. Through such a procedure, unlike a method in which only one common scale is uniformly applied to the entire operation, different scale application policies can be independently reflected in the row direction data flow and the column direction data flow, and precision correction can be performed while maintaining operational consistency even when the scale transition timings of the two axes do not coincide.

[0221] Referring to FIG. 11, in step S1110, the matrix operation device (100) may transmit row direction data and a row direction scale vector. In one embodiment, the matrix operation device (100) may provide row direction input data and a row direction scale vector in the form of time-division or separated control intervals through a row direction data path. In this case, the row direction scale vector may include a plurality of row direction scale values ​​corresponding to each of a plurality of data blocks arranged consecutively in the row direction. For example, when row direction data is divided into blocks of 16 dimensions for processing, the matrix operation device (100) may construct and transmit row direction scale values ​​corresponding to each 16-dimensional block into a single row direction scale vector.

[0222] In one embodiment, the row direction scale vector may be stored in row direction operation units or in a row direction scale storage structure within each operation unit. In this case, the row direction internal scale selection state may be a state that specifies which scale value among the stored multiple row direction scale values ​​is to be used. For example, if the row direction scale vector includes multiple scale values ​​such as R0, R1, R2, and R3, R0 may be activated in the initial state, and subsequently, R1, R2, and R3 may be sequentially selected in response to row direction block boundaries or row direction transition information.

[0223] In step S1120, the matrix operation device (100) may transmit column-direction data and a column-direction scale vector. In one embodiment, the matrix operation device (100) may provide column-direction input data and a column-direction scale vector through a column-direction data path. The column-direction scale vector may include a plurality of column-direction scale values ​​corresponding to a plurality of data blocks or a plurality of column tiles arranged consecutively in the column direction. For example, if the column-direction data is divided into blocks of 32 dimensions and processed, the column-direction scale vector may include column-direction scale values ​​C0, C1, C2, etc., corresponding to each 32-dimensional block. As such, since the row direction and the column direction may have different block sizes, the row-direction scale transition period and the column-direction scale transition period may be set differently.

[0224] In one embodiment, the column-direction scale vector may be stored separately from the row-direction scale vector. Accordingly, each operation unit may have a storage structure for the row direction and a storage structure for the column direction, respectively, and may independently maintain the scale selection state for the two axes.

[0225] In step S1130, the matrix operation device (100) may update or maintain the row direction internal scale selection state. In one embodiment, the matrix operation device (100) may determine whether the row direction block currently being processed is the start of a new scale application section based on row direction transition information associated with the row direction data. If the row direction transition information is activated, the row direction internal scale selection state may be updated to a state corresponding to the next row direction scale value, and if it is not activated, the current state may be maintained. For example, if the row direction block size is 16 and the row direction data has a structure where the scale varies from block to block, the row direction transition information may be activated once every 16 dimensions. In this case, the matrix operation device (100) may use R0 when processing row direction data block B_row0 and transition the row direction internal scale selection state to a state corresponding to R1 at the start of the next block B_row1.

[0226] In one embodiment, when the row direction block size is larger or the same scale is reused in multiple row direction blocks, the current row direction internal scale selection state can be maintained according to the maintenance conditions, even if row direction transition information exists. Accordingly, the row direction scale application cycle can be flexibly adjusted according to data characteristics or design policies.

[0227] In step S1140, the matrix operation device (100) can update or maintain the column direction internal scale selection state. In one embodiment, the column direction internal scale selection state can be controlled independently of the row direction internal scale selection state. That is, when column direction transition information is enabled, the column direction internal scale selection state can be updated to a state corresponding to the next column direction scale value, and when column direction transition information is disabled, the current state can be maintained. For example, if the column direction block size is 32 and one column direction scale value is applied to two or more consecutive column blocks, the first column block and the second column block may use the same column direction scale value C0, and the column direction internal scale selection state may be transitioned to a state corresponding to C1 only at the start of the third column block.

[0228] In this way, when row-direction scale transitions occur every 16 dimensions and column-direction scale transitions occur every 32 dimensions, at some point only the row-direction state may be updated while the column-direction state is maintained, and at other point only the column-direction state may be updated while the row-direction state is maintained. This means that the state of each axis can be managed independently within the computing unit even if scale transitions of the two axes do not necessarily occur at the same time.

[0229] This structure can be distinguished from conventional methods that use a single common scale transition signal for all directions or force the scale transitions of two axes to align at the same period. In this embodiment, by separating the state transitions of the two axes, precision control reflecting axis-specific characteristics can be achieved even when the data distribution differs for each axis.

[0230] In step S1150, the matrix operation device (100) can select a row direction scale value and a column direction scale value, respectively. In one embodiment, the matrix operation device (100) can select one of the stored row direction scale values ​​according to the row direction internal scale selection state and one of the stored column direction scale values ​​according to the column direction internal scale selection state. For example, if the current row direction internal scale selection state points to R3 and the column direction internal scale selection state points to C1, the matrix operation device (100) can select R3 as the row direction scale value and select C1 as the column direction scale value.

[0231] In one embodiment, this selection can be made simultaneously at the same time of operation. That is, each operation unit can obtain one row-direction scale value and one column-direction scale value together within the same operation cycle. Accordingly, scale information independently calculated on each axis can be prepared to be combined into a single final operation unit.

[0232] In step S1160, the matrix operation unit (100) can synthesize the selected row-direction scale value and the column-direction scale value. In one embodiment, the synthesis can be performed by multiplying the row-direction scale value and the column-direction scale value. For example, the final scale value S_final can be calculated as R3 × C1. In this case, a single synthesized scale value (e.g., S_final = S_row × S_col) can be generated in which the normalization coefficient of the row-direction data corresponding to the input activation (e.g., S_row) and the normalization coefficient of the column-direction data corresponding to the weight (e.g., S_col) are simultaneously reflected.

[0233] In one embodiment, the synthesis may not be limited to simple multiplication. For example, in a specific implementation, the larger or smaller of the two values ​​may be selected, or a weighted combination may be performed according to a predefined function. However, the key feature of this embodiment is that scale values ​​independently selected in the row direction and column direction, respectively, are combined immediately before the final operation. For example, if the amplitude distribution of row-direction data changes significantly while the distribution of column-direction data is relatively stable, the row-direction scale value may change frequently, while the column-direction scale value may be maintained for a long period. Even in such situations, the synthesis step can combine the current R and C values ​​and use them for the final operation. Accordingly, even if the data characteristics of each axis differ, it is possible to control the synthesis precision that reflects the characteristics of each of the two axes without enforcing only a single common precision policy.

[0234] In step S1170, the matrix operation device (100) may perform matrix operations by applying a synthesized scale value. In one embodiment, each operation unit may apply the synthesized scale value generated in step S1160 when performing multiplication or accumulation of row-direction data and column-direction data. For example, when quantized row-direction data and quantized column-direction data are input, the matrix operation device (100) may perform MAC operations after simultaneously performing precision restoration or normalization correction of the two inputs using the synthesized scale value.

[0235] For example, when the row direction scale value R3 = 0.25 and the column direction scale value C1 = 0.5, the composite scale value can be 0.125, and the matrix operation device (100) can apply this to the current combination of row direction data and column direction data to perform multiplication and accumulation operations at the corresponding position. By independently selecting and synthesizing the scale values ​​of the two axes in this way, it is possible to respond more precisely to changes in data distribution compared to a method of applying only a single scale to the entire block or entire tile at once. In particular, even when the dynamic range of the row direction data and the column direction data are different, the final calculation error can be reduced by reflecting axis-specific information.

[0236] In step S1180, the matrix operation device (100) may determine whether there is remaining operation data and repeat or terminate the operation. In one embodiment, the matrix operation device (100) may determine whether there are any row-direction data blocks or column-direction data blocks that have not yet been processed, and if there is remaining data, it may repeat the procedure of steps S1110 to S1170. On the other hand, if processing of all row-direction data and column-direction data is completed, the matrix operation may be terminated and the result may be output or stored in the result storage device (20). For example, if there are 32 row-direction blocks and 16 column-direction blocks, the matrix operation device (100) may sequentially perform operations corresponding to each block pair, and each time, independently update or maintain the row-direction state and column-direction state. Through this iterative structure, asynchronous scale synthesis by axis may be applied to the entire matrix area.

[0237] In the procedure according to FIG. 11, row-direction data and column-direction data may each have independent scale vectors and transition rules, and after the row-direction internal scale selection state and the column-direction internal scale selection state are independently updated or maintained, the two selected scale values ​​can be combined into a single composite scale value and applied to matrix operations. Accordingly, precision control reflecting the data characteristics of each axis can be achieved even if the timing of the scale transitions of the two axes does not necessarily coincide. Furthermore, since axis-specific state transitions and axis-specific scale synthesis can be performed autonomously by the internal computing unit, control complexity is reduced compared to a method where the scales of the two axes are externally synchronized and reset every time, and flexible response to various block size combinations and quantization policies is possible.

[0239] FIG. 12 is a flowchart illustrating a block unit matrix operation control procedure according to the application of asynchronous scale by axis according to an embodiment of the present invention.

[0240] The procedure according to FIG. 12 may be a procedure that determines the row state and column state for the current operation block, respectively, independently determines whether a scale transition is required for each axis, updates only the scale selection state of the axis requiring transition while maintaining the scale selection state of the axis not requiring transition, and performs matrix operations reflecting the state of each axis. Through such a procedure, even if the block boundaries of the row and column directions reach different points in time or the scale update cycles of the two axes differ, the state can be selectively transitioned only for the necessary axes without forcibly synchronizing the two axes. Therefore, it is possible to maintain operational consistency while more directly reflecting the data distribution characteristics and block structure differences of the two axes.

[0241] Referring to FIG. 12, in step S1210, the matrix operation device (100) can determine the row direction state of the current operation block. In one embodiment, the matrix operation device (100) can determine which section the operation block currently being processed belongs to based on the row direction scale application criteria. This row direction state can indicate whether the current block is a section where the existing scale should be maintained based on the row direction, or a section where a new row direction scale should be applied. For example, if the row direction data is structured such that it is blocked in 16-dimensional units and a different row direction scale value is applied to each 16-dimensional block, the matrix operation device (100) can determine whether the current operation block belongs to the k-th row direction block. At this time, if the row direction internal scale selection state corresponds to the current R3, it can be checked whether the current operation block still belongs to the range where R3 should be applied, or whether it has moved to the next row direction block where R4 should be applied.

[0242] In one embodiment, the row direction state determination may be performed based on a row direction block index, row direction step information, location information of row direction data, or the accumulated number of operation progresses.

[0243] In step S1220, the matrix operation device (100) can determine the column direction state of the current operation block. In one embodiment, the column direction state may indicate which scale application range the current operation block belongs to based on the column direction. This column direction state can be determined independently of the row direction state and may have different criteria depending on the column direction block size or column direction data characteristics. For example, if the column direction data is block-based in 32-dimensional units and a single column direction scale value is applied to two consecutive column direction blocks, the matrix operation device (100) can separately check whether the current operation block still belongs to the range where C1 should be applied, or whether it has moved to the next range where C2 should be applied.

[0244] In one embodiment, when the row block size is 16 and the column block size is 32, at the same time of operation, the row state may be a state requiring a new scale transition, while the column state may still be a state requiring the maintenance of the existing scale. The fact that the state determination of the two axes is performed independently in this way can be important in that it allows the respective block structures to be directly reflected without binding the two axes to a single common scale boundary.

[0245] In step S1230, the matrix operation device (100) may determine whether a row direction transition is necessary. In one embodiment, the matrix operation device (100) may determine whether to update the row direction internal scale selection state in the current operation block based on the row direction state determined in step S1210. For example, if the current operation block corresponds to the starting point of a new block based on the row direction, or if the number of row direction scale reuses reaches a predetermined limit, the matrix operation device (100) may determine that a row direction transition is necessary. On the other hand, if the current operation block still falls within the existing row direction scale application range, it may determine that a row direction transition is not necessary.

[0246] In one embodiment, the determination of whether a row direction transition is necessary may be made based on the row direction block boundary detection result, the row direction counter value, the row direction maintenance condition, or the row direction transition condition. For example, if the row direction scale value R3 is set to apply to only one row direction block, it may be determined that a row direction transition is necessary because it must be switched to R4 at the start of the next row direction block.

[0247] In step S1240, the matrix operation device (100) may determine whether a column-direction transition is necessary. In one embodiment, the matrix operation device (100) may determine whether the column-direction internal scale selection state in the current operation block needs to be updated based on the column-direction state determined in step S1220. For example, if the current operation block is within a repeating interval where the same column-direction scale value is applied based on the column direction, the matrix operation device (100) may determine that a column-direction transition is not necessary. Conversely, if the current operation block corresponds to the beginning of a new column-direction scale interval, it may determine that a column-direction transition is necessary.

[0248] In one embodiment, if the structure is such that a column-direction scale value C1 is applied to two consecutive blocks and C2 is applied to the next two blocks, it may be determined that a column-direction transition is not required when moving from the first block to the second block, and that a column-direction transition is required when moving from the second block to the third block. As such, since the necessity of a row-direction transition and the necessity of a column-direction transition can be calculated independently according to different criteria and cycles, the necessity of a transition for one axis may not be mechanically dependent on the other axis.

[0249] In step S1250, the matrix operation device (100) can update the scale selection state of the axis to be transitioned. In one embodiment, depending on the result of the judgment in step S1230 or step S1240, the internal scale selection state may be updated to a state corresponding to the next scale value only for the axis determined to require a transition. For example, if only a row-direction transition is required in the current operation block and a column-direction transition is not required, the matrix operation device (100) may transition only the row-direction internal scale selection state from R3 to a state corresponding to R4, and maintain the column-direction internal scale selection state as is. In this case, the axis-specific combination to be applied to the current operation block may be a combination of R4 and the existing C1.

[0250] Conversely, if only a column-direction transition is required, only the column-direction internal scale selection state is transitioned from C1 to the state corresponding to C2, while the row-direction internal scale selection state can be maintained in the existing R3 state. In this case, the axis-specific combination to be applied to the current operation block can be a combination of the existing R3 and C2.

[0251] In one embodiment, if it is determined that transitions are required for both axes, both the row direction internal scale selection state and the column direction internal scale selection state can be updated simultaneously. However, updating both axes simultaneously is only one of the possible cases, and the key point of this embodiment is that the two axes do not always have to be updated simultaneously.

[0252] In step S1260, the matrix operation device (100) may maintain the scale selection state of the non-transition axis. In one embodiment, the axis for which no transition was performed in step S1250 may maintain its current internal scale selection state. For example, if only a row-direction transition is required, the column-direction internal scale selection state may be maintained, and if only a column-direction transition is required, the row-direction internal scale selection state may be maintained. This structure ensures that even if the state of one of the two axes is changed, the state of the other axis is not unnecessarily changed together.

[0253] For example, consider a scenario where row-direction data requires frequent scaling due to minute changes in value distribution, while column-direction data maintains a relatively stable distribution. In this case, if the two axes are processed by grouping them into a common transition cycle as in conventional methods, the column-direction scale may be updated excessively frequently, potentially leading to unnecessary state transitions. In contrast, the present embodiment allows the column-direction state to be maintained even when a row-direction transition is required; thus, stable scale maintenance is possible for the column direction, while fine adaptive scale control is possible only for the row direction.

[0254] This vision-based axis retention structure can be advantageous in that it can reflect differences in axis-specific data characteristics without increasing hardware control complexity.

[0255] In step S1270, the matrix operation device (100) can perform matrix operations that reflect the axis-wise state. In one embodiment, the matrix operation device (100) can determine the row-direction scale value and the column-direction scale value to be applied to the current operation block, respectively, based on the row-direction scale selection state and the column-direction scale selection state determined as a result of steps S1250 and S1260. For example, if the row-direction state in the current operation block is updated to R4 and the column-direction state is maintained as C1, the matrix operation device (100) can perform matrix operations on the current operation block using a combination of R4 and C1. Conversely, if the row-direction state is maintained as R3 and the column-direction state is updated to C2, matrix operations can be performed based on a combination of R3 and C2.

[0256] In one embodiment, the matrix operation may be performed after individually reflecting the scale values ​​determined for each axis, and in another embodiment, it may be performed using a single final scale value formed by synthesizing the scale values ​​of two axes. In either case, the important point is that the axis-specific scale state actually applied to the current operation block is determined by reflecting the independent transition judgment results of each axis. For example, in a system where the scale is switched every 16 dimensions in the row direction and every 32 dimensions in the column direction, R4 and C1 may be applied to operation block (i, j), R5 and C1 may be applied to the next operation block (i+1, j) with only the row direction transitioning, and R5 and C2 may be applied to the next operation block (i+2, j+1) with only the column direction transitioning. Since different axis-specific combinations can be formed for each operation block in this way, much finer precision control can be achieved compared to a single common scale policy.

[0257] In one embodiment, the procedure of FIG. 12 can be repeated. That is, after row-direction state determination and column-direction state determination are performed for the current operation block, the same determination procedure can be repeated for the next operation block. Accordingly, row-direction and column-direction scale transitions can be asynchronously accumulated across the entire matrix, while maintaining an accurate axis-specific state combination for each block. For example, in an environment where the row direction changes at a fast cycle and the column direction changes at a slow cycle, the row-direction state along the entire operation area may change frequently, such as R0, R1, R2, R3, while the column-direction state may change slowly, such as C0, C0, C1, C1. According to the present embodiment, such an asymmetric transition structure can be reflected solely through internal state determination and selective updates without separate complex external control.

[0258] In the procedure according to FIG. 12, the row and column states of the current operation block are determined separately, and after independently determining whether a transition is required for each axis, matrix operations can be performed while updating only the scale selection state of the axis requiring a transition and maintaining the state of the non-transition axis. Accordingly, scale application can be controlled according to the data characteristics of each axis even if row and column transitions do not necessarily occur at the same time, and unnecessary state transitions that may occur in a method of forcibly synchronizing the two axes can be reduced. Furthermore, since it is possible to operate by finely adapting only specific axes while maintaining other axes stably, precision maintenance, control flexibility, and hardware efficiency can be simultaneously improved in a block-unit quantization environment.

[0260] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0261] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer devices and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0262] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0263] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described device, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0264] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

[0265] Although specific embodiments have been described in the detailed description of this document, it will be obvious to those skilled in the art that various modifications are possible within the scope of this document. Explanation of the symbols

[0266] 10: Host device 20: Result storage device 100: Matrix Operation Unit 110: Input control unit 120: Scale storage unit 130: State control unit 140: Scale selection unit 150: Matrix operation unit

Claims

Claim 1 A matrix operation method performed in a matrix operation device in which a plurality of operation units are connected along a data transmission path, comprising: a step of transmitting a scale vector through the data transmission path and providing identification information indicating that the scale vector is being transmitted; a step of causing each of the plurality of operation units to store the scale vector based on the identification information; a step of transmitting operation data through the data transmission path and providing transition information corresponding to a block boundary of the operation data; a step in which each of the plurality of operation units updates an internal scale selection state in response to the transition information; and a step in which each of the plurality of operation units performs a matrix operation by applying a scale value selected from the stored scale vector to the operation data according to the internal scale selection state. Claim 2 A data flow-controlled matrix operation method for block-unit scale application, wherein, in claim 1, the step of transmitting a scale vector through the data transmission path and providing identification information indicating that the scale vector is being transmitted together comprises the step of providing the identification information in a first state while transmitting the scale vector through the data transmission path, and the step of transmitting the operation data comprises the step of providing the identification information in a second state distinct from the first state while transmitting the operation data through the data transmission path. Claim 3 A data flow-controlled matrix operation method for block-unit scale application according to claim 1, wherein the step of having each of the plurality of operation units store the scale vector based on the identification information comprises: storing data input through the data transmission path as the scale vector when the identification information corresponds to the scale vector; and transmitting the same data stored as the scale vector to a subsequent operation unit along the data transmission path. Claim 4 A data flow-controlled matrix operation method for block-unit scale application according to claim 1, further comprising, after the step of having each of the plurality of operation units store the scale vector based on the identification information, the step of setting the internal scale selection state of each of the plurality of operation units to an initial state corresponding to a first scale value of the stored scale vector, and the step of applying the first scale value corresponding to the initial state to the operation data of the first block to perform matrix operation. Claim 5 A data flow-controlled matrix operation method for applying block-unit scale according to claim 1, wherein the step of providing transition information corresponding to the block boundary of the operation data includes the step of activating and providing the transition information when the operation data corresponds to the block boundary, and the step of deactivating and providing the transition information when the operation data does not correspond to the block boundary; and the step of updating the internal scale selection state includes the step of updating the internal scale selection state to a state corresponding to the next scale value when the transition information is activated, and the step of maintaining the internal scale selection state when the transition information is deactivated. Claim 6 A data flow-controlled matrix operation method for block-unit scale application, wherein the step of performing the matrix operation comprises: selecting a scale value corresponding to the internal scale selection state from the stored scale vector; and applying the selected scale value to the operation data of the current block to perform the matrix operation. Claim 7 A data flow-controlled matrix operation method for block-unit scale application, wherein, in claim 1, the step of transmitting a scale vector through the data transmission path and providing identification information indicating that the scale vector is being transmitted comprises: a step of configuring a plurality of scale values ​​corresponding to each of a plurality of blocks into a single scale vector packet and transmitting it through the data transmission path; and the step of causing each of the plurality of operation units to store the scale vector comprises a step of storing the plurality of scale values ​​included in the scale vector packet in parallel. Claim 8 A data flow-controlled matrix operation method for block-unit scale application, wherein the step of updating the internal scale selection state includes the step of performing an update based on the transition information in correspondence with the time when the operation data reaches each operation unit. Claim 9 A data flow-controlled matrix operation method for block-unit scale application according to claim 1, wherein the step of applying the scale value to the operation data to perform matrix operations comprises: a step of selecting a row-direction scale value corresponding to row-direction data and a column-direction scale value corresponding to column-direction data, respectively; a step of generating a composite scale value by multiplying the row-direction scale value and the column-direction scale value; and a step of performing matrix operations on the row-direction data and the column-direction data using the composite scale value. Claim 10 A computer-readable storage medium having a program stored on it for executing the method of any one of paragraphs 1 through 9 on a computer. Claim 11 A data flow-controlled matrix operation device for block-unit scale application, comprising: an input control unit that receives matrix data and a scale vector, and transmits identification information indicating that the scale vector is being transmitted and transition information corresponding to the block boundary of the matrix data through a data transmission path; a scale storage unit that stores the scale vector transmitted through the data transmission path based on the identification information; a state control unit that updates an internal scale selection state in response to the transition information; a scale selection unit that selects a scale value from the scale vector stored in the scale storage unit according to the internal scale selection state; and a matrix operation unit that performs matrix operations by applying the scale value selected by the scale selection unit to the matrix data.

Images