Data operation method, data operation device, and data processor

Abstract

The present invention provides a data operation method, a data operation device, and a data processor. The data operation method includes: receiving a data flushing instruction, determining a flushing target domain, wherein the data flushing instruction includes at least a first field representing the flushing target domain and a second field representing the target data type; performing a flushing operation on the flushing target domain according to the data flushing instruction, and providing target data in the flushing target domain out of the flushing target domain, wherein the target data includes object data that meets the target data type in at least one object data in the flushing target domain. The data operation method can adaptively provide object data that meets the target data type in the flushing target domain without providing all data, especially non-target data, out of the flushing target domain, thereby reducing or avoiding the "overkill" phenomenon of the artificial intelligence processor and mitigating performance losses caused by excessively frequent refresh operations.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data operation method, a data operation device, and a data processor. Background Art

[0002] Artificial intelligence is a cutting-edge, interdisciplinary discipline that integrates computer science, statistics, neuroscience, and social science. Currently, research in AI applications includes robotics, speech recognition, image recognition, natural language processing, and expert systems. The ability of AI systems to acquire knowledge independently is called machine learning. Machine learning uses large amounts of data to "train" models, using various algorithms to learn from this data how to make decisions and predictions about real-world events. Deep learning stems from scientists' discovery that the human visual system processes information in a hierarchical manner. High-level features are combinations of lower-level features, and the expression of features becomes increasingly abstract and conceptual from lower to higher levels. Consequently, as deep learning has become the most advanced and widely used technology in the field of AI, neural networks have also begun to develop. Summary of the Invention

[0003] At least one embodiment of the present disclosure provides a data operation method, comprising: determining a flushing target domain in response to a data flushing request, wherein the data flushing request includes a target data type; performing a flushing operation on the flushing target domain according to the data flushing request, taking object data that conforms to the target data type in at least one item of object data in the flushing target domain as target data and providing the flushing target domain.

[0004] For example, at least one embodiment of the present disclosure provides a data operation method that also includes: in the process of performing data processing on the at least one item of object data before the flushing operation, adding corresponding attribute tag information to the at least one item of object data, wherein the attribute tag information includes sub-information for indicating the data type of the object data.

[0005] For example, in at least one embodiment of the present disclosure, a data operation method is provided, in which corresponding attribute tag information is added to the at least one object data, including: in the data processing process of executing the computing task, based on the hierarchical position of the at least one object data in the computing task, adding attribute tag information corresponding to the hierarchical position.

[0006] For example, in at least one embodiment of the present disclosure, a data operation method is provided, in which a flushing operation is performed on the flushing target domain according to the data flushing request, including: determining operation data from the at least one object data; in response to the data type of the operation data being the same as the target data type, using the operation data as the target data to perform the flushing operation.

[0007] For example, in at least one embodiment of the present disclosure, a data operation method is provided, wherein the data flushing request is in response to a data flushing instruction, and the data flushing instruction includes a first field indicating the flushing target domain and a second field indicating the target data type.

[0008] For example, at least one embodiment of the present disclosure provides a data operation method further comprising: during the flush operation, blocking the writing of other object data having the target data type into the flush target domain.

[0009] For example, in at least one embodiment of the present disclosure, a data operation method is provided, wherein the flushing target domain includes a cache or a buffer zone.

[0010] For example, in at least one embodiment of the present disclosure, a data operation method is provided, wherein the buffer includes an aggregation buffer for data aggregation operations.

[0011] For example, at least one embodiment of the present disclosure provides a data operation method further comprising: during the flushing operation, performing a data operation different from the flushing operation on object data in the flushing target domain whose data type is different from the target data type.

[0012] For example, in at least one embodiment of the present disclosure, a data operation method is provided, wherein the flushing target domain includes a data processing pipeline.

[0013] The data processing pipeline comprises at least one data storage unit, and the at least one data storage unit is used to store the at least one item of object data.

[0014] For example, at least one embodiment of the present disclosure provides a data operation method that also includes: providing a counting unit for the data processing pipeline, wherein the counting unit includes multiple counters corresponding to multiple data types respectively, and in response to changes in the number of object data of each data type in the data processing pipeline, the counter corresponding to each data type is synchronously increased or decreased.

[0015] For example, at least one embodiment of the present disclosure provides a data operation method further comprising: when the at least one item of object data enters the data processing pipeline, using the counter corresponding to each data type to record the number of object data of each data type in the data processing pipeline.

[0016] For example, in at least one embodiment of the present disclosure, a data operation method is provided, in which the flushing operation is performed on the flushing target domain according to the data flushing request, including: performing the flushing operation on each object data in the target data in the data processing pipeline, and after the flushing operation is performed on each object data, increasing or decreasing the counter corresponding to the target data type by 1 until the value of the counter corresponding to the target data type returns to the initial value.

[0017] For example, in at least one embodiment of the present disclosure, a data operation method is provided, in which the flushing operation is performed on each object data in the target data in the data processing pipeline, including: when the at least one object data constitutes a data queue in the data processing pipeline, the operation data is obtained in sequence in the data queue, and in response to the data type of the operation data being the same as the target data type of the data flushing request, the operation data is provided as the target data out of the data processing pipeline.

[0018] For example, in at least one embodiment of the present disclosure, a data operation method is provided, wherein the data processing pipeline is configured in a multi-core processor or a multi-processor system.

[0019] For example, at least one embodiment of the present disclosure provides a data operation method further comprising: during the flushing operation, continuing to execute a request for the flushing target domain initiated after the data flushing instruction.

[0020] For example, at least one embodiment of the present disclosure provides a data operation method further comprising: retaining the target data in the flushing target domain during or after the flushing operation.

[0021] At least one embodiment of the present disclosure provides a data operation device, comprising: a determination unit, configured to determine a flushing target domain in response to a data flushing request, wherein the data flushing request includes a target data type; a flushing unit, configured to perform a flushing operation on the flushing target domain according to the data flushing request, and use object data that conforms to the target data type in at least one object data in the flushing target domain as target data and provide the flushing target domain.

[0022] For example, at least one embodiment of the present disclosure provides a data operation device further including: a buffer, a cache, or a data processing pipeline, wherein the flushing target domain includes the buffer, the cache, or the data processing pipeline.

[0023] At least one embodiment of the present disclosure provides a data processor, comprising: an instruction memory; and an instruction execution unit, wherein the instruction execution unit fetches instructions from the instruction memory and processes the fetched instructions according to the data operation method described in any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0025] Figure 1A A schematic diagram of a neural network structure is shown;

[0026] Figure 1B A typical configuration of a convolutional neural network is shown;

[0027] Figure 1C Abstractly showing the input and output of a neuron in a convolutional neural network;

[0028] Figure 1D A schematic diagram of an analysis module of an input image using a convolutional neural network is shown;

[0029] Figure 2A is a schematic diagram of a multi-processor system;

[0030] Figure 2B is a schematic diagram of another multi-processor system; Figure 3 A schematic flow chart of a data operation method provided in at least one embodiment of the present disclosure;

[0031] Figure 4 A schematic diagram of performing a flushing operation on an aggregation buffer provided in at least one embodiment of the present disclosure;

[0032] Figure 5 A schematic diagram of performing a flush operation on a data processing pipeline according to at least one embodiment of the present disclosure;

[0033] Figure 6 A schematic block diagram of a data operation device provided in at least one embodiment of the present disclosure;

[0034] Figure 7 A schematic structural diagram of a data processor provided for at least one embodiment of the present disclosure;

[0035] Figure 8 A schematic block diagram of an electronic device provided for some embodiments of the present disclosure. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0037] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by persons of ordinary skill in the field to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0038] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and components.

[0039] Neural networks are mathematical computational models inspired by the structure of neurons in the brain and the principles of neural transmission. The method of implementing intelligent computing based on this type of model is called brain-inspired computing. For example, neural networks include various forms of network structures, such as back propagation (BP) neural networks, convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM), etc. For example, convolutional neural networks can be further subdivided into fully convolutional networks, deep convolutional networks, U-net, etc.

[0040] For example, a common convolutional neural network usually includes an input end, an output end, and multiple processing layers. For example, the input end is used to receive data to be processed, such as an image to be processed, and the output end is used to output processing results, such as a processed image. Multiple processing layers may include convolution layers, pooling layers, batch normalization layers (Batch Normalization, abbreviated as BN), fully connected layers, etc. Depending on the structure of the convolutional neural network, the processing layers may include different contents and combinations. After the input data is input into the convolutional neural network, it passes through several processing layers to obtain the corresponding output. For example, the input data can pass through several processing layers to complete operations such as convolution, upsampling, downsampling, standardization, full connection, and flattening.

[0041] Figure 1A A schematic diagram of a neural network structure is shown. The neural network is used, for example, for image processing, using images as input and output, for example, by replacing scalar weights with filters (i.e., convolution).

[0042] like Figure 1A As shown in FIG, the neural network includes three layers of neurons, namely input layer 101, hidden layer 102 and output layer 103. The output of input layer 101 is the input of hidden layer, and the output of hidden layer is the input of output layer. Input layer 101 has 4 inputs, hidden layer 102 has 3 outputs, and output layer 103 has 2 outputs. Each box of corresponds to a filter, where k is a label indicating the input layer number, and i and j are labels indicating the input and output units, respectively. Is a scalar added to the output of the convolution. The sum of several convolutions and biases is processed accordingly by an activation function, which typically corresponds to a rectified linear unit (ReLU), a sigmoid function, or a hyperbolic tangent function. In image processing systems using convolutional neural networks, the filters and biases are fixed during the operation of the system. The filters and biases are obtained in advance by using a set of input / output example images and adjusting them to meet some optimization criteria depending on the application.

[0043] Figure 1B Figure 2 shows a typical configuration of a convolutional neural network. After the image data is input into the convolutional neural network through the input layer, it passes through several processing layers (such as Figure 1B The processing of each processing layer may include convolution, pooling, batch normalization, etc. as needed.

[0044] Figure 1CThe input and output of a neuron in a convolutional neural network are abstractly illustrated. As shown in the figure, C1, C2, through Cn represent different signal channels. For a specific local receptive field (which contains multiple channels), different filters are used to convolve the data on the C1 to Cn signal channels within that local receptive field. The convolution result is input into the stimulation node, which then calculates the corresponding function to obtain feature information. Therefore, the main components of a convolutional neural network typically include: multiple convolutional layers, multiple pooling layers, and fully connected layers. A complete convolutional neural network consists of these three layers stacked together.

[0045] The convolutional layer is the core layer of a convolutional neural network. It applies several filters to the input data (image), which is then used to extract various types of features. The result of applying a filter to the input data is called a feature map, and the number of feature maps is equal to the number of filters. The feature map output by a convolutional layer can be fed into the next convolutional layer for further processing to produce a new feature map. The pooling layer is an intermediate layer sandwiched between consecutive convolutional layers, used to reduce the size of the input data and, to a certain extent, mitigate overfitting. There are many ways to implement pooling, including but not limited to max-pooling, avg-pooling, random pooling, decimation (e.g., selecting fixed pixels), and demux (splitting the input image into multiple smaller images). Typically, the last subsampling layer or convolutional layer is connected to one or more fully connected layers, the output of which is the final output, resulting in a one-dimensional matrix, or vector.

[0046] For example, see Figure 1D In the example shown, the output of the last convolutional layer (i.e., the nth convolutional layer) is input to the flattening module for flattening, converting the feature image (2D image) into a vector (1D), which can be used in the fully connected layer. The flattening operation can be performed as follows:

[0047] v k =f k / j,k％j

[0048] Here, v is a vector containing k elements, and f is a matrix with i rows and j columns.

[0049] The output of the flattening module is then input into a fully connected layer (FCN), which can be, for example, Figure 1A The convolutional network shown can use different scalar values ​​to replace the filters.

[0050] For example, the output of the last convolutional layer (i.e., the Lth convolutional layer) can also be averaged (AVG), that is, the average value of the features is used to represent the image, thereby converting the 2D feature into a single number. If the average operation is performed, the flattening module may not be required.

[0051] The output of a neural network can take various forms. For example, the activation function of the output layer can be a softmax function, a sigmoid function, or a tanh function, depending on the needs. Through the softmax function, each label is assigned a certain probability, and the label with the highest probability is selected as the label or category of the image.

[0052] For example, if the convolutional neural network is a deep convolutional neural network, it may include at least five convolutional layers. For example, the VGG-16 neural network has 16 layers, and the GoogLeNet neural network has 22 layers. Of course, other neural network structures may have more processing layers. The above content is only an exemplary introduction to the neural network, and the present disclosure does not limit the structure of the neural network. For example, a processor used to perform artificial intelligence calculations is called an artificial intelligence processor (AI Processor) or an AI chip. Currently, artificial intelligence processors can be implemented by means of CPU (central processing unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), AISC (Application Specific Integrated Circuit), etc. Neural network calculations are highly dependent on massive amounts of data. Data flows between different processing layers and between processing layers at different stages, thereby forming a powerful pipeline in the computing pipeline and cache system of the artificial intelligence processor.

[0053] To achieve parallel computing, for example, the artificial intelligence processor can be a multi-processor (or multi-processing core) system, such as a multi-core processor or a multi-processor system, that is, the artificial intelligence processor can include multiple processing cores. Since the computational load of the neural network, especially the computational load of the convolution layer, is very large, the computational operations of a processing layer in the neural network may be decomposed. For example, the convolution operations of different parts of the same convolution layer can be performed independently of each other. These decomposed tasks are assigned to multiple processing cores to perform calculations in parallel. The calculation results of these processing cores are then merged to obtain the calculation results of the entire processing layer. The calculation results of the processing layer can then be passed as input to the processing core that performs the calculation operations of the next processing layer.

[0054] See also Figure 2A-2B For example, an exemplary artificial intelligence processor is a multi-processor system, which includes multiple processing cores, such as two or more processing cores. For example, the smallest multi-processor system may include two processing cores. Here, a multi-processor system including four processing cores is used as an example for description, but of course, the present disclosure is not limited to this.

[0055] For example, the multi-processor architecture may have two modes, namely, a centralized shared memory system and a distributed memory system.

[0056] Figure 2A The figure shows a centralized shared memory architecture. The AI ​​processor includes four processing cores: core0, core1, core2, and core3. Core0, core1, core2, and core3 have shared caches and their own dedicated caches, sharing the same main memory and input / output (I / O). Each processing core's dedicated cache can include a level 1 cache (L1 cache) or a level 2 cache (L2 cache). The shared memory includes a reduction engine, for example, for performing batch normalization calculations.

[0057] For example, in one example, core0 and core1 jointly perform processing on layer 1 of a neural network, then provide the processing results to a shared cache. A normalization engine in the shared cache then performs corresponding operations, such as batch normalization calculations. Core0 and core1 then read the normalization engine's processing results from the shared cache into a local dedicated cache. For example, the shared cache provides the normalization engine's processing results to the dedicated caches of core0 and core1, allowing core0 and core1 to continue performing subsequent processing.

[0058] Figure 2BThe distributed memory system is shown, where core0, core1, core2, and core3 each have their own dedicated cache, memory, and I / O, and are connected and communicate with each other via an interconnection network using mechanisms such as messages. The dedicated cache of each processing core may include a level 1 cache (L1 Cache) or a level 2 cache (L2 Cache), etc. The interconnection network may be a bus system, a crossbar, a network on chip (NOC), etc. For example, in some embodiments, the network on chip may also be located elsewhere in the multiprocessor system, such as within a processing core, but this disclosure is not limited thereto.

[0059] The embodiments of the present disclosure do not limit the implementation of the cache, the implementation of the (main) memory, the implementation of the interconnection network, etc.

[0060] For example, when performing neural network processing through the above exemplary artificial intelligence processor structure, core0 performs the processing of processing layer 1 of the neural network, and then sends the processing results of processing layer 1 to core1 and core2, which then jointly perform the processing of processing layer 2. In this process, Figure 2A In the centralized shared memory system shown, core0 can temporarily store the processing results of processing layer 1 in the shared cache, and the normalization engine performs the corresponding processing. Later, when core1 and core2 are able to perform the calculation operations of processing layer 2, they can extract the processing results of the normalization engine from the shared cache according to the decomposed calculation tasks; or Figure 2B In the distributed memory architecture shown, core 0's processing results for processing layer 1 can be temporarily stored in core 0's dedicated cache (or memory). When core 1 and core 2 are able to perform computations for processing layer 2, core 0's processing results for processing layer 1 are provided from core 0's dedicated cache (or memory) to core 1 and core 2 via the interconnect network, based on the decomposed computational tasks. This forms a data pipeline from core 0 to core 1 and core 2.

[0061] Similarly, for example, after core1 and core2 complete processing at processing layer 2, they both send the processing results to core3 via a shared cache or interconnect network, which then performs subsequent processing. This operation includes data coalescing. For example, in this process, core1 performs part of the processing at processing layer 2 of the neural network to obtain processing result A, and core2 performs part of the processing at processing layer 2 of the neural network to obtain processing result B. Core1 provides processing result A to the coalescing buffer of core3, and core2 also provides processing result B to the coalescing buffer of core3, thereby achieving data coalescing.

[0062] In the process described above, the massive computing data of the neural network interacts, is shared, and is stored between the computing units of different processing cores of the artificial intelligence processor, the memory, and the cache of the artificial intelligence processor. Therefore, the artificial intelligence processor has multi-core and multi-level computing characteristics.

[0063] For example, in artificial intelligence processors, it is often necessary to provide data in a target domain (such as a certain level of cache, multiple levels of cache, buffer, pipeline, etc.) to the location where the data is needed for subsequent processing according to the computational data flow. For example, data in one processing core is provided to multiple processing cores that need to synchronize data. For example, in order to further perform global normalization operations (global reduction operations) in batch normalization operations, local normalized data (local reduction data) needs to be provided for inference or training, for example, the normalized forward data is provided for reuse in the backward propagation process, etc.

[0064] In parallel processors or other processors not used in AI scenarios, a flush operation is provided. This flush operation is a fence operation that can provide all the data in the flush target domain. This flush operation may also clear all the data in the target domain, or block other operations on the data in the target domain during or after the flush operation, resulting in a large delay for subsequent instructions. Frequent flush operations often result in significant performance loss. Due to the multi-core and multi-level computing characteristics of AI processors, flush operations may need to be performed frequently. In the case of multi-step pipeline processing in AI, if each flush operation provides all the data in the flush target domain, it will greatly increase the processor overhead and reduce the processor performance, and the system loss will be particularly significant. For example, in the example above, core 1 may also contain other (type) data C. If a flush operation is performed directly on core 1, the processing result A in core 1 and data C will be provided to core 3. However, core 3's processing process does not actually require data C, and data C may not have been fully processed yet, causing the processing of data C to be blocked or data C to be reloaded after the flush in order to be processed accordingly. Therefore, this flush operation that provides all the data in the target domain will lead to excessive flushing, increasing processor overhead and reducing processor performance.

[0065] At least one embodiment of the present disclosure provides a data operation method, a data operation device, and a data processor. The data operation method includes: determining a flushing target domain in response to a data flushing request, wherein the data flushing request includes a target data type; performing a flushing operation on the flushing target domain based on the data flushing request, taking object data that matches the target data type in at least one item of object data in the flushing target domain as target data, and providing the flushing target domain.

[0066] This data operation method can adaptively provide object data that meets the target data type in the flushing target domain without providing all data, especially non-target data, out of the flushing target domain, thereby reducing or avoiding the "overkilling" phenomenon of the artificial intelligence processor and slowing down the performance loss caused by excessively frequent refresh operations.

[0067] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0068] Figure 3 A schematic flowchart of a data operation method provided for at least one embodiment of the present disclosure.

[0069] For example, Figure 3As shown, the data operation method provided by the embodiment of the present disclosure includes steps S10 to S20.

[0070] For example, a data operation method provided in at least one embodiment of the present disclosure is applied to a data processor. The data processor may be an artificial intelligence processor as described above, which is used to perform artificial intelligence calculations, for example, using a neural network for training or reasoning. The embodiments of the present disclosure do not limit the type of neural network.

[0071] In step S10 , in response to a data flushing request, a flushing target domain is determined.

[0072] For example, the data flushing request includes the target data type, for example, includes identification information of the target data to be flushed.

[0073] In step S20, according to the data flushing request, a flushing operation is performed on the flushing target domain, and object data that conforms to the target data type in at least one object data in the flushing target domain is used as target data and the flushing target domain is provided.

[0074] For example, a data flush request is generated in response to a data flush instruction, which includes multiple fields, such as a first field indicating a flush target domain and a second field indicating a target data type. That is, in step S10, upon receiving the data flush instruction, the flush target domain is determined based on the information recorded in the first field of the data flush instruction, and the target data type to be flushed is determined based on the information recorded in the second field of the data flush instruction.

[0075] For example, during the working process, the flushing target domain may include one or more object data, and the multiple object data may have different data types. For example, during the data processing process of executing a computing task, the data obtained at different hierarchical positions in the computing task are labeled as different types of data. For example, if the computing task is a multi-layer neural network processing, the data obtained by different processing layers in the neural network, different positions or parts in the same layer (corresponding to different processing cores), or data at different stages are considered to be different types of data.

[0076] For example, step S20 may include: determining operation data from at least one item of object data; and performing a flushing operation using the operation data as target data in response to a data type of the operation data being the same as a target data type.

[0077] For example, one object data can be selected in sequence from multiple object data included in the flushing target domain as operation data, and it can be determined whether the data type of the operation data is the same as the target data type indicated by the data flushing instruction. If the data type of the operation data is the same as the target data type, the operation data is used as the target data and the target data is provided out of the flushing target domain. After that, the target data follows the data flow to other locations in the data processor, such as other processing cores, to perform subsequent processing. For example, the target data is provided to a shared cache, and a batch standardization operation is performed by a standardization engine in the shared cache, and the data is saved in the shared cache, waiting to be extracted again by the processing core. If the data type of the operation data is different from the target data type, no processing is performed on the operation data, and the next object data is selected in sequence as the operation data and the above process is performed.

[0078] For example, the data operation method provided by at least one embodiment of the present disclosure also includes: in the process of performing data processing on at least one item of object data mentioned above before the flushing operation, adding corresponding attribute tag information to at least one item of object data, for example, the attribute tag information includes sub-information for indicating the data type of the object data.

[0079] For example, adding corresponding attribute tag information to at least one object data may include: during the data processing process of executing the computing task, adding attribute tag information corresponding to the hierarchical position of at least one object data in the computing task.

[0080] For example, if the computing task is a neural network calculation, and the neural network includes multiple processing layers, when object data is generated by different processing layers, different positions or parts of the same layer, or different stages, corresponding attribute tag information is added to the generated object data, and the data type of the object data is indicated by sub-information in the attribute tag information. For example, the data type may include normalized data, data used for batch normalization, data after batch normalization, data processed by an activation function, etc.

[0081] For example, during a flushing operation, the writing of other data of the target data type into the flushing target domain is blocked. For example, when a flushing operation is performed on the flushing target domain, the data of the target data type currently in the flushing target domain is provided out of the flushing target domain, and other data of the target data type that is about to enter the flushing target domain is first blocked and temporarily cannot enter the flushing target domain until the flushing operation is completed.

[0082] For example, during or after a flushing operation, target data is retained in the flushing target domain. That is, in the present disclosure, "data being provided" means that the object data in the flushing target domain is sent to other components in the data processor, and the local data in the flushing target domain may be cleared or may remain in its original state. For example, the state may be set by other parameters in the data flushing instruction, and the present disclosure does not limit this.

[0083] In the data operation method provided in at least one embodiment of the present disclosure, attribute tag information is added to each object data when the object data is generated, so that when a data flushing request is received, the target data type included in the data flushing request is determined, the sub-information of the attribute tag information of each object data is compared with the target data type, and the object data that meets the target data type is provided as target data from the flushing target domain, thereby realizing adaptive adjustment of the flushing target for the target domain, avoiding over-flushing, and improving the performance of the data processor.

[0084] For example, when performing a flushing operation, the target flushing domain may include a data storage area in a data processor, such as a cache or a buffer, so that the target data in the cache or buffer is provided out of the flushing target domain. For example, the target flushing domain may also include a data processing pipeline, such as a fabric pipeline (Fabric Pipelines), so that the target data in the data processing pipeline is provided out of the flushing target domain. For example, the cache may be a dedicated cache (such as a first-level cache or a second-level cache) or a shared cache, and the buffer may include multiple types of buffers, such as an aggregate buffer, which may be implemented by a cache or main memory; for example, the data processing pipeline may include a combination of multiple levels of dedicated caches, a combination of dedicated caches and shared caches, a combination of dedicated caches and interconnected networks, etc., and may further include other units and modules connected between these storage devices, and the embodiments of the present disclosure are not limited to this.

[0085] The following describes in detail the flushing operation execution process in two scenarios with reference to the accompanying drawings.

[0086] For example, in some embodiments, the flushing target domain includes a cache or buffer.

[0087] For example, in one example, a data processor includes three levels of cache, namely cache L0, cache L1, and cache L2. A traditional flush instruction is: Flush.[bp0,bp1,bp2], where Flush is an opcode indicating a flush instruction, and bp0 to bp2 are first fields indicating flush target domains. For example, bp0 is used to indicate whether to provide data in cache L0, i.e., the flush target domain is cache L0; bp1 is used to indicate whether to provide data in cache L1, i.e., the flush target domain is cache L1; and bp2 is used to indicate whether to provide data in cache L2, i.e., the flush target domain is cache L2.

[0088] For example, in at least one embodiment of the present disclosure, a data flush instruction defines a number of mask bits as a second field, which indicate the flush target in the instruction set architecture. For example, the second field can indicate the target data type to be flushed, thereby selectively flushing data of different data types in the flush target field.

[0089] For example, the flush instruction provided in at least one embodiment of the present disclosure is in the form of: Flush.[bp0, bp1, bp2].mask, where mask is the second field. The second field can be understood as a parameter of the data flush instruction, and different values ​​of the second field represent different data types. For example, the second field may include 2 mask bits. When the value of the second field is 1, it indicates that the target data type of the data flush request is standardized data or data used for batch normalization. When the value of the second field is 2, it indicates that the target data type of the data flush request is data after batch normalization. When the value of the second field is 3, it indicates that the target data type of the data flush request is data after activation function processing. Of course, according to actual needs, the second field may include more mask bits, so that more types of data types can be represented.

[0090] For example, the target data can be adaptively adjusted according to the second field in the flush instruction. For example, one data type can perform fewer flush operations, while another data type can perform more flush operations. Each time the flush instruction is executed, only the target data is provided, and not all the data in the flush target domain is provided, thereby avoiding over-flushing and improving the performance of the data processor.

[0091] For example, the first buffer receives a data flush request, the target data type in the data flush request is data type A, a flush operation is performed on the first buffer, and the object data of data type A in the first buffer is used as target data and provided out of the first buffer, for example, the target data enters the aggregation buffer according to the data flow; the second buffer receives a data flush request, the target data type in the data flush request is also data type A, a flush operation is performed on the second buffer, and the object data of data type A in the second buffer is used as target data and provided out of the second buffer, for example, the target data also enters the aggregation buffer according to the data flow, so that the data of data type A in the first buffer and the second buffer are aggregated in the aggregation buffer, and data of other data types will not be provided out of the target flush domain.

[0092] For example, the buffer used as the flush target domain includes an aggregation buffer for a data aggregation operation. For example, when a flush operation is performed on the aggregation buffer, data in the aggregation buffer that meets the target data type determined by the data flush request is provided to the aggregation buffer, while data of other data types is not provided to the aggregation buffer.

[0093] For example, a flush instruction removes data from the flush target domain. If this instruction is set as a disguised data update, the execution of other requests will typically be suspended or delayed. Even requests unrelated to the data in the flush target domain will be suspended. If flush operations are frequently executed in an AI processor, and each flush operation delays or pauses the execution of other requests, this can significantly degrade processor performance.

[0094] To address the above situation, during a flush operation, the system can configure the system to allow data operations other than the flush operation to be performed on object data in the flush target domain whose data type is different from the target data type. For example, when the flush target domain is an aggregation buffer, only object data of the target data type in the aggregation buffer will be provided to the aggregation buffer, while the aggregation operation can continue to be performed on object data of other data types without being affected. This prevents object data of other types from being provided to the aggregation buffer before processing is complete, and allows the aggregation operation to continue on object data of other non-target data types. This improves processor performance and avoids the "over-flushing" issue.

[0095] Figure 4 A schematic diagram of performing a flushing operation on an aggregation buffer according to at least one embodiment of the present disclosure.

[0096] For example, the aggregation buffer includes a plurality of storage blocks, each storage block is used to store object data, each object data has attribute tag information, and the attribute tag information includes sub-information indicating the object data type, for example, Figure 4As shown, the object data type includes data type A and data type B. Of course, the object data type may also include more types of data types. The attribute tag information of the object data may be stored in the aggregation buffer or may be stored separately.

[0097] like Figure 4 As shown, data flushing control means parsing the data flushing instruction, obtaining the data flushing request, and determining the flushing target domain and target data type; data aggregation control means processing the data aggregation instruction to perform the data aggregation operation. For example, the aggregation processing process can be: the length of each storage block in the aggregation buffer is 512 bytes, and the four 128-byte data from the four processing cores are aggregated into one object data and stored in one storage block in the aggregation buffer.

[0098] For example, when the data processor initiates a data flush request, it first determines that the flush target domain is the aggregation buffer according to the first field in the data flush instruction, and determines the target data type according to the second field in the data flush instruction. For example, according to the mask bit of the second field, it determines that the flush target of the data flush request is object data of data type A. Then, each object data is used as operation data in turn. If the data type of the operation data is the same as the target data type, the operation data is provided as target data out of the aggregation buffer, so that only the object data of data type A in the aggregation buffer is provided as target data out of the aggregation buffer, and the object data of data type B can continue to perform data aggregation operations.

[0099] That is to say, when a flush instruction is executed, object data of non-target data types can still perform data operations different from the flush operation, such as data aggregation operations. The flush instruction will not delay / pause other operations on object data of non-target data types. These object data of non-target data types can continue to perform operations such as data aggregation operations, thereby significantly improving the aggregation efficiency. The efficiency of the shared cache or interconnection network in the multi-core processor will also be significantly improved.

[0100] For example, in some other embodiments, the flushing target domain includes a data processing pipeline. For example, the data processing pipeline can be a fabric pipeline, which refers to the transmission process of data from the processing core to the memory through one or more caches, buffers, arbiters, interconnection networks, and other structures.

[0101] For example, the data processing pipeline is configured in a multi-core processor or a multi-processor system, for example, the multi-core processor or the multi-processor system is used to perform artificial intelligence calculations.

[0102] For example, the data processing pipeline includes at least one data storage unit, and the at least one data storage unit is used to store at least one object data. For example, the at least one data storage unit may include a storage structure such as a register for temporarily storing the object data.

[0103] For example, a counting unit is provided for the data processing pipeline, for example, the counting unit includes multiple counters corresponding to multiple data types respectively, and in response to the change in the number of object data of each data type in the data processing pipeline, the counter corresponding to each data type is synchronously increased or decreased.

[0104] For example, when at least one item of object data enters the data processing pipeline, the number of data of each data type in the data processing pipeline is recorded using a counter corresponding to each data type.

[0105] For example, the target data includes one or more items of object data in the flush target domain whose data type is the same as the target data type. For example, when the flush target domain includes a data processing pipeline, performing a flush operation on the flush target domain in response to the data flush request may include: performing the flush operation on each item of object data in the target data in the data processing pipeline, and after performing the flush operation on each item of object data, incrementing or decrementing a counter corresponding to the target data type by 1 until the value of the counter corresponding to the target data type returns to an initial value.

[0106] For example, performing a flushing operation on each object data in the target data in the data processing pipeline may include: when at least one object data constitutes a data queue in the data processing pipeline, sequentially obtaining operation data in the data queue, and in response to the data type of the operation data being the same as the target data type of the data flushing request, providing the operation data as target data out of the data processing pipeline.

[0107] Specifically, if the flushing target domain includes a data processing pipeline, a counter is set for each data type to record the number of object data of that data type currently in the data processing pipeline. For example, the initial value of each counter is 1. When object data of data type A enters the data processing pipeline, the value of counter A corresponding to data type A is incremented by 1. When object data of data type B enters the data processing pipeline, the value of counter B corresponding to data type B is incremented by 1.

[0108] For example, when object data constitutes a data queue in a data processing pipeline, for example, the data queue is a FIFO (first-in-first-out) queue, an object data is used as operation data sequentially or according to a preset rule. If the data type of the operation data is the same as the target data type, the operation data is provided out of the data processing pipeline as the target data, and after the target data leaves the data pipeline, the value of the counter corresponding to the target data type is reduced by 1 until the value of the counter corresponding to the target data type returns to the initial value I.

[0109] It should be noted that when the object data enters the data processing pipeline, the value of the corresponding counter can also be reduced by 1, and when the target data leaves the data processing pipeline, the value of the corresponding counter can be increased by 1. This disclosure does not impose any restrictions on this.

[0110] For example, during a flushing operation, requests to the flush target domain initiated after the data flushing instruction continue to execute, that is, the flushing operation can be executed in parallel with the operation request initiated after the flushing instruction, and subsequent operation requests are not affected. The flushing operation only provides the target data out of the flushing target domain, and does not suspend subsequent operation requests to the flushing target domain.

[0111] For example, when a data processing pipeline uses a ping-pong buffer mechanism, two data sub-queues are provided for alternating reading and writing. A ping counter and a pong counter can be set for each data type in the two data sub-queues. The ping counter and the pong counter are used to respectively record the number of data of the data type entering the two data sub-queues. The flush instruction is completed when the values ​​of the ping counter and the pong counter both return to the initial value 1. The use of the ping counter and the pong counter can further reduce the flush range.

[0112] Figure 5 A schematic diagram of performing a flush operation on a data processing pipeline is provided for at least one embodiment of the present disclosure.

[0113] like Figure 5 As shown, the object data in the data processing pipeline constitutes a data queue, which includes two data types: data type A and data type B. Counter 0 is provided for data type A, and counter 1 is provided for data type B. The initial values ​​of counter 0 and counter 1 are both 0. When object data of data type A enters the data pipeline, the value of counter 0 is increased by 1. When object data of data type B enters the data pipeline, the value of counter 1 is increased by 1.

[0114] For example, the instructions following the data flush instruction are page table buffer lookup instructions. For example, the mappings between virtual addresses and physical addresses are stored in tables in memory, and accessing these tables in memory also requires hundreds of clock cycles. To reduce these memory accesses, the processing core uses multiple levels of cache to store recently used mappings. These caches are called page table buffers (TLBs). A page table buffer lookup instruction searches for a physical address based on a virtual address to read data cached in the page table buffer.

[0115] For example, object data flows in a data queue. When a data flush instruction is received, for example, the data flush instruction indicates that the target data type is data type A, the object data of data type A in the data queue is provided out of the data queue to the memory sub-system (memory sub-system, such as the main memory), and when the object data of data type A leaves the data queue, the value of counter 0 is reduced by 1 until the value of counter 0 is 0, indicating that the flush operation is completed.

[0116] At the same time, the object data of data type B can continue to execute the page table buffer search instruction without being paused, thereby improving the processor processing efficiency and reducing system latency.

[0117] The data operation method provided by at least one embodiment of the present disclosure can significantly improve the performance of an artificial intelligence processor used in an artificial intelligence computing system, and is particularly suitable for scenarios where data at different processing layers or stages are processed by a full pipeline.

[0118] Corresponding to the above-mentioned data operation method, at least one embodiment of the present disclosure further provides a data operation device, Figure 6 A schematic block diagram of a data operation device provided in at least one embodiment of the present disclosure.

[0119] For example, Figure 6 As shown, the data operation device 600 includes a determination unit 601 and a flushing unit 602. The data operation device 600 is used in a computing device of a multi-processor system, for example.

[0120] The determining unit 601 is configured to determine a flushing target domain in response to a data flushing request, wherein the data flushing request includes a target data type.

[0121] The flushing unit 602 is configured to perform a flushing operation on the flushing target domain according to the data flushing request, take the object data of at least one object data in the flushing target domain that conforms to the target data type as the target data, and provide the target data out of the flushing target domain.

[0122] For example, the determination unit 601 and the flushing unit 602 include codes and programs stored in a memory; the processor can execute the codes and programs to implement some or all of the functions of the determination unit 601 and the flushing unit 602 as described above. For example, the determination unit 601 and the flushing unit 602 can be dedicated hardware devices or firmware to implement some or all of the functions of the determination unit 601 and the flushing unit 602 as described above. For example, the determination unit 601 and the flushing unit 602 can be a circuit board or a combination of multiple circuit boards to implement the functions as described above. In an embodiment of the present application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processor; and (3) firmware stored in the memory that is executable by the processor.

[0123] For example, the data operation device may further include a buffer, cache, or data processing pipeline, wherein the flushing target domain includes the buffer, cache, or data processing pipeline. For details about the buffer, cache, and data processing pipeline, please refer to the above data operation method, and the repeated parts will not be repeated.

[0124] It should be noted that the determining unit 601 is used to implement Figure 3 In step S10 shown, the flushing unit 602 is used to implement Figure 3 Therefore, the detailed description of the determination unit 601 can refer to the embodiment of the above data operation method. Figure 3 For the description of step S10 shown in FIG. 1 , the specific description of the flushing unit 602 can refer to the embodiment of the above data operation method. Figure 3 In addition, the data operation device can achieve similar technical effects as the aforementioned data operation method, which will not be described in detail here.

[0125] At least one embodiment of the present disclosure further provides a data processor. Figure 7 A schematic structural diagram of a data processor provided for at least one embodiment of the present disclosure.

[0126] like Figure 7 As shown, the data processor 700 includes an instruction memory 701 , an instruction execution unit 702 and an instruction scheduler 703 .

[0127] like Figure 7As shown, the data processor 700 may include an instruction cache 701, an instruction execution unit 702, and an instruction scheduler 703. For example, the instruction cache 701 may include a cache located outside the instruction scheduler 703, which is used to cache instruction data retrieved from an external storage space by the instruction scheduler 703; the instruction scheduler 703 is used to obtain instruction data, perform instruction decoding, and dispatch instructions. For example, the instruction scheduler 703 performs an instruction fetch operation to obtain instruction data from the instruction cache 701, and the instruction scheduler 703 performs a series of instruction arbitration and instruction decoding on the obtained instruction data, and finally dispatches the selected instruction to the corresponding instruction execution unit 702 for execution.

[0128] For example, the instruction execution unit 702 fetches instructions from the instruction memory 701 and processes the fetched instructions according to the data operation method provided in at least one embodiment of the present disclosure. Figure 3-Figure 5 The content mentioned above will not be repeated here.

[0129] For example, the data processor is an artificial intelligence processor for performing artificial intelligence calculations, such as a multi-processor system. For example, the data operator can adopt a general-purpose processor architecture, such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit); for example, the data operator can adopt a dedicated processor architecture, such as a customized processor designed specifically for intelligent computing and based on an ASIC. This type of processor is specifically designed for specific types of computing tasks using dedicated hardware circuits and can complete high-energy-efficiency computing with very low power consumption; for example, the data operator can adopt a reconfigurable processor architecture, such as an FPGA (Field Programmable Gate Array) and a coarse-grained reconfigurable array (Coarse Grained Reconfigurable Architectures), which has hardware programmable capabilities. The present disclosure does not limit the structure of the data processor, and any processor structure that performs artificial intelligence calculations can be applied to the data processor provided in at least one embodiment of the present disclosure.

[0130] Figure 8 This is a schematic block diagram of an electronic device provided in some embodiments of the present disclosure. The electronic device 800 is suitable for implementing the data operation method provided in the embodiments of the present disclosure. The electronic device 800 may be a terminal device, etc. It should be noted that Figure 8 The electronic device 800 shown is merely an example and does not limit the functions and scope of use of the embodiments of the present disclosure.

[0131] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit) 810, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 820 or a program loaded from a storage device 880 to a random access memory (RAM) 830.

[0132] Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 830. The processing device 810, the ROM 820, and the RAM 830 are connected to each other via a bus 840. An input / output (I / O) interface 850 is also connected to the bus 840.

[0133] Typically, the following devices may be connected to the I / O interface 850: an input device 860, for example, the input device 860 may include a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 870, for example, the output device 870 may include a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 880, for example, the storage device 880 may include a magnetic tape, a hard disk, etc.; and a communication device 890. The communication device 890 may allow the electronic device 800 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 800 may alternatively implement or possess more or fewer devices.

[0134] For example, according to an embodiment of the present disclosure, the processing device 810 can execute the above-mentioned data operation method and realize the functions defined in the data operation method provided by the embodiment of the present disclosure.

[0135] Regarding this disclosure, the following points need to be explained:

[0136] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0137] (2) For the sake of clarity, the thickness and size of layers or structures in the drawings used to describe the embodiments of the present invention are exaggerated. It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element can be "directly on" or "under" the other element, or intervening elements may be present.

[0138] (3) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0139] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A data operation method, comprising: receiving a data flushing instruction and determining a flushing target domain, wherein the data flushing instruction at least includes a first field indicating the flushing target domain and a second field indicating a target data type; In response to the data flushing instruction, a flushing operation is performed on the flushing target domain, and target data in the flushing target domain determined based on the second field is provided to the flushing target domain, wherein the target data includes object data that conforms to the target data type in at least one item of object data in the flushing target domain, and the data type of the object data is used to indicate a hierarchical position of the object data in a process of performing data processing of a computing task; The computing task includes neural network computing, the neural network includes multiple processing layers, and the object data includes the object data generated at different processing layers, different positions or parts of the same processing layer, or different stages.

2. The method according to claim 1, further comprising: During the data processing of the at least one item of object data before the flushing operation, corresponding attribute tag information is added to the at least one item of object data, wherein the attribute tag information includes sub-information for indicating a data type of the object data.

3. The method according to claim 1, wherein In response to the data flushing instruction, performing a flushing operation on the flushing target domain includes: determining operational data from the at least one item of object data; In response to the data type of the operation data being the same as the target data type, the flushing operation is performed using the operation data as the target data.

4. The method according to any one of claims 1 to 3, further comprising: During the flushing operation, writing other object data having the target data type into the flushing target domain is blocked.

5. The method according to any one of claims 1 to 3, wherein: The flushing target domain includes a cache or a buffer zone.

6. The method according to claim 5, wherein: The buffer includes an aggregation buffer for data aggregation operations.

7. The method according to claim 5, further comprising: During the flushing operation, a data operation different from the flushing operation is performed on the object data in the flushing target domain whose data type is different from the target data type.

8. The method according to any one of claims 1 to 3, wherein: The flushing target domain includes a data processing pipeline, The data processing pipeline comprises at least one data storage unit, and the at least one data storage unit is used to store the at least one item of object data.

9. The method according to claim 8, further comprising: A counting unit is provided for the data processing pipeline, wherein the counting unit includes multiple counters corresponding to multiple data types respectively, and in response to changes in the number of object data of each data type in the data processing pipeline, the counter corresponding to each data type is synchronously increased or decreased.

10. The method according to claim 9, further comprising: When the at least one item of object data enters the data processing pipeline, the counter corresponding to each data type is used to record the number of object data of each data type in the data processing pipeline.

11. The method according to claim 10, wherein: In response to the data flushing instruction, performing the flushing operation on the flushing target domain includes: The flush operation is performed on each object data included in the target data in the data processing pipeline, and after the flush operation is performed on each object data, the counter corresponding to the target data type is increased or decreased by 1 until the value of the counter corresponding to the target data type returns to the initial value.

12. The method according to claim 11, wherein Performing the flushing operation on each object data in the target data in the data processing pipeline includes: When the at least one object data constitutes a data queue in the data processing pipeline, one object data is sequentially obtained from the data queue as operation data, and in response to the data type of the operation data being the same as the target data type of the data flush instruction, the operation data is provided out of the data processing pipeline as the target data.

13. The method according to claim 8, wherein The data processing pipeline is configured in a multi-core processor or a multi-processor system.

14. The method according to claim 8, further comprising: During the flushing operation, requests to the flushing target domain initiated after the data flushing instruction continue to be executed.

15. The method according to any one of claims 1 to 3, further comprising: The target data is retained in the flush target domain during and after the flush operation.

16. A data operation device comprising: a determining unit configured to receive a data flushing instruction and determine a flushing target domain, wherein the data flushing instruction at least includes a first field indicating the flushing target domain and a second field indicating a target data type; a flushing unit configured to, in response to the data flushing instruction, perform a flushing operation on the flushing target domain, and provide target data in the flushing target domain determined based on the second field out of the flushing target domain, wherein the target data includes object data that conforms to the target data type in at least one item of object data in the flushing target domain, and the data type of the object data is used to indicate a hierarchical position of the object data in a process of data processing for executing a computing task; The computing task includes neural network computing, the neural network includes multiple processing layers, and the object data includes the object data generated at different processing layers, different positions or parts of the same processing layer, or different stages.

17. The data operation device according to claim 16, further comprising: A buffer, a cache or a data processing pipeline, wherein the flushing target domain includes the buffer, the cache or the data processing pipeline.

18. A data processor comprising: Instruction memory; An instruction execution unit is configured to fetch instructions from the instruction memory and process the fetched instructions according to the data operation method according to any one of claims 1 to 15.

Images