Data processing method and device, storage medium and electronic equipment

Abstract

The invention provides a data processing method and device, a storage medium and electronic equipment. The data processing method comprises the steps that first data are read and written into a first area of a first target cache region, the first target cache region further comprises a second area, and first multiplex data written after last processing is stored in the second area; reading first data to be processed from the first area and the second area; and processing the first to-be-processed data by using a first deep learning sub-model. The data processing method can reduce bandwidth waste.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and relates to a data processing method and device, a storage medium and an electronic device. Background Art

[0002] A neural processing unit (NPU) is a type of hardware specifically designed to accelerate neural network computing. With the widespread application of deep learning and neural networks in various fields, traditional general-purpose computing hardware often cannot meet the needs of large-scale neural network computing, so NPU came into being. The internal structure of NPU includes a large number of computing units and high-speed caches, which can perform large-scale neural network computing such as matrix multiplication and convolution in parallel, which is conducive to improving computing efficiency. Summary of the invention

[0003] The embodiments of the present disclosure provide a data processing method and device, a storage medium, and an electronic device for reducing the read and write bandwidth of an NPU to an external memory.

[0004] In a first aspect, an embodiment of the present disclosure provides a data processing method. The data processing method includes: reading first data and writing the first data into a first area of ​​a first target cache area, the first target cache area also includes a second area, the second area stores first multiplexed data written after the last processing; reading first data to be processed from the first area and the second area; and processing the first data to be processed using a first deep learning sub-model.

[0005] In an implementation of the first aspect, reading first data and writing the first data to a first area of ​​a first target cache area includes: for at least one layer in a neural network model, reading input features corresponding to the at least one layer as the first data; and writing the first data to the first area of ​​the cache in the neural network processor.

[0006] In an implementation of the first aspect, processing the first data to be processed using the first deep learning sub-model includes: processing the first data to be processed using the first deep learning sub-model corresponding to the at least one layer to generate second multiplexed data.

[0007] In an implementation of the first aspect, the data processing method also includes: for a next layer of the at least one layer in the neural network model, reading second data corresponding to the next layer and writing the second data into the first area of ​​the first target cache area; reading second data to be processed from the first area and the second area; and processing the second data to be processed using a second deep learning sub-model corresponding to the next layer.

[0008] In an implementation of the first aspect, reading first data to be processed from the first area and the second area includes: reading data starting from the starting address of the second area until the ending address of the first target buffer area to read the first multiplexed data; continuing to read data from the starting address of the first target buffer area until the ending address of the first area to read the first data; and splicing the first multiplexed data and the first data to read the first data to be processed.

[0009] In an implementation of the first aspect, the data processing method also includes: reading third data and writing the third data into a third area of ​​a second target cache area, the second target cache area completely overlapping or partially overlapping with the first target cache area, the second target cache area also including a fourth area, the fourth area storing third multiplexed data written after the last processing; reading third data to be processed from the third area and the fourth area; and processing the third data to be processed using a third deep learning sub-model.

[0010] In an implementation of the first aspect, the first deep learning sub-model and the third deep learning sub-model are obtained by segmenting the deep learning model according to cache capacity.

[0011] In an implementation of the first aspect, the size and position of the second area are determined by a segmentation method of the deep learning model.

[0012] In an implementation of the first aspect, reading first data to be processed from the first area and the second area includes: reading data starting from a starting address of the second area, and when the ending address of the first target cache area is read, continuing to read data from the starting address of the first target cache area until the reading of the first data to be processed is completed.

[0013] In an implementation of the first aspect, the start address and the end address of the first target cache area are determined by the size of the first data, the free capacity of the cache, and the second area.

[0014] In an implementation of the first aspect, the configuration of the start address and the end address of the first target cache area is obtained by segmenting the deep learning model in the offline planning stage.

[0015] In a second aspect, an embodiment of the present disclosure provides a data processing device. The data processing device includes: a data reading and writing module, configured to read first data and write the first data into a first area of ​​a first target cache area, the first target cache area includes the first area and the second area, and the second area stores the first multiplexed data written after the last processing; a to-be-processed data reading module, configured to read the first to-be-processed data from the first area and the second area; and a data processing module, configured to process the first to-be-processed data using a first deep learning sub-model.

[0016] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed to implement the data processing method described in any one of the first aspects of the present disclosure.

[0017] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory configured to store a processor-executable program; and a processor configured to call the program to execute the data processing method according to any one of the first aspects of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a schematic diagram of the structure of the NPU in an embodiment of the present disclosure.

[0019] Figure 2A and Figure 2B Shown is a schematic diagram of a convolution operation in an embodiment of the present disclosure.

[0020] Figure 2C Shown is a schematic diagram of segmenting a deep learning model in an embodiment of the present disclosure.

[0021] Figure 2D and Figure 2E Shown is a schematic diagram of convolution 1 and convolution 2 in an embodiment of the present disclosure.

[0022] Figure 2F and Figure 2G Shown is a schematic diagram of the storage status of the cache in an embodiment of the present disclosure.

[0023] Figure 3A Shown is a flow chart of a data processing method in an embodiment of the present disclosure.

[0024] Figure 3B Shown is a schematic diagram of the storage status of the cache in an embodiment of the present disclosure.

[0025] Figure 3C Shown is a flowchart of writing first data into a first area in an embodiment of the present disclosure.

[0026] Figure 3D Shown is a flowchart of processing the second data to be processed in an embodiment of the present disclosure.

[0027] Figure 3E Shown is a flowchart of reading the first data to be processed in an embodiment of the present disclosure.

[0028] Figure 4A Shown is a flowchart of processing the third data in an embodiment of the present disclosure.

[0029] Figure 4B Shown is a schematic diagram of the storage status of the cache in an embodiment of the present disclosure.

[0030] Figure 5 Shown is a flowchart for planning cache in an embodiment of the present disclosure.

[0031] Figure 6 Shown is a partial schematic diagram of a deep learning model in an embodiment of the present disclosure.

[0032] Figure 7 Shown is a structural schematic diagram of a data processing device in an embodiment of the present disclosure.

[0033] Figure 8 Shown is a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following is an explanation of the embodiments of the present disclosure by specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0035] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present disclosure in a schematic manner, and thus the illustrations only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0036] The multiplication and addition operations in deep learning networks are mainly implemented through NPU. Figure 1 FIG. 1 is a schematic diagram showing the structure of the NPU 1 provided in the embodiment of the present disclosure. Figure 1 As shown, the NPU 1 includes a multiplier-accumulator 11 , an input circuit 12 , a controller 13 , an output circuit 14 , and a cache 15 .

[0037] The computing power of NPU 1 is mainly determined by the specifications of the multiplier accumulator 11. For example, if the number of multipliers 11 in NPU 1 is 1024, each multiplier accumulator 11 operates twice in one clock cycle, and its operating frequency is 1 GHz, then the computing power of NPU 1 is TOPs=1024×2×1 GHz=2 TOPs, indicating that NPU 1 can perform 2T operations per second.

[0038] The input circuit 12 may use a direct memory read access (RDMA) technology to read data, but the present disclosure is not limited thereto. Direct memory read access is used to bypass the central processor during data transmission and directly transfer data from an external device to the cache 15, thereby improving data transmission efficiency.

[0039] The controller 13 is responsible for managing and coordinating the various modules and functions within the NPU 1 to achieve efficient neural network computing.

[0040] The output circuit 14 may use a Write Direct Memory Access (WDMA) technology to output the multiplication and addition operation results in the cache, but the present disclosure is not limited thereto.

[0041] In some possible implementations, the NPU can be used for single-layer convolution operations in a neural network, but the present disclosure is not limited to this. In these implementations, the convolution operation includes input features, weights, biases, and output features. The input circuit reads the input features and weights into the cache. The multiplier accumulator reads the input features and weights from the cache, and multiplies and adds the input features and weights to the bias to obtain the output features. The output features are written to the cache. The specific method of the convolution operation is shown in Formula 1 below.

[0042]

[0043] In the formula, Represents the complete output feature map of the convolution, N i represents a batch of feature maps, represents the output feature map channel, express The corresponding bias, C in Represents the input feature map channel, input(N i ,k) represents the complete input feature map.

[0044] In some possible implementations, the NPU needs to read and write a large amount of data from an external memory. However, due to cost and power consumption considerations, external memories, such as DDR (Double Data Rate), are usually configured with a small bandwidth, which can lead to a bandwidth bottleneck.

[0045] In some possible implementations, the capacity of the NPU's internal cache is limited. In order to achieve data operations, these implementations will split the deep learning model. For example Figure 2A and Figure 2B As shown, the input feature is 1x64x64x8 (based on NHWC arrangement), a set of weights is 16x8x3x3 (based on CoCiKhKw arrangement), the parameter pad is (1, 1, 0, 0), that is, (left, top, right, bottom), the step size in the width and height directions is 2, and after convolution operation (in this example, the convolution INT8 type is used for operation and bias is ignored), the output feature is 1x32x32x16 (based on NHWC arrangement).

[0046] Assuming that the internal cache of the NPU is limited, for example, 32KB, the size of all input features and weights read by the input circuit will exceed the capacity of the cache. In this case, the cache cannot store the entire input feature. In this case, a split is required. Figure 2C The following is an example of segmentation in the height direction. Figure 2D and Figure 2E The two convolutions shown in Figure 1. The input feature of convolution 1 is 1x32x64x8, the weights and other parameters remain unchanged, and the output feature is 1x16x32x16. The input feature of convolution 2 is 1x33x64x8, the step size becomes all 0, the weights and other parameters remain unchanged, and the output feature is 1x16x32x16. Since the size of the convolution kernel is larger than the step size, there is an overlap between the two convolutions.

[0047] The addressing of cache in NPU is physically continuous, such as Figure 2F and 2G shown. Figure 2F It shows the cache status of convolution 1. Figure 2GIt shows the cache status of Convolution 2. The starting and ending addresses of the weight storage area and feature storage area correspond to the continuous area of ​​the cache. After reading the input features of Convolution 1, the remaining capacity of the cache is insufficient to store the input features of Convolution 2. Therefore, after the Convolution 1 operation is completed, the input features of Convolution 2 need to be read from the external memory, and the original data in the cache will be overwritten when the input features of Convolution 2 are stored in the cache. In the height direction of the input features of Convolution 2, it is necessary to read the last line of the input features of Convolution 1, Line_last, as shown in Figure 2F As shown in the figure, since the input features are stored in the external memory, the line Line_last is read from the external memory during the convolution 1 process, and the line Line_last is repeatedly read from the external memory during the convolution 2 process. This repeated reading will cause a waste of bandwidth.

[0048] It should be understood that, for ease of understanding, the above example is only introduced as an example of segmentation once. In actual applications, since the cache of the NPU is small and the input features are generally large, a large number of segmentations and repeated readings will be performed, which will result in considerable bandwidth waste, thereby increasing the bandwidth burden.

[0049] At least for the above-mentioned problems, the embodiments of the present disclosure provide a data processing method. The data processing method configures a second area in the cache. The second area stores the first reused data that was written after the last processing and used for this processing. Therefore, there is no need to read the first reused data from the external memory during this processing, which is conducive to reducing bandwidth requirements and avoiding bandwidth bottlenecks. In some embodiments, if the model has at least two layers and a feature is input, the result is output after the first layer is processed. This result is a new feature map, and this feature map will be input to the next layer as reused data. In some embodiments, the deep learning sub-model targets one or several layers in the model, and the outputs of these one or more layers are input to the processing of the next layer as reused data. In some embodiments, a model may be a graph, which can be divided into multiple sub-graphs.

[0050] Figure 3A is a flowchart showing a data processing method according to an embodiment of the present disclosure. The data processing method can be applied to Figure 1 Controller 13 in. Figure 3B Schematic diagram showing the cache status of the NPU according to an embodiment of the present disclosure. Figure 3A As shown, a processing process P2 of the data processing method provided by the embodiment of the present disclosure includes the following steps S31 to S33.

[0051] In step S31, first data is read and written into a first area of ​​a first target buffer. Figure 3BAs shown, the first target buffer area includes a first area and a second area. The first area is used to store the first data, and the second area is used to store the first multiplexed data written after the last processing. In some embodiments, the first data is read from the DDR memory and written into the buffer area in the NPU.

[0052] In step S32, first data to be processed are read from the first area and the second area. The first data to be processed includes the first data stored in the first area and the first multiplexed data stored in the second area.

[0053] In step S33, the first to-be-processed data is processed P2 using the first deep learning sub-model. The first deep sub-model is a sub-model obtained by segmenting the deep learning model.

[0054] In the disclosed embodiment, the current processing P2 will use the first multiplexed data written as the processing result after the previous processing P1. In the previous processing P1, the input data will be read from the external memory, and the first multiplexed data will be output after processing. The first multiplexed data is stored in the second area, and the input data is stored in the cache area outside the second area. The current processing P2 will read the first data from the external memory and store it in the first area. This process will not overwrite the data stored in the second area. In the current processing P2, the first multiplexed data and the first data are spliced ​​and read in the cache area.

[0055] Figure 3C 1 is a flow chart showing how to read first data and write the first data into the first area of ​​the first target cache area according to an embodiment of the present disclosure. Figure 3C As shown, in the embodiment of the present application, reading the first data and writing the first data into the first area of ​​the first target cache area includes the following steps S311 and S312.

[0056] In step S311, for at least one layer in the neural network model, input features corresponding to the at least one layer are read as first data.

[0057] In step S312, the first data is written into a first area of ​​a cache in the neural network processor.

[0058] In some possible implementations, processing the first data to be processed using the first deep learning sub-model includes: processing the first data to be processed using the first deep learning sub-model corresponding to the at least one layer to generate second multiplexed data.

[0059] Figure 3D FIG. 4 is a flowchart showing the processing of the second data to be processed according to an embodiment of the present disclosure. Figure 3DAccording to one embodiment of the present disclosure, the data processing method may further include the following steps S34 to S36.

[0060] In step S34, for a next layer of at least one layer in the neural network model, second data corresponding to the next layer is read from the memory and the second data is written to the first area of ​​the first target cache area.

[0061] In step S35, second data to be processed is read from the first area and the second area. The second data to be processed includes the second data and the second multiplexed data.

[0062] In step S36, the second data to be processed is processed using a second deep learning sub-model corresponding to the next layer.

[0063] Figure 3E FIG. 4 is a flowchart showing how to read first data to be processed from a first area and a second area according to an embodiment of the present disclosure. Figure 3E As shown, reading the first to-be-processed data from the first area and the second area in the embodiment of the present disclosure includes the following steps S321 to S323.

[0064] In step S321, data is read from the start address of the second area until the end address of the first target buffer area to read the first multiplexed data.

[0065] In step S322, data is continuously read from the start address of the first target buffer area until the end address of the first area, so as to read the first data.

[0066] In step S323, the first multiplexed data and the first data are spliced ​​to read the first data to be processed.

[0067] Figure 4A FIG. 4 is a flowchart showing another processing procedure P3 according to an embodiment of the present disclosure. Figure 4B Schematic diagram showing the cache status of the P3 process. Figure 4B As shown, the third area in the embodiment of the present disclosure includes two dispersed cache areas. Figure 4A As shown, the processing process P3 in the embodiment of the present disclosure includes the following steps S41 to S43.

[0068] In step S41, the third data is read from the memory and written into the third area of ​​the second target buffer area. The second target buffer area completely overlaps or partially overlaps with the first target buffer area. The second target buffer area includes a third area and a fourth area. The third area is used to store the third data, and the fourth area is used to store the third multiplexed data written after the last processing.

[0069] In step S42, third data to be processed are read from the third area and the fourth area. The third data to be processed includes the third data and the third multiplexed data.

[0070] In step S43, the third data to be processed is processed using the third deep learning sub-model.

[0071] According to one embodiment of the present disclosure, the first target cache area is a local circular cache. Reading the first data to be processed from the first area and the second area includes: starting to read data from the starting address of the second area, and when the ending address of the first target cache area is read, continuing to read data from the starting address of the first target cache area until the reading of the first data to be processed is completed.

[0072] In some possible implementations, any position segment in the cache can be configured as the first target cache area, provided that storage requirements are met.

[0073] In some possible implementations, the control of the local loop cache is not limited to one region. In these implementations, a combination of multiple regions can be used to reduce the fragmentation of cache management, so as to make full use of the limited cache to reduce the burden of the external memory.

[0074] According to an embodiment of the present disclosure, the first deep learning sub-model and the third deep learning sub-model are obtained by segmenting the deep learning model according to the capacity of the cache. In some possible implementations, a similar Figure 2C The deep learning model is segmented in this way, but the present disclosure is not limited to this.

[0075] According to one embodiment of the present disclosure, the size and position of the second area are determined by the segmentation method of the deep learning model. Specifically, when the segmentation method of the deep learning model is determined, the overlapping data of the two previous and next processing processes are determined, and then it can be determined which positions of data can be shared by the two previous and next processing processes, and the cache space required for these data.

[0076] According to an embodiment of the present disclosure, the start address and the end address of the first target cache area are determined by the size of the first data, the free capacity of the cache, and the second area.

[0077] According to one embodiment of the present disclosure, the configuration of the start address and the end address of the first target cache area is obtained by segmenting the deep learning model performed in the offline planning stage.

[0078] Figure 5 FIG. 1 is a flow chart showing the planning of cache in an embodiment of the present disclosure. Figure 5 As shown, the process includes the following steps S51 to S58.

[0079] In step S51, the deep learning model is divided into sub-models of appropriate sizes according to the cache size.

[0080] In step S52, one of the sub-models is selected as the target sub-model.

[0081] In step S53, one layer of the target sub-model is selected as the target layer.

[0082] In step S54, the cache start address and end address corresponding to the target layer are configured according to the size of the target layer, the remaining cache capacity and the overlapping data. The space between the start address and the end address is combined into a local circular cache.

[0083] In step S55, the overlap between the local loop buffer determined in step S54 and other sub-models is calculated and the overlap is locked. In the next round of configuration, the locked buffer space is not used.

[0084] In step S56, if the cache space configuration of the target sub-model is completed, step S57 is executed, otherwise the process jumps to step S53 to continue execution.

[0085] In step S57, if the cache space configuration of all sub-models is completed, step S58 is executed, otherwise the process jumps to step S52 to continue execution.

[0086] In step S58, an execution command of the deep learning model is generated, and the NPU executes the command to infer the entire deep learning model.

[0087] In some possible implementations, the deep learning model may be a multi-layer model. Figure 6 As shown in the figure, the deep learning model includes a convolutional layer (conv), an activation function layer (relu) and a pooling layer (pool). When the cache capacity is limited, the deep learning model is segmented, and each segmentation result is a sub-model. The feature control of the middle layer is looped in the cache, and the overlapping areas can be easily retained and looped after the next sub-model arrives, thereby saving bandwidth.

[0088] According to the above description, the data processing method provided by the embodiment of the present disclosure can reduce the increase in bandwidth caused by feature map segmentation, reduce the operating power consumption of the NPU, reduce the delay of the NPU reading overlapping data, and improve efficiency. In addition, the data processing method makes the inner loop of the NPU cache more convenient, which is conducive to reducing the fragmentation of the internal cache, thereby making full use of the limited cache space. In order to implement the data processing method, the design changes of the NPU are relatively small, and the area cost is also relatively small.

[0089] The protection scope of the data processing method described in the embodiments of the present disclosure is not limited to the execution order of the steps listed in the embodiments of the present disclosure. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present disclosure are included in the protection scope of the present disclosure.

[0090] The embodiments of the present disclosure also provide a data processing device, which can implement the data processing method described in the present disclosure. However, the implementation device of the data processing method described in the present disclosure includes but is not limited to the structure of the data processing device listed in the embodiments of the present disclosure. All structural deformations and replacements of the prior art made according to the principles of the present disclosure are included in the protection scope of the present disclosure.

[0091] Figure 7 Schematic diagram showing the structure of a data processing device 7 provided according to an embodiment of the present disclosure. Figure 7 As shown, the data processing device 7 includes a data reading and writing module 71, a data to be processed reading module 72 and a data processing module 73. The data reading and writing module 71 is configured to read the first data and write the first data into the first area of ​​the first target buffer area. The first target buffer area includes a first area and a second area, and the second area stores the first multiplexed data written after the last processing. The data to be processed reading module 72 is configured to read the first data to be processed from the first area and the second area. The data processing module 73 is configured to process the first data to be processed using the first deep learning sub-model.

[0092] It should be noted that the data reading and writing module 71, the to-be-processed data reading module 72 and the data processing module 73 included in the data processing device 7 can correspond one-to-one to each relevant step in the data processing method described above, and no further details will be given here.

[0093] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices or methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0094] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure. For example, the functional modules / units in the various embodiments of the present disclosure may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0095] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0096] The embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed to implement the data processing method according to any embodiment of the present disclosure.

[0097] Figure 8 1 is a schematic diagram showing the structure of an electronic device 8 provided according to an embodiment of the present disclosure. Figure 8 As shown, the electronic device 8 in the embodiment of the present disclosure includes a memory 81 and a processor 82.

[0098] The memory 81 is used to store computer programs. In some possible implementations, the memory 81 may include a computer system readable medium in the form of a volatile memory, such as a RAM and / or a cache memory. The electronic device 8 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 81 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0099] The processor 82 is connected to the memory 81 and is used to execute the computer program stored in the memory 81 so that the electronic device 8 executes the data processing method provided by any embodiment of the present disclosure.

[0100] In the disclosed embodiment, the processor 82 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0101] In some possible implementations, the electronic device 8 provided in the embodiments of the present disclosure may further include a display 83. The display 83 is communicatively connected to the memory 81 and the processor 82, and is used to display a graphical user interface (GUI) related to the data processing method provided in any embodiment of the present disclosure.

[0102] In the disclosed embodiment, the display 83 may include a display screen (display panel). In some implementations, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. In addition, the display 83 may also be a touch panel (touch screen, touch screen), and the touch panel may include a display screen and a touch-sensitive surface. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 82 to determine the type of touch event, and then the processor 82 provides a corresponding visual output on the display device according to the type of touch event.

[0103] The above embodiments are merely illustrative of the principles and effects of the present disclosure, and are not intended to limit the present disclosure. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present disclosure shall still be covered by the claims of the present disclosure.

Claims

1. A data processing method, characterized in that: include: Reading first data and writing the first data into a first area of ​​a first target cache area, the first target cache area also including a second area, the second area storing first multiplexed data written after the last processing; Reading first data to be processed from the first area and the second area; as well as The first data to be processed is processed using a first deep learning sub-model.

2. The data processing method according to claim 1, characterized in that: Reading first data and writing the first data into a first area of ​​a first target cache area includes: For at least one layer in the neural network model, reading an input feature corresponding to the at least one layer as the first data; and The first data is written into the first area of ​​a cache in a neural network processor.

3. The data processing method according to claim 2, characterized in that: Processing the first to-be-processed data using the first deep learning sub-model includes: The first data to be processed is processed using the first deep learning sub-model corresponding to the at least one layer to generate second multiplexed data.

4. The data processing method according to claim 3, characterized in that: Also includes: For a subsequent layer of the at least one layer in the neural network model, read second data corresponding to the subsequent layer from the memory and write the second data into the first area of ​​the first target buffer area; Reading second data to be processed from the first area and the second area; as well as The second data to be processed is processed using a second deep learning sub-model corresponding to the next layer.

5. The data processing method according to claim 1, characterized in that: Reading first to-be-processed data from the first area and the second area includes: Reading data from a starting address of the second area until an ending address of the first target buffer area to read the first multiplexed data; Continue reading data from the start address of the first target buffer area until the end address of the first area to read the first data; and The first multiplexed data and the first data are spliced ​​to read the first data to be processed.

6. The data processing method according to claim 1, characterized in that: Also includes: Reading third data from the memory and writing the third data into a third area of ​​a second target cache area, the second target cache area completely overlaps or partially overlaps with the first target cache area, the second target cache area also includes a fourth area, the fourth area stores the third multiplexed data written after the last processing; Reading third data to be processed from the third area and the fourth area; as well as The third data to be processed is processed using a third deep learning sub-model.

7. The data processing method according to claim 6, characterized in that: The first deep learning sub-model and the third deep learning sub-model are obtained by segmenting the deep learning model according to the cache capacity.

8. The data processing method according to claim 7, characterized in that: The size and position of the second region are determined by the segmentation method of the deep learning model.

9. The data processing method according to claim 1, characterized in that: Reading first to-be-processed data from the first area and the second area includes: Data is read starting from a starting address of the second area, and when the ending address of the first target buffer area is read, data is continued to be read from a starting address of the first target buffer area until the reading of the first data to be processed is completed.

10. The data processing method according to claim 1, characterized in that: The start address and the end address of the first target cache area are determined by the size of the first data, the free capacity of the cache and the second area.

11. The data processing method according to claim 10, characterized in that: The configuration of the start address and the end address of the first target cache area is obtained by segmenting the deep learning model in the offline planning stage.

12. A data processing device, characterized in that: include: a data reading and writing module, configured to read first data and write the first data into a first area of ​​a first target buffer area, the first target buffer area including the first area and a second area, the second area storing first multiplexed data written after a previous processing; a to-be-processed data reading module, configured to read first to-be-processed data from the first area and the second area; as well as The data processing module is configured to process the first data to be processed using a first deep learning sub-model.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed to implement the data processing method according to any one of claims 1 to 11.

14. An electronic device, characterized in that: include: a memory configured to store a processor executable program; as well as A processor is configured to call the program to execute the data processing method according to any one of claims 1 to 11.

Images