Data processing apparatus, method, electronic device, and storage medium

Abstract

The present disclosure provides a data processing device, relates to the field of artificial intelligence technology, in particular to the field of chip technology and the field of multi-thread parallel technology. The device comprises: a first target storage unit; and a processor configured to: in response to determining that the data quantity of target data is less than or equal to the capacity of the first target storage unit, determining an initial thread number according to the data quantity of the target data and the capacity of the first target storage unit, wherein the target data comprises to-be-processed input data, to-be-processed weight data and output data; and in response to determining that the initial thread number is greater than or equal to a preset thread number, determining a first executable task number according to the initial thread number. The present disclosure also provides a data processing method, an electronic device and a storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of chip technology and multithreaded parallel technology. More specifically, this disclosure provides a data processing apparatus, method, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, model inference or model training tasks can be performed in parallel. Summary of the Invention

[0003] This disclosure provides a data processing apparatus, method, device, and storage medium.

[0004] According to one aspect of this disclosure, a data processing apparatus is provided, the apparatus comprising: a first target storage unit; and a processor configured to: in response to determining that the amount of target data is less than or equal to the capacity of the first target storage unit, determine an initial number of threads based on the amount of target data and the capacity of the first target storage unit, wherein the target data includes input data to be processed, weighted data to be processed, and output data; and in response to determining that the initial number of threads is greater than or equal to a preset number of threads, determine a first number of executable tasks based on the initial number of threads.

[0005] According to another aspect of this disclosure, a data processing method is provided, the method comprising: in response to determining that the amount of target data is less than or equal to the capacity of a first target storage unit, determining an initial number of threads based on the amount of target data and the capacity of the first target storage unit, wherein the target data includes input data to be processed, weight data to be processed, and output data; and in response to determining that the initial number of threads is greater than or equal to a preset number of threads, determining a first number of executable tasks based on the initial number of threads.

[0006] According to another aspect of this disclosure, an electronic device is provided, including the electronic device provided in this disclosure.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to this disclosure.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods provided according to this disclosure.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to this disclosure.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a schematic block diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0013] Figure 2 This is a schematic diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0014] Figure 3 This is a flowchart of a data processing method according to an embodiment of the present disclosure;

[0015] Figure 4 This is a schematic block diagram of an electronic device according to an embodiment of the present disclosure; and

[0016] Figure 5 This is a block diagram of an electronic device to which a data processing method can be applied, according to an embodiment of the present disclosure. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] In the process of inference using deep learning models, a task mapping strategy based on a Direction Asyclic Graph (DAG) can divide the model inference task into multiple subtasks. These subtasks can be placed in different task queues to be executed in parallel. After all subtasks have been completed, the execution result of the model inference task can be obtained.

[0019] Model inference tasks can be performed using heterogeneous hardware platforms, including Central Processing Units (CPUs) and Graphics Processing Units (GPUs). Based on the execution relationships of different tasks, cyclic undirected graphs can be identified, allowing the parallel processing capabilities of GPUs to be combined with these graphs to improve the efficiency of heterogeneous device utilization. For example, model inference tasks involve numerous matrix operations. GPUs can significantly reduce matrix computation time, thereby improving the execution efficiency of model inference tasks.

[0020] Resources on heterogeneous hardware platforms can be scheduled based on dependencies between different data sets. However, for heterogeneous hardware platforms, it is difficult to break down model inference tasks into finer-grained parts. For example, it is difficult to effectively process large batch sizes of data.

[0021] In a heterogeneous hardware platform, the central processing unit (CPU) can act as the host, and the graphics processing unit (GPU) can act as the device. Data can be transferred from the host to the device. For example, data can be transferred to multiple GPU cores on the device to accelerate matrix computations in parallel. The GPU can include different levels of high-speed dynamic random access memory (DRAM). For example, a GPU can include Level 0 DRAM (L0), Level 1 DRAM (L1), Level 2 DRAM (L2), Level 3 DRAM (L3), and Level 4 DRAM (L4). Level 4 DRAM has the largest capacity. All processor cores can read and write data from Level 4 DRAM. Levels 0 to 2 DRAM have smaller capacities, making it difficult to store weight data or input data for deep learning models.

[0022] The bandwidth of the Level 3 high-speed dynamic random access memory (MRAM) can, for example, be twice that of the Level 4 high-speed dynamic random access memory (MRAM). To fully utilize the capacity and bandwidth of the Level 3 MRAM, this disclosure provides a data processing apparatus, which will be described below.

[0023] Figure 1 This is a schematic block diagram of a data processing apparatus according to an embodiment of the present disclosure.

[0024] like Figure 1 As shown, the data processing device 100 may include a first target storage unit 110 and a processor 120.

[0025] The first target storage unit 110 can be a level 3 high-speed dynamic random access memory.

[0026] The processor 120 can be configured to: determine an initial number of threads based on the amount of target data and the capacity of the first target storage unit in response to determining that the amount of target data is less than or equal to the capacity of the first target storage unit; and determine a first number of executable tasks based on the initial number of threads in response to determining that the initial number of threads is greater than or equal to a preset number of threads.

[0027] In embodiments of this disclosure, the processor may include multiple processor cores. The processor may be at least one of a graphics processor, a neural network processing unit (NPU), or a similar processor.

[0028] In this embodiment of the disclosure, the target data includes input data to be processed, weight data to be processed, and output data. For example, the input data to be processed may be the input data of a deep learning model. The weight data to be processed may include the weight data of each of the multiple operators of the deep learning model. The output data may include the output data of the deep learning model.

[0029] In this embodiment of the disclosure, it can be determined whether the amount of target data is less than or equal to the capacity of the first target storage unit. For example, taking a target data amount of 16 megabytes (Mbytes) as an example, the capacity of the first target storage unit can be 64 megabytes. It can be determined that the amount of target data is less than the capacity of the first target storage unit. Next, based on the amount of target data and the capacity of the first target storage unit, the initial number of threads can be determined to be 4. That is, the first storage unit can store 4 pieces of target data.

[0030] In this embodiment of the disclosure, the preset thread count can be 1. If the initial thread count is 4, it can be determined that the initial thread count is greater than the preset thread count. The initial thread count can be used as the first executable task count.

[0031] Through the embodiments of this disclosure, during model training or inference, when the amount of data is small, storing multiple target data in the first target storage unit can fully utilize the bandwidth and capacity of the first target storage unit and improve data processing efficiency.

[0032] Once the number of executable tasks is determined, those tasks can be executed in parallel, as will be explained further below.

[0033] In some embodiments, the processor 120 may further be configured to write a first number of executable tasks of data to be processed into a first target storage unit. In embodiments of this disclosure, the data to be processed includes input data to be processed and weight data to be processed. For example, if the first number of executable tasks is 4, 4 sets of input data to be processed and weight data to be processed can be written into the first target storage unit.

[0034] In some embodiments, the processor 120 may also be configured to execute a first number of executable tasks in parallel to obtain a first number of output data. In embodiments of this disclosure, a task may include processing input data using weighted data to be processed. For example, if the first number of executable tasks is 4, 4 tasks may be executed in parallel to obtain 4 output data.

[0035] In some embodiments, the processor 120 may also be configured to write a first executable number of output data items to a first target storage unit. For example, four output data items may be written to the first target storage unit. Through the embodiments of this disclosure, the input data to be processed stored in the first target storage unit can be processed in parallel, which can fully utilize the parallel processing capabilities of the artificial intelligence chip and improve data processing efficiency.

[0036] It is understood that the above description uses the example of the target data volume being less than or equal to the capacity of the first target storage unit, but this disclosure is not limited to this, and will be further explained below.

[0037] Figure 2 This is a schematic diagram of a data processing apparatus according to an embodiment of the present disclosure.

[0038] like Figure 2 As shown, the processor can be configured to execute at least one instruction to implement operation S201. In operation S201, it is determined whether the sum of the data amounts of the input data to be processed and the output data to be processed is less than or equal to the capacity of the first target storage unit. For example, taking the first target data as an example, the data amount of the first target data is 16 megabytes. The capacity of the first target storage unit can be 64 megabytes. It can be determined that the sum of the data amounts of the input data to be processed and the weight data to be processed of the first target data is less than the capacity of the first target storage unit, which will be further explained below in conjunction with operation S202.

[0039] In some embodiments, the processor may further be configured to execute at least one instruction to implement operation S202 in response to determining that the sum of the data amounts of the input data to be processed and the output data to be processed is less than or equal to the capacity of the first target storage unit. In operation S202, it is determined whether the data amount of the target data is less than or equal to the capacity of the first target storage unit. For example, it may be determined that the data amount of the first target data is less than the capacity of the first target storage unit.

[0040] In this embodiment of the disclosure, the processor may further be configured to execute at least one instruction to implement operation S210 in response to determining that the amount of target data is less than or equal to the capacity of the first target storage unit. In operation S210, the initial number of threads is determined based on the amount of target data and the capacity of the first target storage unit. For example, the initial number of threads may be determined to be 4 based on the amount of first target data and the capacity of the first target storage unit. The first storage unit may store 4 pieces of first target data.

[0041] Next, in this embodiment of the disclosure, the processor may further be configured to execute at least one instruction to implement operation S221. In operation S221, it is determined whether the initial number of threads is greater than a preset number of threads. Taking a preset number of threads of 1 as an example, if the initial number of threads is 4, it can be determined that the initial number of threads is greater than the preset number of threads.

[0042] In this embodiment of the disclosure, the processor may further be configured to: in response to determining that the initial number of threads is greater than a preset number of threads, execute at least one instruction to implement operation S222. In operation S222, a first number of executable tasks is determined based on the initial number of threads. For example, the initial number of threads can be used as the first number of executable tasks. The first number of executable tasks can be 4.

[0043] As we have explained above, the methods for determining the number of executable tasks will now be explained below.

[0044] In this embodiment of the disclosure, the processor can also be configured to write a first number of executable tasks of data to be processed into a first target storage unit. For example, the data to be processed may include input data to be processed and weight data to be processed. If the number of first executable tasks is 4, the input data to be processed and the weight data to be processed for each of the 4 first target data can be written into the first target storage unit respectively.

[0045] In this embodiment of the disclosure, the processor can also be configured to execute a first number of executable tasks in parallel to obtain a first number of output data. For example, a task may include processing input data to be processed using weighted data to be processed. When the first number of executable tasks is 4, 4 tasks can be executed in parallel to obtain 4 output data.

[0046] In this embodiment of the disclosure, the processor may also be configured to write a first executable number of output data items to a first target storage unit. For example, four output data items may be written to the first target storage unit.

[0047] It is understood that the above description used the first target data as an example to illustrate this disclosure, and the following description will use the second target data as an example to further illustrate this disclosure. The size of the second target data can be 64 megabytes.

[0048] In some embodiments, the data processing apparatus may further include a second target storage unit, the capacity of which may be greater than that of the first target storage unit. For example, the second target storage unit may be a global memory (GM) unit or the aforementioned level 4 high-speed dynamic random access memory.

[0049] like Figure 2 As shown, the processor can be configured to execute at least one instruction to implement operation S201. In operation S201, it is determined whether the sum of the data amounts of the input data to be processed and the output data to be processed is less than or equal to the capacity of the first target storage unit. For example, the data amount of the second target data can be 64 megabytes, and the capacity of the first target storage unit can be 64 megabytes. The second target data may include the input data to be processed, the weight data to be processed, and the output data. The sum of the data amounts of the input data to be processed and the output data of the second target data can be less than the capacity of the first target storage unit.

[0050] In some embodiments, the processor may further be configured to execute at least one instruction to implement operation S202 in response to determining that the sum of the data amounts of the input data to be processed and the output data to be processed is less than or equal to the capacity of the first target storage unit. In operation S202, it is determined whether the data amount of the target data is less than or equal to the capacity of the first target storage unit. For example, it may be determined that the data amount of the second target data is equal to the capacity of the first target storage unit.

[0051] In this embodiment of the disclosure, the processor may further be configured to execute at least one instruction to implement operation S210 in response to determining that the amount of target data is less than or equal to the capacity of the first target storage unit. In operation S210, an initial number of threads is determined based on the amount of target data and the capacity of the first target storage unit. For example, based on the amount of second target data and the capacity of the first target storage unit, the initial number of threads can be determined to be 1. The first storage unit can store one piece of second target data.

[0052] Next, in this embodiment of the disclosure, the processor may further be configured to execute at least one instruction to implement operation S221. In operation S221, it is determined whether the initial number of threads is greater than a preset number of threads. Taking a preset number of threads of 1 as an example, if the initial number of threads is 1, it can be determined that the initial number of threads is equal to the preset number of threads.

[0053] In this embodiment of the disclosure, the processor may further be configured to: in response to determining that the initial number of threads is less than or equal to a preset number of threads, execute at least one instruction to implement operation S231. In operation S231, the first number of tasks is determined based on the amount of resources required by the processor to process the target data. For example, the first input data to be processed and the first weight data to be processed of the first second target data are written to the first target storage unit. The second input data to be processed and the second weight data to be processed of the second second target data are written to the second target storage unit. The processor can process the first input data to be processed and the second input data to be processed respectively to determine the amount of resources required by the processor to process the target data based on trial runs. Next, multiple trial runs can be performed. In the i-th trial run, the number of input data to be processed in the second target storage unit can be i. In the (i+1)-th trial run, the number of input data to be processed in the second target storage unit can be i+1. In the first trial run, the number of input data to be processed in the second target storage unit can be 1. If the processor utilization rate is close to 100% in the first trial run, the first number of tasks can be determined to be I. I can be an integer greater than 1, and i can be an integer greater than or equal to 1 and less than I.

[0054] In this embodiment of the disclosure, the processor may also be configured to execute at least one instruction to implement operation S232. In operation S232, a second number of executable tasks is determined based on the first number of tasks and the initial number of threads. For example, if the first number of tasks is I and the initial number of threads is 1, the second number of executable tasks can be determined to be I+1. Through this embodiment of the disclosure, when the data volume is large, storing multiple target data in a first target storage unit and a second target storage unit respectively can fully utilize the bandwidth of the first target storage unit and the capacity of the second target storage unit, thus helping to further improve data processing efficiency.

[0055] As we have explained above, the methods for determining the number of executable tasks will now be explained below.

[0056] In this embodiment of the disclosure, the processor can also be configured to write an initial number of data items to be processed into a first target storage unit. The data items to be processed include input data to be processed and weight data to be processed. For example, the input data to be processed and the weight data to be processed for one second target data item can be written into the first target storage unit.

[0057] In this embodiment of the disclosure, the processor can also be configured to write the first number of data items to be processed into the second target storage unit. For example, the input data and weight data to be processed for each of the first target data items can be written into the second target storage unit.

[0058] In this embodiment of the disclosure, the processor may further be configured to execute a number of tasks in parallel to obtain a number of output data items. The tasks include processing the input data to be processed using the weighted data to be processed. For example, I+1 tasks may be executed in parallel to obtain I+1 output data items.

[0059] In this embodiment of the disclosure, the processor may further be configured to write an initial number of output data items to a first target storage unit. For example, one output data item may be written to the first target storage unit, and this output data may correspond to the input data to be processed in the first target storage unit.

[0060] In this embodiment, the processor can further be configured to write a first number of output data items into the second target storage unit. For example, one output data item can be written into the second target storage unit, and this one output data item can correspond to one input data item to be processed in the second target storage unit. Through this embodiment, the input data to be processed stored in the first target storage unit and the second target storage unit can be processed in parallel, which can further leverage the parallel processing capability of the artificial intelligence chip and improve data processing efficiency.

[0061] It is understood that the above description used the second target data as an example to illustrate this disclosure, and the following description will use the third target data as an example to further illustrate this disclosure. The data size of the third target data can be greater than 64 megabytes.

[0062] like Figure 2As shown, the processor can be configured to execute at least one instruction to implement operation S201. In operation S201, it is determined whether the sum of the data amounts of the input data to be processed and the output data to be processed is less than or equal to the capacity of the first target storage unit. For example, the data amount of the third target data can be greater than 64 megabytes, and the capacity of the first target storage unit can be 64 megabytes. The third target data includes the input data to be processed, the weight data to be processed, and the output data. The sum of the data amounts of the input data to be processed and the output data of the third target data can be less than the capacity of the first target storage unit.

[0063] In some embodiments, the processor may further be configured to execute at least one instruction to implement operation S202 in response to determining that the sum of the data amounts of the input data to be processed and the output data to be processed is less than or equal to the capacity of the first target storage unit. In operation S202, it is determined whether the data amount of the target data is less than or equal to the capacity of the first target storage unit. For example, it may be determined that the data amount of the third target data is greater than the capacity of the first target storage unit.

[0064] In this embodiment of the disclosure, the processor may further be configured to execute at least one instruction in response to determining that the amount of target data is greater than the capacity of the first target storage unit, to implement operation S240. In operation S240, the number of third executable tasks is determined based on the amount of resources required by the processor to process the target data. For example, taking the third target data as an example, the input data and weight data to be processed for three third target data items can be written into the second target storage unit. The processor processes the input data to be processed for the three third target data items respectively to determine the amount of resources required by the processor to process the third target data. If the processor utilization rate required to process one third target data item is 5%, the number of third executable tasks can be determined to be 20. It is understood that the amount of resources required to process the number of third executable tasks of third target data items may not exceed the total resources of the processor. For example, the processor utilization rate required to process the number of third executable tasks of third target data items may not exceed 100%. Through this embodiment of the disclosure, when the data volume is large, storing multiple target data items in the second target storage unit can fully utilize the capacity of the second target storage unit and improve data processing efficiency.

[0065] As we have explained above, the methods for determining the number of executable tasks will now be explained below.

[0066] In this embodiment of the disclosure, the processor may further be configured to write a third number of executable tasks of data to be processed into the second target storage unit. The data to be processed includes input data to be processed and weight data to be processed. For example, when the number of third executable tasks is 20, the input data and weight data to be processed for each of the 20 third target data can be written into the second target storage unit.

[0067] In this embodiment of the disclosure, the processor can also be configured to execute a third number of executable tasks in parallel to obtain a third number of output data. The tasks include processing the input data to be processed using the weighted data to be processed. For example, 20 tasks can be executed in parallel to obtain 20 output data.

[0068] In this embodiment, the processor can also be configured to write a third executable number of output data items into the second target storage unit. For example, 20 output data items can be written into the second target storage unit. Through this embodiment, the input data to be processed stored in the second target storage unit can be processed in parallel, fully utilizing the parallel processing capabilities of the artificial intelligence chip and improving data processing efficiency.

[0069] It is understood that the above description used the third target data as an example to illustrate this disclosure, and the following description will use the fourth target data as an example to further illustrate this disclosure. The sum of the data volume of the input data to be processed and the weight data to be processed in the fourth target data can be greater than 64 megabytes.

[0070] like Figure 2 As shown, the processor can be configured to execute at least one instruction to implement operation S201. In operation S201, it is determined whether the sum of the data amounts of the input data to be processed and the output data to be processed is less than or equal to the capacity of the first target memory unit. For example, the capacity of the first target memory unit can be 64 megabytes. The sum of the data amounts of the input data to be processed and the output data of the fourth target data can be greater than the capacity of the first target memory unit.

[0071] In some embodiments, the processor may further be configured to execute at least one instruction to implement operation S202 in response to determining that the sum of the data amounts of the input data to be processed and the output data to be processed is greater than the capacity of the first target storage unit. In operation S202, it is determined whether the data amount of the target data is less than or equal to the capacity of the first target storage unit. For example, it may be determined that the data amount of the third target data is greater than the capacity of the first target storage unit.

[0072] In this embodiment of the disclosure, the processor may further be configured to execute at least one instruction to implement operation S251 in response to determining that the amount of target data is greater than the capacity of the first target storage unit. In operation S251, the input data to be processed is split into multiple sub-input data to be processed. For example, the input data to be processed for the fourth target data may be input image data to be processed. The shape of the input image data to be processed may be [n, c, h, w]. n may be the batch size, indicating the number of images in the input image data to be processed. c may be the number of channels of the image, for example, 3. h may be the height of the image, and w may be the width of the image. The input image data to be processed may be split into multiple sub-input image data to be processed according to the batch size. If n is 64, the input image data to be processed may include 64 images. The input image data to be processed may be split into 16 sub-input image data to be processed. The batch size of each sub-input image data to be processed is 4.

[0073] In this embodiment, the processor can also be configured to execute at least one instruction to implement operation S252. In operation S252, the number of fourth executable tasks is determined based on the amount of resources required by the processor to process the input sub-data to be processed. For example, the weight data to be processed and three input image sub-data to be processed can be written into the second target storage unit. The processor processes the three input image sub-data to be processed respectively to determine the amount of resources required by the processor to process the input image sub-data to be processed. If the processor utilization rate required to process one input image sub-data to be processed is 6%, the number of fourth executable tasks can be determined to be 16. It is understood that the amount of resources required to process the number of fourth executable tasks of input image sub-data to be processed may not exceed the total resources of the processor. For example, the processor utilization rate required to process the number of fourth executable tasks of input image sub-data to be processed may not exceed 100%. Through this embodiment, when the amount of data is large, splitting the input data to be processed can fully utilize the capacity of the second target storage unit and improve data processing efficiency.

[0074] As we have explained above, the methods for determining the number of executable tasks will now be explained below.

[0075] In this embodiment of the disclosure, the processor may further be configured to write the weight data to be processed and the number of sub-input data to be processed (a fourth number of executable tasks) into the second target storage unit. For example, the weight data to be processed may be written into the second target storage unit, or 16 sub-input image data to be processed may be written into the second target storage unit.

[0076] In this embodiment of the disclosure, the processor can also be configured to execute a fourth number of executable tasks in parallel to obtain a fourth number of output sub-data. The tasks include processing the input sub-data using the weighted data to be processed. For example, 16 tasks can be executed in parallel to obtain 16 output sub-data.

[0077] In this embodiment of the disclosure, the processor may also be configured to write a fourth executable number of output sub-data items to the second target storage unit. For example, 16 output sub-data items may be written to the second target storage unit.

[0078] In this embodiment of the disclosure, the processor can also be configured to concatenate multiple output sub-data into output data. For example, 16 output sub-data can be concatenated into output data corresponding to the input data to be processed. Through this embodiment of the disclosure, when the data volume is large, the data can be split according to the batch size of the input image data to be processed, realizing dynamic data splitting and enabling efficient parallel data processing.

[0079] It is understandable that the processor core can also be configured to execute a number of second tasks based on the data in the first-level high-speed dynamic random access memory.

[0080] It is understood that the data processing apparatus of this disclosure has been described above, and the data processing method of this disclosure will be described below.

[0081] Figure 3 This is a flowchart of a data processing method according to an embodiment of the present disclosure.

[0082] like Figure 3 As shown, the method 300 may include operations S310 to S320.

[0083] In operation S310, in response to determining that the amount of target data is less than or equal to the capacity of the first target storage unit, the initial number of threads is determined based on the amount of target data and the capacity of the first target storage unit.

[0084] In this embodiment of the disclosure, the target data includes input data to be processed, weight data to be processed, and output data.

[0085] In operation S320, in response to determining that the initial number of threads is greater than or equal to the preset number of threads, the first number of executable tasks is determined based on the initial number of threads.

[0086] It is understandable that the processor 120 described above can be used to execute method 300.

[0087] In some embodiments, method 300 may further include: writing a first number of executable tasks of data to be processed into a first target storage unit. For example, the data to be processed includes input data to be processed and weight data to be processed. The first number of executable tasks are executed in parallel to obtain a first number of executable tasks of output data. For example, the tasks include: processing the input data to be processed using the weight data to be processed. The first number of executable tasks of output data are then written into the first target storage unit.

[0088] In some embodiments, the capacity of the first target storage unit is less than or equal to the capacity of the second target storage unit.

[0089] In some embodiments, method 300 may further include: in response to determining that the initial number of threads equals a preset number of threads, determining a first number of tasks based on the amount of resources required by the processor to process the target data; and determining a second number of executable tasks based on the first number of tasks and the initial number of threads.

[0090] In some embodiments, method 300 may further include: writing an initial number of threads of data to be processed into a first target storage unit. For example, the data to be processed includes input data to be processed and weight data to be processed. Writing a first number of tasks of data to be processed into a second target storage unit. Executing a second number of executable tasks in parallel to obtain a second number of executable tasks of output data. For example, the task includes: processing the input data to be processed using the weight data to be processed. Writing the initial number of threads of output data into the first target storage unit. Writing the first number of tasks of output data into the second target storage unit.

[0091] In some embodiments, method 300 may further include: in response to determining that the amount of target data is greater than the capacity of the first target storage unit, determining the number of third executable tasks based on the amount of resources required by the processor to process the target data.

[0092] In some embodiments, method 300 may further include: writing a third number of executable tasks of data to be processed into a second target storage unit. For example, the data to be processed includes input data to be processed and weight data to be processed. The third number of executable tasks are executed in parallel to obtain a third number of executable tasks of output data. For example, the tasks include: processing the input data to be processed using the weight data to be processed. The third number of executable tasks of output data are then written into the second target storage unit.

[0093] In some embodiments, method 300 may further include: in response to determining that the sum of the data amounts of the input data to be processed and the output data is greater than the capacity of the first target storage unit, splitting the input data to be processed into a plurality of sub-input data to be processed. Determining a fourth number of executable tasks based on the amount of resources required by the processor to process the sub-input data to be processed.

[0094] In some embodiments, method 300 may further include: writing the weight data to be processed and a fourth number of executable tasks of sub-input data to be processed into a second target storage unit; executing the fourth number of executable tasks in parallel to obtain a fourth number of output sub-data. For example, the tasks include: processing the sub-input data to be processed using the weight data to be processed; writing the fourth number of executable output sub-data into the second target storage unit; and concatenating the multiple output sub-data into output data.

[0095] Figure 4 This is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0096] like Figure 4 As shown, the device 40 may include the data processing apparatus 400 provided in this disclosure. For example, the data processing apparatus 400 may be the data processing apparatus 100 described above.

[0097] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0098] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0099] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0100] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded into random access memory (RAM) 503 from storage unit 508. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0101] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0102] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform data processing methods by any other suitable means (e.g., by means of firmware).

[0103] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0105] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) monitor or an LCD (liquid crystal display)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0107] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0108] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0109] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing apparatus, comprising: First target storage unit; The second target storage unit has a capacity greater than that of the first target storage unit; as well as The processor is configured as follows: In response to determining that the amount of target data is less than or equal to the capacity of the first target storage unit, an initial number of threads is determined based on the amount of target data and the capacity of the first target storage unit, wherein the target data includes input data to be processed, weight data to be processed, and output data; and in response to determining that the initial number of threads is greater than or equal to a preset number of threads, a first number of executable tasks for writing target data into the first target storage unit is determined based on the initial number of threads. In response to determining that the amount of the target data is greater than the capacity of the first target storage unit, a third number of executable tasks for writing the target data into the second target storage unit is determined based on the amount of resources required by the processor to process the target data.

2. The apparatus of claim 1, wherein, The processor is also configured to: Write the first number of executable tasks of data to be processed into the first target storage unit, wherein the data to be processed includes the input data to be processed and the weight data to be processed; The first number of executable tasks are executed in parallel to obtain the first number of output data, wherein the task includes: processing the input data to be processed using the weight data to be processed; and Write the output data of the first number of executable tasks into the first target storage unit.

3. The apparatus of claim 1, wherein, The processor is also configured to: In response to determining that the initial number of threads is equal to the preset number of threads, the first number of tasks is determined based on the amount of resources required by the processor to process the target data; as well as The number of second executable tasks is determined based on the number of the first tasks and the number of the initial threads.

4. The apparatus of claim 3, wherein, The processor is also configured to: Write the initial number of data to be processed into the first target storage unit, wherein the data to be processed includes the input data to be processed and the weight data to be processed; Write the first number of data to be processed into the second target storage unit; The number of tasks to be executed in parallel is equal to the number of output data to be obtained, wherein the task includes: processing the input data to be processed using the weight data to be processed; Write the initial number of output data into the first target storage unit; and Write the first number of output data items into the second target storage unit.

5. The apparatus of claim 1, wherein, The processor is also configured to: Write the number of pending data items of the third executable task into the second target storage unit, wherein the pending data items include the pending input data and the pending weight data; The number of third executable tasks are executed in parallel to obtain the number of output data, wherein each task includes: processing the input data to be processed using the weight data to be processed; and Write the output data of the third executable task number into the second target storage unit.

6. The apparatus of claim 1, wherein, The processor is also configured to: In response to determining that the sum of the data volume of the input data to be processed and the output data is greater than the capacity of the first target storage unit, the input data to be processed is split into multiple sub-input data to be processed; as well as The number of fourth executable tasks is determined based on the amount of resources required by the processor to process the sub-input data to be processed.

7. The apparatus of claim 6, wherein, The processor is configured as follows: Write the weight data to be processed and the number of the fourth executable tasks of the sub-input data to be processed into the second target storage unit; The number of tasks to be executed in parallel is equal to the number of output sub-data items to be obtained, wherein the task includes: processing the input sub-data items to be processed using the weight data to be processed; Write the number of output sub-data items of the fourth executable task into the second target storage unit; and The multiple output sub-data are concatenated into output data.

8. A data processing method, comprising: In response to determining that the amount of target data is less than or equal to the capacity of the first target storage unit, an initial number of threads is determined based on the amount of target data and the capacity of the first target storage unit, wherein the target data includes input data to be processed, weight data to be processed, and output data; and in response to determining that the initial number of threads is greater than or equal to a preset number of threads, a first number of executable tasks for writing target data into the first target storage unit is determined based on the initial number of threads. In response to determining that the amount of target data is greater than the capacity of the first target storage unit, a third number of executable tasks for writing target data into the second target storage unit is determined based on the amount of resources required by the processor to process the target data; the capacity of the second target storage unit is greater than the capacity of the first target storage unit.

9. The method according to claim 8, further comprising: Write the first number of executable tasks of data to be processed into the first target storage unit, wherein the data to be processed includes the input data to be processed and the weight data to be processed; The first number of executable tasks are executed in parallel to obtain the first number of output data, wherein the task includes: processing the input data to be processed using the weight data to be processed; and Write the output data of the first number of executable tasks into the first target storage unit.

10. The method of claim 8, further comprising: In response to determining that the initial number of threads is equal to the preset number of threads, the first number of tasks is determined based on the amount of resources required by the processor to process the target data; as well as The number of second executable tasks is determined based on the number of the first tasks and the number of the initial threads.

11. The method of claim 10, further comprising: Write the initial number of data to be processed into the first target storage unit, wherein the data to be processed includes the input data to be processed and the weight data to be processed; Write the first number of data to be processed into the second target storage unit; The number of tasks to be executed in parallel is equal to the number of output data to be obtained, wherein the task includes: processing the input data to be processed using the weight data to be processed; Write the initial number of output data into the first target storage unit; and Write the first number of output data items into the second target storage unit.

12. The method according to claim 8, further comprising: Write the number of pending data items of the third executable task into the second target storage unit, wherein the pending data items include the pending input data and the pending weight data; The number of third executable tasks are executed in parallel to obtain the number of output data, wherein each task includes: processing the input data to be processed using the weight data to be processed; and Write the output data of the third executable task number into the second target storage unit.

13. The method of claim 8, further comprising: In response to determining that the sum of the data volume of the input data to be processed and the output data is greater than the capacity of the first target storage unit, the input data to be processed is split into multiple sub-input data to be processed; as well as The number of fourth executable tasks is determined based on the amount of resources required by the processor to process the sub-input data to be processed.

14. The method of claim 13, further comprising: Write the weight data to be processed and the number of the fourth executable tasks of the sub-input data to be processed into the second target storage unit; The number of tasks to be executed in parallel is equal to the number of output sub-data items to be obtained, wherein the task includes: processing the input sub-data items to be processed using the weight data to be processed; Write the number of output sub-data items of the fourth executable task into the second target storage unit; and The multiple output sub-data are concatenated into output data.

15. An electronic device comprising: The data processing apparatus according to any one of claims 1 to 7.

16. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 8 to 14.

17. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 8 to 14.

18. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 8 to 14.

Images