Task processing method and device and readable storage medium
By performing task processing in parallel in the autonomous driving system, obtaining the data to be inferred and processing the second data during the inference process, the problem of task submission serialization of TensorRT on DLA is solved, and the inference performance and resource utilization are improved.
Patent Information
- Application Number
- CN202311112211.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-30
- Publication Date
- 2025-02-28
AI Technical Summary
In the field of autonomous driving, when using the deep learning inference acceleration engine (TensorRT), the problem of task submission serialization occurs on DLA (a dedicated inference engine for autonomous driving domain controllers), resulting in low inference performance and insufficient utilization of computing power resources.
By obtaining the first data to be inferred, the second data to be processed is obtained during the execution of the first data inference task, and the first data processing task is performed based on the second data. This method can execute tasks in parallel, avoid serialization of task submissions, improve inference performance and make full use of computing resources.
By performing task processing in parallel, the performance of inference of the autonomous driving model is improved, computing resources are fully utilized, and the problems of low inference performance and insufficient resource utilization are solved.
Smart Images
Figure CN119536972A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a task processing method, device, and readable storage medium. Background Art
[0002] The specific use process of deep neural networks can be divided into two parts: model training and model reasoning. Autonomous driving requires a large number of deep neural network models for model reasoning. Usually, in the process of model reasoning, the deep learning reasoning acceleration engine (TensorRT) is used to accelerate model reasoning. However, since TensorRT is an acceleration engine for GPU (embedded integrated graphics card), when TensorRT is used on DLA (dedicated reasoning engine of autonomous driving domain controller) to accelerate model reasoning, there will be problems with serialization of task submission, resulting in low reasoning performance and insufficient utilization of computing resources. Summary of the invention
[0003] In order to overcome the problems existing in the related art, the present disclosure provides a task processing method, device and readable storage medium.
[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a task processing method, including:
[0005] Acquire first data to be inferred;
[0006] In the process of performing the first data reasoning task according to the first data, obtaining second data to be processed;
[0007] A first data processing task is performed according to the second data.
[0008] Optionally, executing the first data processing task according to the second data includes:
[0009] The second data is quantized to obtain second target data.
[0010] Optionally, the performing quantization processing on the second data to obtain second target data includes:
[0011] The data type of the second data is quantified into a first data type to obtain second target data, where the first data type is the data type used when performing the first data reasoning task.
[0012] Optionally, the method further comprises:
[0013] The inference result is dequantized to obtain first target data, where the inference result is an inference result obtained after the first data inference task is performed on the first data.
[0014] Optionally, performing dequantization processing on the inference result to obtain the first target data includes:
[0015] The data type of the inference result is dequantized into a second data type to obtain first target data, where the second data type is a data type used when performing a first data processing task.
[0016] Optionally, the method further comprises:
[0017] After executing the first data processing task according to the second data, the data processing result obtained by the first data processing task is used as input data for executing a specified data reasoning task, and the specified data reasoning task is the next data reasoning task of the first data reasoning task.
[0018] Optionally, performing a first data reasoning task according to the first data includes:
[0019] According to the first data, a first data reasoning task is performed by a first processor;
[0020] The performing the first data processing task according to the second data includes:
[0021] A first data processing task is performed by a second processor according to the second data, and the first processor and the second processor are different processors.
[0022] According to a second aspect of an embodiment of the present disclosure, there is provided a task processing device, the device comprising:
[0023] A first acquisition module is configured to acquire first data to be inferred;
[0024] A second acquisition module is configured to acquire second data to be processed in the process of performing the first data reasoning task according to the first data;
[0025] The execution module is configured to execute the first data processing task according to the second data.
[0026] Optionally, the execution module includes:
[0027] The quantization submodule is configured to perform quantization processing on the second data to obtain second target data.
[0028] Optionally, the quantization submodule is used to quantize the data type of the second data into a first data type to obtain second target data, where the first data type is a data type used when performing a first data inference task.
[0029] Optionally, the execution module further includes:
[0030] The dequantization submodule is configured to perform dequantization processing on the inference result to obtain the first target data, where the inference result is the inference result obtained after executing the first data inference task on the first data.
[0031] Optionally, the dequantization submodule is used to dequantize the data type of the shown inference result into a second data type to obtain first target data, where the second data type is a data type used when performing the first data processing task.
[0032] Optionally, the device further comprises:
[0033] The input module is configured to, after executing the first data processing task based on the second data, use the data processing result obtained by the first data processing task as input data for executing a specified data reasoning task, and the specified data reasoning task is the next data reasoning task of the first data reasoning task.
[0034] Optionally, the second acquisition module includes:
[0035] A first execution submodule is configured to execute a first data reasoning task through a first processor according to the first data;
[0036] The execution module comprises:
[0037] The second execution submodule is configured to execute the first data processing task through a second processor according to the second data, and the first processor and the second processor are different processors.
[0038] According to a third aspect of an embodiment of the present disclosure, there is provided a task processing device, including:
[0039] processor;
[0040] a memory for storing processor-executable instructions;
[0041] The processor is configured to implement the steps of the task processing method provided in the first aspect of the present disclosure when executing.
[0042] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the task processing method provided in the first aspect of the present disclosure are implemented.
[0043] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:
[0044] By acquiring the first data to be inferred; in the process of executing the first data inference task according to the first data, acquiring the second data to be processed; and executing the first data processing task according to the second data. In this way, the task of acquiring the second data to be processed can be executed in the process of executing the first data inference task, and the tasks can be executed in parallel, avoiding the problem of serialization of task submission, improving the inference performance, and making full use of computing resources.
[0045] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0047] Figure 1 The figure is a flowchart of a task processing method according to an exemplary embodiment.
[0048] Figure 2 is a flowchart of another task processing method shown according to the above step S13.
[0049] Figure 3 The figure is a computational flow chart showing a method of performing model reasoning by serial task submission according to an exemplary embodiment.
[0050] Figure 4 The present invention is a computational flow chart showing a method of performing model reasoning by using parallel task submission according to an exemplary embodiment.
[0051] Figure 5 The figure is a quantification flowchart according to an exemplary embodiment.
[0052] Figure 6 It is a flowchart of another task processing method shown according to the above steps S12 and S13.
[0053] Figure 7 The figure is a flowchart of another task processing method according to an exemplary embodiment.
[0054] Figure 8 The figure is a block diagram of a task processing device according to an exemplary embodiment.
[0055] Fig. 9 is based on Figure 7 The illustrated embodiment shows a block diagram of an execution module.
[0056] Fig.10 is based on Figure 7The illustrated embodiment shows a block diagram of a task processing device.
[0057] Fig.11 is based on Figure 7 The illustrated embodiment shows a block diagram of another task processing device.
[0058] Fig.12 It is a block diagram of a device for task processing according to an exemplary embodiment. DETAILED DESCRIPTION
[0059] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0060] Before introducing the specific implementation methods of the present disclosure in detail, the application scenarios of the present disclosure are first described. The present disclosure can be applied to application scenarios where deep neural network models are used for model reasoning. At present, with the continuous development of artificial intelligence technology, models have been widely used in the fields of intelligent recommendation, classification, autonomous driving cloud, decision-making systems, etc., with the advantages of high degree of automation and low labor costs. For example, data can be input into the models in the autonomous driving cloud and decision-making system, and model reasoning can be performed on the models to obtain reasoning results. The reasoning results may include planned routes, obstacle locations, collision risks, etc.
[0061] Usually, in the process of model reasoning, the deep learning reasoning acceleration engine (TensorRT) is used to accelerate model reasoning. However, since TensorRT is an acceleration engine for GPU (embedded integrated graphics card), when TensorRT is used on DLA (dedicated reasoning engine for autonomous driving domain controller) to accelerate model reasoning, there will be problems with task submission serialization, resulting in low reasoning performance and insufficient utilization of computing resources.
[0062] In order to overcome the technical problems existing in the above related technologies, the present disclosure provides a task processing method, device and readable storage medium, by obtaining the first data to be inferred; in the process of executing the first data inference task according to the first data, obtaining the second data to be processed; and executing the first data processing task according to the second data. In this way, the task of obtaining the second data to be processed can be executed in the process of executing the first data inference task, and the tasks can be executed in parallel, avoiding the problem of serialization of task submission, improving the inference performance, and making full use of computing resources.
[0063] The present disclosure is described below in conjunction with specific embodiments.
[0064] Figure 1 is a flowchart of a task processing method according to an exemplary embodiment. Figure 1 As shown, the method can be applied to electronic devices, which may include mobile phones, notebooks, desktop computers, vehicle-mounted terminals, smart home appliances, etc. The task processing method of the embodiment of the present disclosure can be executed by the task processing device of the embodiment of the present disclosure, and the task processing device of the embodiment of the present disclosure can be configured in any electronic device to execute the task processing method of the embodiment of the present disclosure. The method includes the following steps.
[0065] In step S11, first data to be inferred is obtained.
[0066] The data format of the first data may include a vector format, which is not limited here.
[0067] Optionally, the first data may include but is not limited to the user's physiological sign data, the user's behavior data on the electronic device, the user's behavior data on the APP (Application) side and the web page side, road network data, image data, vehicle driving data, etc. For example, the behavior data includes but is not limited to the number of user logins, login duration, login time, installation or uninstallation behavior data of objects, resource change data, etc. Among them, the objects include but are not limited to APP, skin, font, etc., the resources may be the transaction amount, or may be the points used for the transaction, the resource change data includes but is not limited to the cumulative transaction amount, the cumulative number of transactions, the cumulative consumption points, the cumulative number of points used, the image data includes the image data of the vehicle's surroundings (such as the image data of the road ahead), and the vehicle's driving data includes but is not limited to the vehicle's location, destination location, heading angle, front wheel steering angle, remaining power, etc.
[0068] In step S12, in the process of executing the first data reasoning task according to the first data, second data to be processed is obtained.
[0069] Among them, the model that executes the first data reasoning task can be a first model. Specifically, the first data reasoning task can include the overall process of inputting the first data into the first model and outputting the reasoning result through the first model; the first model can be a trained neural network model.
[0070] In this step, since executing the first data reasoning task can be an execution process that requires a certain execution time, the first data can be first input into the first model, and the reasoning result can be output through the first model. In the process of executing the first data reasoning task, the second data to be processed can be synchronously acquired.
[0071] In step S13, the first data processing task is executed according to the second data.
[0072] In this step, after acquiring the second data, the first data processing task may be performed on the second data to obtain processed second data, and the processed second data may be used as input data for the next execution of the data inference task.
[0073] By adopting the above technical solution, the first data to be inferred is obtained; in the process of executing the first data inference task according to the first data, the second data to be processed is obtained; and the first data processing task is executed according to the second data. In this way, the task of obtaining the second data to be processed can be executed in the process of executing the first data inference task, and the tasks can be executed in parallel, avoiding the problem of serialization of task submission, improving the inference performance, and making full use of computing resources.
[0074] In some embodiments, Figure 2 As shown, the above step S13 can adopt the following steps.
[0075] In step S131, the second data may be quantized to obtain second target data.
[0076] The method disclosed in the present application may include at least two inference engines for model inference, such as DLA, and a CPU for task scheduling.
[0077] Considering that DLA is a dedicated reasoning engine for autonomous driving domain controllers, TensorRT is usually used to accelerate model reasoning on DLA. However, when using DLA for model reasoning, Figure 3 As shown, the CPU's data acquisition tasks, submission tasks, and DLA's reasoning tasks are usually encapsulated as serial interfaces, resulting in serial execution of tasks during processing, causing DLA to wait, which results in low reasoning performance and insufficient utilization of computing resources.
[0078] Therefore, in some embodiments, Figure 4 As shown, the data acquisition task and submission task of the CPU and the reasoning task of the DLA can be encapsulated as a parallel interface, so that the DLA can execute the reasoning task in parallel while the CPU is executing the data acquisition task and submission task.
[0079] Optionally, in step S14, after executing the first data processing task according to the second data, the data processing result obtained by the first data processing task is used as input data for executing the specified data reasoning task.
[0080] The designated data reasoning task is the next data reasoning task of the first data reasoning task.
[0081] For example, while the CPU is executing the second data acquisition task and submitting the task, the DLA can execute the first reasoning task in parallel. Similarly, while the CPU is executing the third data acquisition task and submitting the task, the DLA can execute the second reasoning task in parallel. In this way, parallel tasks can be processed between multiple heterogeneous hardware, thereby improving the overall task throughput and the reasoning efficiency.
[0082] It should be noted that when the method disclosed in the present application is executed for the first time, the DLA is in an idle state during the CPU's data acquisition and task submission processes.
[0083] Also, because DLA is programmed using the underlying API, the DLA hardware only supports INT8 / FP16 type data, and DLA generally uses NCHW / NHWC / NCxHWX as the data memory layout type; however, the data preprocessing and post-processing of the autonomous driving algorithm model generally use single-precision floating point (FP32) type, or even higher-precision data types, and the data memory layout type of the autonomous driving algorithm model is also different, and the NCHW layout type is usually used.
[0084] Therefore, in some embodiments, Figure 5 As shown, the data type of the second data can be quantized into the first data type to obtain second target data.
[0085] Among them, the first data type is a data type used when performing the first data inference task, that is, the first data type is a data type of INT8 / FP16 type, NCHW / NHWC / NCxHWX layout type.
[0086] By way of example, the method disclosed in the present application may further include a GPU as an inference engine to execute the first data processing task on the second target data to be processed acquired when the CPU executes the data acquisition task. In this way, the second target data of single-precision floating point (FP32) type and NCHW data memory layout type may be quantized to obtain a data processing result, and the data processing result may be a data type of INT8 / FP16 type and NCHW / NHWC / NCxHWX layout type supported by DLA hardware.
[0087] By adopting the above technical solution, the DLA can be used as an inference engine to perform data inference tasks, and the GPU can be used as an inference engine to perform the first data processing task. In this way, the second data of single-precision floating point (FP32) type and NCHW data memory layout type can be quantized to obtain second target data. The data processing result can be data of INT8 / FP16 type and NCHW / NHWC / NCxHWX layout type supported by DLA hardware, and the GPU can perform the first data processing task while DLA is executing the last data inference task. This can improve the inference performance and make full use of computing resources.
[0088] In some embodiments, the above step S13 may adopt the following steps.
[0089] In step S132, the inference result is dequantized to obtain the first target data.
[0090] The inference result is the inference result obtained after executing the first data inference task.
[0091] Optionally, after executing the first data reasoning task according to the first data, the obtained reasoning result may be dequantized to obtain dequantized first target data.
[0092] For example, the data type of the inference result may be dequantized into the second data type.
[0093] The second data type is a data type used when performing the first data processing task, that is, the second data type can be a data type of a single-precision floating point (FP32) type or an NCHW data memory layout type.
[0094] In this way, the inference result data of INT8 / FP16 type and NCHW / NHWC / NCxHWX layout type can be dequantized into the target inference result data of single-precision floating point (FP32) type and NCHW data memory layout type generally used by the autonomous driving algorithm model, which can facilitate the next step of processing of the target inference result by the autonomous driving algorithm model and improve the inference efficiency.
[0095] In some embodiments, Figure 6 As shown, the above steps S12 and S13 may include the following steps.
[0096] In step S121, a first data reasoning task is performed by a first processor according to the first data.
[0097] In this embodiment, the data acquisition task and submission task of the CPU and the reasoning task of the DLA can be encapsulated as a parallel interface, so that the DLA can execute the reasoning task in parallel while the CPU is executing the data acquisition task and submission task.
[0098] For example, after obtaining the first data from a pre-allocated memory address through the CPU, the first data can be submitted to the DLA, on which a deep neural network model for model reasoning is deployed, so that the DLA can use the first data as input data based on the deep neural network model to output the reasoning result obtained by model reasoning.
[0099] For example, when the first data is the vehicle's driving data including but not limited to the vehicle's position, destination position, heading angle, front wheel steering angle, remaining power, etc., the vehicle's driving data can be used as input data for the deep neural network model deployed on the DLA to obtain an output vehicle control result, which can be used to control the vehicle's driving.
[0100] In step S133, the first data processing task is executed by the second processor according to the second data.
[0101] Optionally, the second data may be quantized to obtain a data processing result.
[0102] Among them, since DLA is programmed using the underlying API, the DLA hardware only supports INT8 / FP16 type data, and DLA generally uses NCHW / NHWC / NCxHWX as the data memory layout type; however, the data preprocessing and post-processing of the autonomous driving algorithm model generally use single-precision floating point (FP32) type, or even higher-precision data types, and the data memory layout type of the autonomous driving algorithm model is also different, and the NCHW layout type is usually used.
[0103] Therefore, the data type of the second data can be quantified into a first data type, and the first data type is the data type used when performing the first data reasoning task, that is, the first data type can be a data type of INT8 / FP16 type, NCHW / NHWC / NCxHWX layout type.
[0104] By way of example, the method disclosed in the present application may further include a GPU as an inference engine to execute the first data processing task on the second data to be processed acquired when the CPU executes the data acquisition task, so that the second data of single-precision floating point (FP32) type and NCHW data memory layout type can be quantized to obtain a data processing result, and the data processing result can be a data type of INT8 / FP16 type and NCHW / NHWC / NCxHWX layout type supported by DLA hardware.
[0105] The first processor and the second processor are different processors. The first processor may be a GPU, and the second processor may be a DLA.
[0106] By adopting the above technical solution, it is possible to implement the first data processing task by the second processor while the first data reasoning task is executed by the first processor. This can realize parallel task processing between multiple heterogeneous hardware, thereby improving the overall task throughput and improving the reasoning efficiency.
[0107] It should be noted that in the relevant technology, at the beginning and end of each DLA reasoning task, the input data needs to be registered and deregistered. However, under normal circumstances, in order to reduce the performance loss of data copying, the autonomous driving system usually uses pre-allocated memory to save input and output data. In other words, the tensor data for each reasoning has the same memory address. Therefore, repeated registration and deregistration is unnecessary, which will waste computing power and affect the reasoning rate.
[0108] When executing the method disclosed in the present application, before obtaining the first target data to be inferred, the pre-allocated memory can be pre-bound to achieve that each time the data acquisition task is executed by the CPU, the input data is obtained from the pre-allocated memory, and then when the execution of the method disclosed in the present application is completed, the registration is performed to release the pre-bound pre-allocated memory address.
[0109] By adopting the above technical solution, during the entire task processing process, registration is performed only once in the initialization phase, and deregistration is performed only once in the destruction phase. In the task processing phase, this step can be omitted, which can improve the reasoning efficiency and make full use of computing resources.
[0110] Figure 7 is a flowchart of another task processing method according to an exemplary embodiment. Figure 7 As shown, the method can be applied to an electronic device, and the method includes the following steps.
[0111] In step S701, first data to be inferred is obtained.
[0112] The data format of the first data may include a vector format or a tensor data format, which is not limited here.
[0113] Optionally, the first data may include but is not limited to the user's physiological sign data, the user's behavior data on the electronic device, the user's behavior data on the APP (Application) side and the web page side, road network data, image data, vehicle driving data, etc. For example, the behavior data includes but is not limited to the number of user logins, login duration, login time, installation or uninstallation behavior data of objects, resource change data, etc. Among them, the objects include but are not limited to APP, skin, font, etc., the resources may be the transaction amount, or may be the points used for the transaction, the resource change data includes but is not limited to the cumulative transaction amount, the cumulative number of transactions, the cumulative consumption points, the cumulative number of points used, the image data includes the image data of the vehicle's surroundings (such as the image data of the road ahead), and the vehicle's driving data includes but is not limited to the vehicle's location, destination location, heading angle, front wheel steering angle, remaining power, etc.
[0114] The method disclosed in the present application may include at least two inference engines for model inference, such as DLA, and a CPU for task scheduling.
[0115] For example, the first target data may be acquired by the CPU from a pre-allocated memory address.
[0116] In step S702, a first data reasoning task is performed by a first processor according to the first data.
[0117] In this embodiment, the data acquisition task and submission task of the CPU and the reasoning task of the DLA can be encapsulated as a parallel interface, so that the DLA can execute the reasoning task in parallel while the CPU is executing the data acquisition task and submission task.
[0118] For example, after obtaining the first data from a pre-allocated memory address through the CPU, the first data can be submitted to the DLA, on which a deep neural network model for model reasoning is deployed, so that the DLA can use the first data as input data based on the deep neural network model to output the reasoning result obtained by model reasoning.
[0119] For example, when the first data is the vehicle's driving data including but not limited to the vehicle's position, destination position, heading angle, front wheel steering angle, remaining power, etc., the vehicle's driving data can be used as input data for the deep neural network model deployed on the DLA to obtain an output vehicle control result, which can be used to control the vehicle's driving.
[0120] In step S703, in the process of executing the first data reasoning task according to the first data, second data to be processed is obtained.
[0121] In this step, since the data acquisition task and submission task of the CPU are encapsulated as a parallel interface with the reasoning task of the DLA, the DLA can execute the reasoning task in parallel while the CPU is executing the data acquisition task and submission task. Therefore, while the DLA is executing the reasoning task in parallel, the CPU can obtain the second data to be processed from the pre-allocated memory address.
[0122] In step S704, the first data processing task is executed by the second processor according to the second data.
[0123] The first processor and the second processor are different processors.
[0124] Optionally, the second target data may be quantized to obtain a data processing result.
[0125] Among them, since DLA is programmed using the underlying API, the DLA hardware only supports INT8 / FP16 type data, and DLA generally uses NCHW / NHWC / NCxHWX as the data memory layout type; however, the data preprocessing and post-processing of the autonomous driving algorithm model generally use single-precision floating point (FP32) type, or even higher-precision data types, and the data memory layout type of the autonomous driving algorithm model is also different, and the NCHW layout type is usually used.
[0126] Therefore, the data type of the second data can be quantized to the first data type to obtain the second target data. The first data type is the data type used when performing the first data reasoning task, that is, the first data type can be a data type of INT8 / FP16 type or NCHW / NHWC / NCxHWX layout type.
[0127] By way of example, the method disclosed in the present application may further include a GPU as an inference engine to execute the first data processing task on the second data to be processed acquired when the CPU executes the data acquisition task, so that the second data of single-precision floating point (FP32) type and NCHW data memory layout type can be quantized to obtain a data processing result, and the data processing result can be a data type of INT8 / FP16 type and NCHW / NHWC / NCxHWX layout type supported by DLA hardware.
[0128] In step S705, the inference result is dequantized to obtain the first target data.
[0129] The inference result is the inference result obtained after executing the first data inference task on the first data.
[0130] Optionally, the data type of the inference result can be dequantized into a second data type to obtain the first target data, and the second data type is the data type used when performing the first data processing task, that is, the second data type can be single-precision floating point (FP32) type, NCHW data memory layout type data.
[0131] In step S706, after executing the first data processing task according to the second data, the data processing result obtained by the first data processing task is used as input data for executing the specified data reasoning task.
[0132] The designated data reasoning task is the next data reasoning task of the first data reasoning task.
[0133] By adopting the above technical solution, it is possible to implement the first data processing task by the second processor while the first data reasoning task is executed by the first processor. This can realize parallel task processing between multiple heterogeneous hardware, thereby improving the overall task throughput and improving the reasoning efficiency.
[0134] Figure 8 FIG. 8 is a block diagram of a task processing device 800 according to an exemplary embodiment. Figure 8 The device includes a first acquisition module 801, a second acquisition module 802 and an execution module 803.
[0135] A first acquisition module 801 is configured to acquire first data to be inferred;
[0136] The second acquisition module 802 is configured to acquire second data to be processed in the process of performing the first data reasoning task according to the first data;
[0137] The execution module 803 is configured to execute the first data processing task according to the second data.
[0138] Fig. 9 is based on Figure 8 The block diagram of an execution module shown in the embodiment shown in FIG. Fig. 9 , the execution module 803 includes:
[0139] The quantization submodule 8031 is configured to perform quantization processing on the second data to obtain second target data.
[0140] Optionally, the quantization submodule 8031 is used to quantize the data type of the second data into a first data type to obtain second target data, where the first data type is a data type used when performing the first data inference task.
[0141] Optionally, the execution module 803 further includes:
[0142] The dequantization submodule 8032 is configured to perform dequantization processing on the inference result to obtain the first target data, where the inference result is the inference result obtained after executing the first data inference task on the first data.
[0143] Optionally, the dequantization submodule 8032 is used to dequantize the data type of the inference result into a second data type to obtain first target data, and the second data type is a data type used when performing the first data processing task.
[0144] Fig.10 is based on Figure 8 The block diagram of a task processing device 1000 shown in the embodiment shown in FIG. Fig.10 , the device 1000 further includes:
[0145] The input module 804 is configured to, after executing the first data processing task based on the second data, use the data processing result obtained by the first data processing task as input data for executing a specified data reasoning task, and the specified data reasoning task is the next data reasoning task of the first data reasoning task.
[0146] Fig.11 is based on Figure 8 The block diagram of another task processing device shown in the embodiment shown in FIG. Fig.11 The task processing device 1100 comprises:
[0147] A first execution submodule 8021 is configured to execute a first data reasoning task through a first processor according to the first data;
[0148] The execution module 803 includes:
[0149] The second execution submodule 8033 is configured to execute the first data processing task through the second processor according to the second data, and the first processor and the second processor are different processors.
[0150] By adopting the above technical solution, the first data to be inferred is obtained; in the process of executing the first data inference task according to the first data, the second data to be processed is obtained; and the first data processing task is executed according to the second data. In this way, the task of obtaining the second data to be processed can be executed in the process of executing the first data inference task, and the tasks can be executed in parallel, avoiding the problem of serialization of task submission, improving the inference performance, and making full use of computing resources.
[0151] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0152] The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the task processing method provided by the present disclosure are implemented.
[0153] Fig.12 1 is a block diagram of an apparatus 1200 for task processing according to an exemplary embodiment. For example, the apparatus 1200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0154] Reference Fig.12 , the device 1200 may include one or more of the following components: a processing component 1202 , a memory 1204 , a power component 1206 , a multimedia component 1208 , an audio component 1210 , an input / output interface 1212 , a sensor component 1214 , and a communication component 1216 .
[0155] The processing component 1202 generally controls the overall operation of the device 1200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1202 may include one or more processors 1220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 1202 may include one or more modules to facilitate the interaction between the processing component 1202 and other components. For example, the processing component 1202 may include a multimedia module to facilitate the interaction between the multimedia component 1208 and the processing component 1202.
[0156] The memory 1204 is configured to store various types of data to support the operation of the device 1200. Examples of such data include instructions for any application or method operating on the device 1200, contact data, phone book data, messages, pictures, videos, etc. The memory 1204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0157] The power supply component 1206 provides power to the various components of the device 1200. The power supply component 1206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1200.
[0158] The multimedia component 1208 includes a screen that provides an output interface between the device 1200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1208 includes a front camera and / or a rear camera. When the device 1200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0159] The audio component 1210 is configured to output and / or input audio signals. For example, the audio component 1210 includes a microphone (MIC), and when the device 1200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1204 or sent via the communication component 1216. In some embodiments, the audio component 1210 also includes a speaker for outputting audio signals.
[0160] The input / output interface 1212 provides an interface between the processing component 1202 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0161] The sensor assembly 1214 includes one or more sensors for providing various aspects of the status assessment of the device 1200. For example, the sensor assembly 1214 can detect the open / closed state of the device 1200, the relative positioning of components, such as the display and keypad of the device 1200, the sensor assembly 1214 can also detect the position change of the device 1200 or a component of the device 1200, the presence or absence of user contact with the device 1200, the orientation or acceleration / deceleration of the device 1200, and the temperature change of the device 1200. The sensor assembly 1214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0162] The communication component 1216 is configured to facilitate wired or wireless communication between the device 1200 and other devices. The device 1200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0163] In an exemplary embodiment, the device 1200 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned methods.
[0164] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1204 including instructions, and the instructions can be executed by the processor 1220 of the device 1200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0165] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned task processing method when executed by the programmable device.
[0166] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0167] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A task processing method, characterized in that: The method comprises: Acquire first data to be inferred; In the process of performing the first data reasoning task according to the first data, obtaining second data to be processed; A first data processing task is performed according to the second data.
2. The method according to claim 1, characterized in that The performing the first data processing task according to the second data includes: The second data is quantized to obtain second target data.
3. The method according to claim 2, characterized in that The quantizing the second data to obtain the second target data comprises: The data type of the second data is quantified into a first data type to obtain second target data, where the first data type is the data type used when performing the first data reasoning task.
4. The method according to claim 2, characterized in that: The method further comprises: The inference result is dequantized to obtain first target data, where the inference result is an inference result obtained after the first data inference task is performed on the first data.
5. The method according to claim 4, characterized in that The performing dequantization processing on the inference result to obtain the first target data includes: The data type of the inference result is dequantized into a second data type to obtain first target data, where the second data type is a data type used when performing a first data processing task.
6. The method according to claim 1, characterized in that After executing the first data processing task according to the second data, the data processing result obtained by the first data processing task is used as input data for executing a specified data reasoning task, and the specified data reasoning task is the next data reasoning task of the first data reasoning task.
7. The method according to any one of claims 1 to 6, characterized in that: Executing a first data reasoning task according to the first data includes: According to the first data, a first data reasoning task is performed by a first processor; The performing the first data processing task according to the second data includes: A first data processing task is performed by a second processor according to the second data, and the first processor and the second processor are different processors.
8. A task processing device, characterized in that: include: A first acquisition module is configured to acquire first data to be inferred; A second acquisition module is configured to acquire second data to be processed in the process of performing the first data reasoning task according to the first data; The execution module is configured to execute the first data processing task according to the second data.
9. A task processing device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the steps of the method according to any one of claims 1 to 7 when executed.
10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.