Deep learning task allocation method and apparatus, and electronic device

Abstract

The invention relates to the technical field of smart home, and discloses a deep learning task allocation method and device, and electronic equipment, and the method comprises the steps: analyzing heterogeneous computing resources of each processor in target equipment, and obtaining an analysis result; determining a target factor according to the demand of the to-be-allocated deep learning task and the resource condition of the target device; according to the parameter value corresponding to the target factor and the task scheduling function, calculating the priority of the to-be-allocated deep learning task to obtain a calculation result; according to the analysis result and the calculation result, distributing the deep learning task to a target processor; the target processor is provided with a compressed large model and is used for executing a deep learning task. And task allocation is performed according to the analysis result and the calculation result, and the deep learning task is allocated to the most suitable processor, so that the execution efficiency of the deep learning task is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home, for example to a method and device for allocating deep learning tasks, and an electronic device. BACKGROUND

[0002] With the rapid development of deep learning technology, its application in various fields is becoming more and more widespread. In the smart home system, various complex deep learning tasks need to be processed, such as image recognition, voice interaction, environment monitoring, etc. In order to improve the execution efficiency and performance of these tasks, deep learning models are usually used, and the use of deep learning models requires a large amount of computing resources and memory. Smart home devices usually have multiple heterogeneous computing resources, such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), NPU (Neural Processing Unit), etc. The performance characteristics of these resources are different, and how to effectively use these resources for deep learning task allocation to improve resource utilization and task execution efficiency is a problem to be solved.

[0003] In related technologies, a static allocation method is used to implement task allocation of heterogeneous computing resources, and deep learning tasks are allocated to different processors according to pre-set rules.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art:

[0005] The related technology cannot adapt to changes in task requirements and device states, thereby affecting the execution efficiency of deep learning tasks.

[0006] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0007] To have a basic understanding of some aspects of the disclosed embodiments, a brief overview is given below. The summary is not a general review, nor is it intended to determine key / important constituent elements or delineate the scope of protection of these embodiments, but as a prelude to the detailed description below.

[0008] The embodiments of the present disclosure provide a method and device for allocating deep learning tasks, and an electronic device, to improve the execution efficiency of deep learning tasks.

[0009] In some embodiments, the method for assigning a deep learning task comprises: analyzing heterogeneous computing resources of each processor in a target device to obtain an analysis result; determining a target factor according to a requirement of the deep learning task to be assigned and a resource condition of the target device; calculating a priority of the deep learning task to be assigned according to a parameter value corresponding to the target factor and a task scheduling function to obtain a calculation result; assigning the deep learning task to a target processor according to the analysis result and the calculation result; and the target processor is provided with a compressed large model for executing the deep learning task.

[0010] Optionally, the analyzing the heterogeneous computing resources of each processor in the target device comprises: traversing the heterogeneous computing resources of each processor in the target device; determining performance parameters and characteristics of each processor, and analyzing performance differences of each processor in different deep learning tasks.

[0011] Optionally, the target factor comprises one or more of a calculation complexity, a memory requirement, a time limit, a model characteristic, a device state, and real-time running information; and the determining the target factor comprises one or more of: determining the calculation complexity according to an algorithm and a data size of the deep learning task; determining the memory requirement according to an amount of device memory required for executing the deep learning task; determining the time limit required for completing the task according to a requirement and a priority of the deep learning task; determining the model characteristic according to a type, a size, and a structure of a large model in the processor; determining the device state according to running state information of the target device; and determining the real-time running information according to a current load and a task execution condition of the target device.

[0012] Optionally, the task scheduling function is determined according to the following formula:

[0013] P(t) = exp(w1 x Comp(t)) + arctan(w2 x Mem(t)) + tanh(w3 x Limit(t)) + w4 x Comp(t) x Mem(t) + w5 x Model(task) + w6 x Comp(t) x Limit(t) + w7 x Mem(t) x Limit(t) + f(RuntimeInfo)

[0014] P(t) = w1*Comp(t) + w2*Mem(t) + w3*Limit(t) + w4*Comp(t)*Mem(t) + w5*Model(task) + w6*Comp(t)*Limit(t) + w7*Mem(t)*Limit(t) + f(RuntimeInfo), wherein P(t) is a priority of the deep learning task, w1 is a weight value of the computational complexity, Comp(t) is a parameter value of the computational complexity, w2 is a weight value of the memory requirement, Mem(t) is a parameter value of the memory requirement, w3 is a weight value of the time limit, Limit(t) is a parameter value of the time limit, w4 is a weight value of an interaction term of the computational complexity and the memory requirement, w5 is a weight value of a model characteristic, Model(task) is a parameter value of the model characteristic, w6 is a weight value of an interaction term of the computational complexity and the time limit, w7 is a weight value of an interaction term of the memory requirement and the time limit, and f(RuntimeInfo) is an influence of real-time running information on the priority of the task.

[0015] Optionally, the deep learning task is assigned to the target processor according to the analysis result and the calculation result, including: determining the influence degree of the target factor on the task execution efficiency according to the analysis result, adjusting the task assignment strategy; adjusting the priority of the deep learning task in the calculation result according to the task execution situation of the deep learning task; assigning the deep learning task to the target processor according to the adjusted task assignment strategy and the calculation result.

[0016] Optionally, the influence degree of the target factor on the task execution efficiency is determined according to the analysis result, and the task assignment strategy is adjusted, including: determining the correlation of the target factor with the task execution efficiency according to the analysis result; preliminarily adjusting the weight value of the target factor in the task assignment strategy according to the correlation of the target factor; and re-adjusting the preliminarily adjusted weight value based on historical data of task execution and device state data through reinforcement learning.

[0017] Optionally, the deep learning task is assigned to the target processor according to the adjusted task assignment strategy and the calculation result, including: determining the priority order of the deep learning task according to the adjusted calculation result; determining the target processor corresponding to the deep learning task according to the adjusted task assignment strategy; and assigning the deep learning task to the target processor according to the priority order.

[0018] Optionally, the large model is compressed to the target processor in the following manner: a target model compression algorithm is determined according to the characteristics of the large model; a target processor is determined according to the execution efficiency of the large model on different processors; and the large model is compressed to the target processor based on the target model compression algorithm.

[0019] In some embodiments, the deep learning task assignment apparatus includes a processor and a memory storing program instructions, the processor being configured to execute the deep learning task assignment method as described above when running the program instructions.

[0020] In some embodiments, the electronic device comprises: an electronic device body; and the deep learning task allocation apparatus as described above, which is installed on the electronic device body.

[0021] The deep learning task allocation method and apparatus, and the electronic device provided by the embodiments of the present disclosure can achieve the following technical effects:

[0022] In the embodiments of the present disclosure, by analyzing the heterogeneous computing resources of each processor in the device, the performance characteristics of different processors can be understood, thereby providing accurate basic information for the allocation of deep learning tasks. According to the requirements of the deep learning task to be allocated and the resource status of the device, the introduction of target factors can be realized, and then according to the parameter values corresponding to the target factors and the task scheduling function, the priority of the deep learning task can be accurately evaluated and calculated. Thus, the task allocation can be performed according to the analysis result and the calculation result, the deep learning task can be allocated to the most suitable processor, and the execution efficiency of the deep learning task can be improved.

[0023] The foregoing general description and the following description are merely exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0024] One or more embodiments are exemplarily illustrated by corresponding drawings, which are not intended to limit the embodiments, elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute a proportional limit, and wherein:

[0025] Figure 1 is a system environment schematic diagram of a deep learning task allocation method provided by the embodiments of the present disclosure;

[0026] Figure 2 is a schematic diagram of a deep learning task allocation method provided by the embodiments of the present disclosure;

[0027] Figure 3 is a schematic diagram of another deep learning task allocation method provided by the embodiments of the present disclosure;

[0028] Figure 4 is a schematic diagram of a large model compression method provided by the embodiments of the present disclosure;

[0029] Figure 5 is a schematic diagram of another deep learning task allocation method provided by the embodiments of the present disclosure;

[0030] Figure 6 is a schematic diagram of a deep learning task allocation apparatus provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0031] In order to enable a person skilled in the art to more fully understand the features and technical contents of the embodiments of the present disclosure, the implementation of the embodiments of the present disclosure is described in detail below with reference to the accompanying drawings, which are only used for reference and do not limit the embodiments of the present disclosure. In the following technical description, in order to facilitate explanation, a plurality of details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be simplified to facilitate the drawings.

[0032] The terms "first", "second", and the like in the technical solutions described in the present application are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0033] Unless otherwise specified, the term "a plurality of" means two or more.

[0034] In the embodiments of the present disclosure, the character " / " represents an "or" relationship between the objects before and after it. For example, A / B represents: A or B.

[0035] The term "and / or" is a description of the association between objects, which means that there can be three relationships. For example, A and / or B means: A or B, or, A and B, the three relationships.

[0036] The term "corresponding" can refer to an association or binding relationship. A and B correspond to each other means that there is an association or binding relationship between A and B.

[0037] According to an aspect of the embodiments of the present application, a deep learning task allocation method is provided. The deep learning task allocation method is widely used in smart home, smart home, smart home device ecosystem, intelligent house ecosystem, and whole-house intelligent digital control application scenarios. Optionally, in the present embodiment, the deep learning task allocation method can be applied to the hardware environment composed of a terminal device 102 and a server 104 as shown in Figure 1 As shown in Figure 1 The server 104 is connected with the terminal device 102 through a network, which can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal, a database can be set on the server or independently of the server, which is used to provide data storage services for the server 104, cloud computing and / or edge computing services can be configured on the server or independently of the server, which is used to provide data operation services for the server 104.

[0038] The network can include, but is not limited to, at least one of the following: a wired network, a wireless network. The wired network can include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, a local area network. The wireless network can include, but is not limited to, at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 can be, but is not limited to, a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart oven, a smart refrigerator, a smart oven, a smart oven, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart television, a smart clothesline, a smart curtain, a smart audio and video, a smart socket, a smart sound, a smart sound box, a smart fresh air device, a smart kitchen and bathroom device, a smart bathroom device, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steamer, a smart microwave oven, a smart kitchen, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0039] In combination Figure 2 As shown, the deep learning task allocation method provided by the embodiments of the present disclosure can be executed by the central control device of the smart home, and the allocation method comprises:

[0040] In S201, the central control device analyzes the heterogeneous computing resources of each processor in the target device and obtains an analysis result.

[0041] In S202, the central control device determines a target factor according to the requirements of the deep learning task to be allocated and the resource status of the target device.

[0042] In S203, the central control device determines a task scheduling function according to the target factor and the weight of the target factor, and calculates the priority of the deep learning task according to the task scheduling function, and obtains a calculation result.

[0043] In S204, the central control device allocates the deep learning task to the target processor according to the analysis result and the calculation result; the target processor is provided with a compressed large model for executing the deep learning task.

[0044] In the embodiments of the present disclosure, by analyzing the heterogeneous computing resources of each processor in the device, the performance characteristics of different processors can be understood, thereby providing accurate basic information for the allocation of deep learning tasks. According to the requirements of the deep learning task to be allocated and the resource status of the device, the introduction of the target factor can be realized, and then according to the parameter value corresponding to the target factor and the task scheduling function, the priority of the deep learning task can be accurately evaluated. Therefore, the task allocation can be performed according to the analysis result and the calculation result, and the deep learning task can be allocated to the most suitable processor, thereby improving the execution efficiency of the deep learning task.

[0045] Optionally, the target device includes any one of the terminal devices described above.

[0046] Optionally, the heterogeneous computing resources of each processor in the target device are analyzed, including: traversing the heterogeneous computing resources of each processor in the target device; determining the performance parameters and characteristics of each processor, and analyzing the performance differences of each processor in different deep learning tasks.

[0047] In this embodiment, by traversing the heterogeneous computing resources in the target device, all available processor types in the target device can be comprehensively identified, and the performance parameters of each processor can be obtained. The computing power, memory access performance, parallel processing capability and other key performance indicators of each processor in the target device can be accurately understood. Accurate basic information is provided for subsequent task allocation, ensuring that tasks can be executed on the most suitable processor, thereby improving task execution efficiency. Moreover, by analyzing the performance differences of each processor in different deep learning tasks, the strengths and weaknesses of each processor in a specific task can be determined, so as to identify which processors perform better in a specific task. For example, GPUs perform well in parallel computing tasks such as image recognition, while CPUs are more advantageous in logical processing tasks such as speech recognition. By analyzing the performance differences of each processor, task allocation strategies can be optimized to ensure that tasks can be reasonably allocated according to the characteristics of the processor.

[0048] Optionally, traversing the heterogeneous computing resources of each processor in the target device includes: comprehensively scanning the computing resources in the target device to determine the processor types present in the device.

[0049] In this embodiment, the computing resources in the device can be comprehensively scanned through a system interface or a hardware detection tool. The processor types present in the device are determined, including common CPUs, GPUs and NPUs, etc. At the same time, other auxiliary computing resources that may exist are identified, such as DSPs (Digital Signal Processors), FPGAs (Field-Programmable Gate Arrays), etc. For each processor type, its basic hardware information can be obtained, such as the model, core count, clock frequency, cache size, etc. of the processor. These parameters directly affect the computing power and performance of the processor.

[0050] Optionally, the performance parameters and characteristics of each processor are determined, including one or more of the following: computing power evaluation, memory access performance evaluation, parallel processing capability evaluation, and specific task adaptability evaluation.

[0051] Optionally, the computing capability evaluation includes evaluating the computing capability of each processor by executing benchmark programs or referring to the technical specifications of the processors. For CPUs, the number of instructions executed per second, floating-point operation performance, and other indicators can be considered; for GPUs and NPUs, their parallel processing capabilities, acceleration performance of specific neural network operations, and other indicators can be focused on. The memory access performance evaluation includes analyzing the memory architecture and access speed of different processors. For example, GPUs usually have large amounts of video memory, but the access delay of video memory may be different from that of CPU memory. It is helpful to consider memory requirements and data transfer efficiency when assigning tasks. The parallel processing capability evaluation includes evaluating the parallel processing capability of the processor, especially for processors such as GPUs and NPUs that are suitable for large-scale parallel computing. Consider the number of parallel processing units, parallelism, vector processing capability, and other factors. These parameters are crucial for large-scale data parallelism and model parallelism in deep learning tasks. The specific task adaptability evaluation includes analyzing the adaptability of different processors in specific deep learning tasks. For example, some processors may perform well in image recognition tasks, while others may be more suitable for natural language processing tasks.

[0052] Optionally, the performance differences of each processor in different deep learning tasks are analyzed, including analyzing the computing speed, memory occupation, power consumption, and other indicators of the processor in different deep learning tasks. In this embodiment, different models or versions of processors may have significant differences in performance. By performing a series of tests for specific deep learning tasks, the performance of different models or versions of processors can be compared. For example, in the image classification task, the processing time, accuracy, and video memory occupation of different models of GPUs are compared. Through these tests, the processor model or version with the best performance in a specific task can be determined. In addition, the software support of different models or versions of processors can also be analyzed. For example, some processors may have better optimization for specific deep learning frameworks or libraries, which will also affect their performance in actual tasks.

[0053] Optionally, other auxiliary computing resources are also included in the target device. For example, DSPs can be used for specific digital signal processing tasks, and FPGAs can be programmed to implement customized hardware acceleration functions. For these auxiliary computing resources, analyze their potential application scenarios and performance advantages in deep learning tasks. For example, DSPs may have high efficiency in audio processing tasks, and FPGAs can be customized according to the requirements of specific tasks to achieve higher performance and energy efficiency. Consider the cooperative working mode of auxiliary computing resources and other processors. For example, FPGAs can be used in combination with CPUs or GPUs to achieve hardware acceleration for specific tasks. Analyze the performance improvement and implementation difficulty of this cooperative working mode to provide more choices for task allocation decisions.

[0054] Optionally, the target factors include one or more of computational complexity, memory requirement, time limit, model characteristics, device status, and real-time running information.

[0055] In this embodiment, in order to realize efficient dynamic allocation of deep learning tasks, data of multiple target factors need to be collected. By comprehensively considering these target factors, the demand of the task and the resource status of the device can be more accurately evaluated, so that more reasonable task allocation decisions can be made. The data of these target factors are collected by using the built-in monitoring tools and sensors of the system, as well as the algorithm analysis tools related to deep learning tasks. Ensure the accuracy and real-time of the data, so as to timely adjust the task allocation strategy, improve the resource utilization and task execution efficiency of the device.

[0056] Optionally, the computational complexity is determined according to the algorithm and data size of the deep learning task. In this embodiment, the amount of computation required by the task is an important indicator of computational complexity. The computational complexity is determined by analyzing the algorithm and data size of the task. For different types of deep learning tasks, the complexity of their algorithms and data sizes may differ greatly. For example, for image recognition tasks, the structure and parameter quantity of the neural network model used, as well as the resolution and quantity of the input images, are analyzed. According to the computation graph of the model and the data processing flow, the number of floating point operations, integer operations, etc. required by the task are estimated, so as to determine the computational complexity. For natural language processing tasks, factors such as the length of the text, the size of the vocabulary, the depth and width of the model, etc. are considered. The calculation steps and operation types of the model when processing text data, such as word vector calculation, neural network layer operation, etc. are analyzed to determine the computational complexity.

[0057] Optionally, the memory requirement is determined according to the amount of device memory required for the execution of the deep learning task. In this embodiment, the amount of device memory required for the execution of the task is a key indicator of memory requirement. The memory requirement is determined by analyzing the data size and memory access pattern of the task. Different deep learning tasks may require different amounts of memory to store data and model parameters. The data type and size of the task are analyzed, such as the resolution and channel number of image data, the length and vocabulary size of text data, etc. The storage mode of data in memory is considered, such as array, matrix, tensor, etc. and the dynamic change of data, such as data growth and shrinkage. At the same time, the memory access pattern of the task is analyzed, including sequential access, random access, locality access, etc. Different access patterns will affect the memory usage efficiency and cache hit rate. According to the data size and access pattern, the memory capacity required by the task is estimated, including the memory required for input data, intermediate results and output data.

[0058] Optionally, determine the time limit required to complete the task according to the requirements and priorities of the deep learning task. In this embodiment, the longest time acceptable or the expected completion time of the task is determined according to the requirements and priorities of the task. Time limit is one of the important constraints of task allocation, which directly affects the priority and allocation strategy of the task. For tasks with high real-time requirements, such as autonomous driving, video monitoring, etc., the time limit may be very strict, and the task needs to be completed in the shortest time. For other tasks, a relatively loose time limit can be determined according to their importance and urgency. Considering the priority and resource requirements of the task, as well as the current load and available resources of the device, a reasonable time limit is determined. At the same time, a certain amount of time is reserved to deal with possible unexpected situations and resource competition.

[0059] Optionally, determine the model characteristics according to the type, size and structure of the large model in the processor. In this embodiment, the influence of the type, size and structure of the large model on the data of the degree of computational intensity is analyzed. Different types of large models may have great differences in computational requirements and resource consumption. For example, convolutional neural networks perform well in image recognition tasks, but require a large amount of computational resources to process large-scale image data. Recurrent neural networks are widely used in natural language processing tasks, but due to their sequence processing characteristics, they may have longer training time and memory requirements. Analyze the size of the model, including the number of parameters, the number of layers, the number of neurons, etc. Large-scale models usually require more computational resources and memory to store and process. At the same time, analyze the structure of the model, such as the depth, width, connection method of the network, etc. Different structures may affect the computational efficiency and parallelism of the model.

[0060] Optionally, determine the device state according to the running state information of the target device. In this embodiment, the device running temperature, current CPU load and other information are obtained through the system interface or sensor. Device state is one of the important reference factors for task allocation, which directly affects the performance and stability of the device. Use the built-in temperature sensor and performance monitoring tool of the device to obtain the running temperature and CPU load of the device in real time. High temperature may cause performance degradation or even damage to the device, while high CPU load may affect the execution speed and stability of the task. At the same time, consider other device state information, such as memory usage, disk space, network bandwidth, etc.

[0061] Optionally, real-time running information is determined according to the current load and task execution of the target device. In this embodiment, instant feedback such as the current task queue length, processor usage, and average processing time of completed tasks is collected. Real-time running information can help understand the current load and task execution of the device, so as to adjust the task allocation strategy in a timely manner. The length of the task queue and the priority of the task are obtained in real time by using the task scheduler and performance monitoring tool of the system. According to the queue length and priority, the execution order of the task and the allocation of resources are reasonably arranged. At the same time, the usage of the processor is monitored, including the utilization rate and load of the CPU, GPU, NPU, etc. Understanding the usage of the processor can help select the appropriate processor to execute the task, improve resource utilization and task execution efficiency. The average processing time of completed tasks is collected to analyze the execution efficiency and resource consumption of the task. According to the average processing time, the time limit and resource allocation of the task are adjusted to improve overall performance.

[0062] The steps of determining target factors are described below in the real-time face recognition task in the intelligent security system. In the intelligent security system, the system built-in monitoring tool continuously collects data of various target factors. For example, the device status and task execution are sampled once every second.

[0063] The determination of computational complexity requires analyzing the deep learning algorithm used in the face recognition task. Suppose a face recognition model based on deep convolutional neural network is used. By analyzing the structure of the model, it is determined that it contains multiple convolutional layers, pooling layers and fully connected layers. Then the amount of computation required to process an image is calculated. For example, according to the number of parameters and the calculation operation of each layer, it is estimated that about 10 million floating point operations are required to process an image. Considering that the system needs to process multiple image streams from cameras in real time, the computational complexity will increase with the increase of image quantity and processing frequency.

[0064] The analysis of memory requirements requires analyzing the data size of the task. The resolution of each face image is 1920x1080, the color channel is 3 (RGB), and each pixel occupies 8 bits (1 byte), so a single image occupies about 6MB of memory. Then the memory access mode needs to be considered. Since the face recognition task needs to process consecutive image frames, the data access has a certain locality. At the same time, the model needs to store intermediate results and parameters during running, which will also occupy a certain amount of memory. Finally, the amount of memory required by the entire system when running the face recognition task is estimated. Assuming that the system processes 10 camera image streams simultaneously and needs to store the last 10 seconds of image data for analysis, the required memory is approximately: (6MBx10x10 secondsx30 frames / second) + model parameter memory (assuming 50MB) ≈ 2GB.

[0065] The determination of time constraints needs to consider the requirements and priorities of tasks. In an intelligent security system, face recognition tasks need to be completed in the shortest possible time to ensure timely identification and response to suspicious personnel. Suppose the system requires face recognition of an image to be completed within 1 second. According to this time limit, combined with the calculation complexity and device performance, adjust the task allocation strategy.

[0066] The analysis of model characteristics first analyzes the type of face recognition model as a deep convolutional neural network. This model has strong capabilities in processing image data, but the calculation is large. Then consider the size of the model. Suppose the model has 10 convolutional layers and 3 fully connected layers, with about 10 million parameters. Such a size requires a large amount of computing resources and memory to store and process. Finally, analyze the structure of the model to affect the degree of calculation-intensive. For example, the number and size of convolutional layers, the setting of pooling layers, etc. will affect the calculation efficiency of the model. Through the analysis of the model structure, it can be determined which parts can be optimized or parallel processed.

[0067] The acquisition of device status, first through the system interface to obtain the running temperature of the device. Suppose the device is equipped with a temperature sensor, which monitors the temperature of the device in real time to be 45℃. If the temperature is too high, it may affect the performance and stability of the device, and the task allocation strategy needs to be adjusted to reduce the device load. Then get the current CPU load. Suppose through the performance monitoring tool found that the CPU utilization is 70%. If the CPU load is too high, some tasks may need to be allocated to other processors.

[0068] The collection of real-time running information, first collect the current task queue length. Suppose there are 50 face images waiting to be processed in the system, according to the queue length and the priority of the task, reasonably arrange the execution order of the task. Then monitor the usage of the processor. For example, it is found that the utilization rate of GPU is 50%, and the utilization rate of NPU is 30%. Some tasks suitable for parallel processing can be allocated to GPU or NPU to improve resource utilization. Finally, collect the average processing time of completed tasks. Suppose in the past period, the average time to complete the recognition of a face image is 0.5 seconds. According to this average processing time, the time limit and resource allocation can be adjusted to improve the overall performance.

[0069] Optionally, the task scheduling function is determined according to the following formula:

[0070] P(t) = exp(w1 x Comp(t)) + arctan(w2 x Mem(t)) + tanh(w3 x Limit(t)) + w4 x Comp(t) x Mem(t) + w5 x Model(task) + w6 x Comp(t) x Limit(t) + w7 x Mem(t) x Limit(t) + f(RuntimeInfo)

[0071] wherein P(t) is the priority of the deep learning task, w1 is the weight value of the computation complexity, Comp(t) is the computation complexity parameter value, w2 is the weight value of the memory requirement, Mem(t) is the memory requirement parameter value, w3 is the weight value of the time limit, Limit(t) is the time limit parameter value, w4 is the weight value of the interaction term of the computation complexity and the memory requirement, w5 is the weight value of the model characteristic, Model(task) is the model characteristic parameter value, w6 is the weight value of the interaction term of the computation complexity and the time limit, w7 is the weight value of the interaction term of the memory requirement and the time limit, and f(RuntimeInfo) is the influence of the real-time running information on the task priority.

[0072] In the task scheduling function, the exponential function and the arctangent function are introduced to replace part of the traditional functions. For the computation complexity, the exponential function is used, which can highlight the influence of high computation complexity on the task priority. For the memory requirement, the arctangent function is used, which makes the influence of the memory requirement on the task priority more smooth within a certain range. For the time limit, the hyperbolic tangent function is used, which maintains its reasonable adjustment effect on the time limit factor. For the interaction term, the product term of the computation complexity and the memory requirement, the interaction term of the computation complexity and the time limit, and the interaction term of the memory requirement and the time limit are used, which can better consider the mutual influence between multiple target factors. In addition, the device state parameter value will affect the calculation results of the related terms in the task scheduling function, thereby indirectly affecting the calculation of the task priority. The task scheduling function comprehensively considers the target factors, which can more comprehensively and accurately evaluate the priority of the task. By dynamically adjusting the weight values in the task scheduling function, the task allocation strategy can be optimized in real time, ensuring that the task can be executed on the most suitable processor, and improving the task execution efficiency.

[0073] Optionally, the parameter value corresponding to the computation complexity is determined according to the operation amount required by the task, the algorithm complexity rating, and the complexity characteristics of the data. In this embodiment, by introducing an algorithm complexity rating system, the task algorithm can be rated according to factors such as the logical structure, the number of loop nesting layers, and the recursion depth of the algorithm. At the same time, the characteristics of the data, such as the sparsity and dimensionality of the data, are analyzed to fine-tune the computation complexity parameter value.

[0074] Optionally, the parameter value corresponding to the memory requirement is determined according to the amount of device memory occupied by the task execution and the memory growth trend during the task execution. In this embodiment, a memory growth model is established by monitoring the memory usage of the task at different stages, the memory requirement change of the task in a future period of time is predicted, and the parameter value of the memory requirement is adjusted according to the prediction result.

[0075] Optionally, the parameter value corresponding to the time limit is determined according to the urgency of the task, the degree of influence on the overall performance of the system, and the dependency relationship with other tasks. In this embodiment, for a task that is urgent, has a greater influence on the performance of the system, and has less dependency on other tasks, a higher parameter value of the time limit is given; otherwise, the parameter value is reduced.

[0076] Optionally, the parameter value corresponding to the model characteristics is determined according to the type, size, structure, and training state of the model. In this embodiment, different initial parameter values are given to different types of models according to their performance on specific tasks and resource requirement characteristics. The larger the size of the model and the more complex the structure, the higher the parameter value. At the same time, the training state of the model is considered, such as whether it is in the training stage, the training progress, etc., to dynamically adjust the parameter value.

[0077] Optionally, the parameter value corresponding to the device state is determined according to the comprehensive evaluation of the running temperature of the device, the CPU load, the memory usage, and other key hardware indicators. In this embodiment, a device state evaluation model is established, the various hardware indicators are weighted and summed to obtain a comprehensive state value, and the parameter value of the device state is determined according to the value.

[0078] Optionally, the parameter value corresponding to the real-time running information is determined according to the length of the current task queue to be processed, the usage of the processor, the average processing time of the completed tasks, and the actual execution progress of the tasks. In this embodiment, the longer the task queue to be processed, the more tense the usage of the processor, the longer the average processing time of the completed tasks, and the slower the actual execution progress of the tasks, the lower the parameter value.

[0079] Optionally, a machine learning-based weight value adjustment system is established to dynamically adjust the weight values in the task scheduling function. In this embodiment, the weight value adjustment system continuously learns and adjusts the weight values by collecting historical data of task execution and device state data using reinforcement learning algorithms. The task allocation process is regarded as a Markov decision process, the state space includes device state, task state and various parameter values, the action space includes allocating tasks to different processors, and the reward function is designed according to the execution time of the task, resource utilization, system overall performance and other indicators. By constantly interacting with the environment, the system learns the optimal weight combination. At the same time, according to the type of task and the current state of the device, different weight value adjustment strategies are set. For example, when the device load is high, the weight of the computational complexity and memory requirement is reduced, and the weight of the time limit is increased to ensure that urgent tasks can be processed in priority; when processing large-scale data tasks, the weight of memory requirement and computational complexity is increased to ensure that the task can be completed within a reasonable time.

[0080] The following takes the image classification and target detection task system as an example to illustrate the priority of deep learning tasks calculated according to the task scheduling function.

[0081] First, the parameter values of each target factor need to be calculated. For the calculation complexity, the convolutional neural network algorithm used by the image classification task is rated as medium complexity according to its logical structure and the number of parameters. At the same time, considering that the input image data has high dimensions and certain complexity, the calculation complexity parameter value is fine-tuned to a medium-high value, such as setting Comp(t) = 0.6. For memory requirements, during the task execution process, the memory usage gradually increases with the continuous input of image data and the storage of intermediate results. Through the established memory growth model prediction, the memory requirement may increase by 20% in the future. Considering the current memory occupation and the predicted growth, the memory requirement parameter value Mem(t) = 0.7 is determined. For time limit, the image classification task has certain requirements for real-time performance, because its results will be used for subsequent target detection tasks and have a greater impact on the overall system performance. At the same time, the task depends less on other tasks. Considering these factors, a higher time limit parameter value is given, set Limit(t) = 0.8. For model characteristics, the convolutional neural network model used is large in size and complex in structure, and is in the training stage. According to these characteristics, an initial parameter value Model(task) = 0.7 is given. As the training progresses, if the model gradually stabilizes, the parameter value can be appropriately reduced. For device status, assuming that the device operating temperature is 40 degrees Celsius, the CPU load is 60%, and the memory usage is 70%. Through the device state evaluation model, these indicators are weighted and summed, and assuming that the comprehensive state value obtained is 0.6, the device state parameter value is determined as 0.6. For real-time running information, the current task queue length is 10, the processor usage is relatively tense, the average processing time of completed tasks is relatively long, and the current image classification task actual execution progress is 30%. Considering these factors, the real-time running information parameter value is determined as 0.5.

[0082] By collecting historical data of task execution and device state data, the weight values are continuously learned and adjusted using reinforcement learning algorithm. Assuming that after a period of learning, the weight values under the current state are obtained: w1 = 0.4, w2 = 0.3, w3 = 0.3, w4 = 0.2, w5 = 0.2, w6 = 0.15, w7 = 0.15. Assuming that the value calculated by f(RuntimeInfo) according to the real-time running information is 0.1.

[0083] Substituting the parameter values of each target factor and the weight values into the task scheduling function, the priority P(t) = 1.74 of the deep learning task is obtained. By calculating the priority of multiple deep learning tasks, the priority can be sorted to better allocate the deep learning tasks.

[0084] Optionally, according to the analysis result and the calculation result, the deep learning task is allocated to the target processor, including: determining the influence degree of the target factor on the task execution efficiency according to the analysis result, adjusting the task allocation strategy; adjusting the priority of the deep learning task in the calculation result according to the task execution of the deep learning task; and allocating the deep learning task to the target processor according to the adjusted task allocation strategy and the calculation result.

[0085] In combination Figure 3 As shown in the drawings, the embodiment of the present disclosure provides another method for allocating a deep learning task, including:

[0086] S301, the central control device analyzes the heterogeneous computing resources of each processor in the target device, and obtains an analysis result.

[0087] S302, the central control device determines a target factor according to the requirements of the deep learning task to be allocated and the resource status of the target device.

[0088] S303, the central control device determines a task scheduling function according to the target factor and the weight of the target factor, and calculates the priority of the deep learning task according to the task scheduling function, and obtains a calculation result.

[0089] S304, the central control device determines the influence degree of the target factor on the task execution efficiency according to the analysis result, and adjusts the task allocation strategy.

[0090] S305, the central control device adjusts the priority of the deep learning task in the calculation result according to the task execution of the deep learning task.

[0091] S306, the central control device allocates the deep learning task to the target processor according to the adjusted task allocation strategy and the calculation result; the target processor is provided with a compressed large model for executing the deep learning task.

[0092] In this embodiment, the historical data of the task execution in the target device can be obtained according to the analysis result, so as to analyze the influence degree of the target factor on the task execution efficiency, and then adjust the task allocation strategy to ensure that the task can be executed on the most suitable processor. According to the actual execution of the task, the priority of the deep learning task is further adjusted to enhance flexibility and adaptability, and ensure that the high-priority task can be processed in time.

[0093] Optionally, the influence degree of the target factor on the task execution efficiency is determined according to the analysis result, and the task allocation strategy is adjusted, including: determining the correlation between the target factor and the task execution efficiency according to the analysis result; preliminarily adjusting the weight value of the target factor in the task allocation strategy according to the correlation of the target factor; and based on the historical data of the task execution and the device state data, the weight value after the preliminary adjustment is adjusted again through reinforcement learning.

[0094] In this embodiment, by calculating the correlation between the target factors and the task execution efficiency, the weight value of the target factor in the task allocation strategy can be accurately adjusted to ensure that the weight value accurately reflects the influence of each factor on the task execution efficiency. For example, if the calculation complexity is positively correlated with the task execution efficiency and the correlation coefficient is large, the weight value of the calculation complexity is increased to ensure that tasks with high calculation complexity are preferentially allocated to suitable processors for execution. By introducing the reinforcement learning algorithm, the weight value can be dynamically adjusted according to the historical data of task execution and the device state data to ensure that the task allocation strategy always remains optimal. At the same time, different weight value adjustment strategies are set according to the type of task and the current state of the device. For example, when the device load is high, the weight of the calculation complexity and the memory requirement is reduced, and the weight of the time limit is increased to ensure that urgent tasks can be processed preferentially; when processing large-scale data tasks, the weight of the memory requirement and the calculation complexity is increased to ensure that the tasks can be completed within a reasonable time.

[0095] Optionally, the correlation between the target factor and the task execution efficiency is determined according to the analysis result, including: obtaining an actual value of the target factor; calculating the task execution efficiency according to the task execution result data in the analysis result; and calculating the correlation between the target factor and the task execution efficiency based on the Pearson correlation coefficient.

[0096] Optionally, the operation amount required by the task is determined by analyzing the task algorithm and the data size to obtain an actual value C of the calculation complexity i . i represents the i-th deep learning task. The device memory amount required for task execution is determined by analyzing the task data size and the memory access mode to obtain an actual value M of the memory requirement i . The actual value L of the time limit is obtained according to the longest time or the expected completion time that can be accepted for completing the task i . The actual value V of the model characteristics is obtained by using a feature vector to represent the influence of the type, size and structure of the model on the calculation-intensive degree and other data model，i . The actual value V of the device state is obtained by using a vector to represent the device running temperature, the current CPU load and other information DevState，i . The actual value V of the real-time running information is obtained by using a vector to represent the current task queue length, the processor usage, the average processing time of the completed tasks and other instant feedback RuntimeInfo，i .

[0097] Optionally, the task execution efficiency is calculated according to the following formula:

[0098]

[0099] wherein, Efficiency iTask execution efficiency of the i-th task, T actual，i Actual execution time of the i-th deep learning task, ResourceUtilization i Resource utilization of the i-th task.

[0100] Optionally, the resource utilization includes comprehensive utilization of CPU, GPU, memory, etc.

[0101] Optionally, the correlation between the computational complexity and the task execution efficiency is calculated according to the following formula:

[0102]

[0103] wherein, R Comp，Efficiency is the correlation between the computational complexity and the task execution efficiency, is the average value of the computational complexity of all deep learning tasks, is the average value of the task execution efficiency of all deep learning tasks.

[0104] Based on the calculation formula of the correlation between the computational complexity and the task execution efficiency, only the computational complexity in the formula is replaced by other target factors, the correlation R Mem，Efficiency between the memory requirement and the task execution efficiency, the correlation R Limit，Efficiency between the time limit and the task execution efficiency, the correlation R Model，Efficiency between the model characteristics and the task execution efficiency, the correlation R DevState，Efficiency between the device state and the task execution efficiency, and the correlation R RuntimeInfo，Efficiency between the real-time running information and the task execution efficiency can be obtained.

[0105] Optionally, the weight value of the target factor in the task allocation strategy is preliminarily adjusted according to the correlation of the target factor, including: if the correlation coefficient of the target factor is greater than 0 and the absolute value is large, it means that the target factor is positively correlated with the task execution efficiency and has a large impact, and the weight value is increased; if the correlation coefficient of the target factor is less than 0 and the absolute value is large, it means that the target factor is negatively correlated with the task execution efficiency and has a large impact, and the weight value is reduced.

[0106] In this embodiment, the initial weight value W Comp，0 of the computational complexity, the initial weight value W Mem，0 of the memory requirement, the initial weight value W Limit，0 of the time limit, the initial weight value W Model，0 of the model characteristics, the initial weight value W DevState，0 of the device state, and the initial weight value W RuntimeInfo，0 of the real-time running information are set. For the computational complexity, if R Comp，Efficiency > 0.5, then W Comp，0Adjusting W Comp，1 = W Comp，0 + a. For memory requirement, if R Mem，Efficiency < -0.5, adjust W Mem，0 to obtain W Mem，1 = W Mem，0 - b. Wherein, a and b are adjustment steps.

[0107] Optionally, reinforcement learning is performed based on historical data of task execution and device state data, including: defining a state space, an action space and a reward function of reinforcement learning; and performing reinforcement learning by using a deep Q network algorithm. In this embodiment, by defining the state space, the action space and the reward function, and performing reinforcement learning by using the deep Q network algorithm, the dynamic task allocation strategy can be optimized, the task execution efficiency can be improved, the flexibility and adaptability of the system can be enhanced, and the resource utilization rate can be improved.

[0108] Optionally, the state space is defined according to the following formula:

[0109] S = {C i , M i , L i , V model，i , V DevState，i , V RuntimeInfo，i , W Comp , W Mem , W Limit , W Model , W DevState , W RuntimeInfo}

[0110] Wherein, S is the state space, W Comp , W Mem , W Limit , W Model , W DevState , W RuntimeInfo are weight values of the calculation complexity, the memory requirement, the time limit, the model characteristics, the device state and the real-time running information in the task allocation strategy respectively.

[0111] Optionally, the action space is defined according to the following formula:

[0112] A = {AssignTaskToCPU, AssignTaskToGPU, AssignTaskToNPU, Adjust Priority, AdjustResourceAllocation}

[0113] Where A is the action space, AssignTaskToCPU assigns the task to the CPU for execution, AssignTaskToGPU assigns the task to the GPU for execution, AssignTaskToNPU assigns the task to the NPU for execution, AdjustPriority adjusts the task priority, and AdjustResourceAllocation adjusts the resource allocation strategy.

[0114] Optionally, the reward function can be defined according to the following formula:

[0115] R = λ1 / T actual，i +λ2×ResourceUtilization i +λ3×StabilityFactor

[0116] Where R is the reward function, StabilityFactor is a measure of system stability, which can be determined based on factors such as equipment operating temperature and load fluctuations, and λ1, λ2, and λ3 are the corresponding weighting coefficients.

[0117] Optionally, a deep Q-network algorithm is used for reinforcement learning, including: initializing the Q-network Q(s, a; θ) and the target network Q(s, a; θ). - ), where θ is the network parameter; for each time step t, action a is selected. t =arg max a Q(s t , a; θ), according to the ∈-greedy strategy, randomly select an action with a certain probability ∈; execute action a t Observe the new state s t+1 and reward r t ; will (s t a t r t s t+1 Store the data in the experience playback buffer; randomly sample a batch of samples (s) from the experience playback buffer. j a j r j s j+1 ); Calculate the target value y j =r j +γmax a′ Q(s j+1 , a′; θ - ), where γ is the discount factor; by minimizing the loss function L(θ)=∑ j (y j -Q(s j a j ;θ)) 2Update the Q-network parameters θ; update the target network θ periodically - ← θ.

[0118] The weight adjustment scheme of the target factor of the image recognition task system is described below.

[0119] For the image recognition task system, assume that an image recognition task system runs on a device with CPU, GPU, and NPU. First, collect and organize data, and record the actual values of the target factors of multiple image recognition tasks. For example, for computational complexity, a complex image recognition task may require a large number of convolution operations and matrix multiplication, and its computational complexity value is high, set as C1 = 800 (assuming this is a relative quantitative value). For memory requirements, the task processes high-resolution images and requires large memory to store intermediate results and model parameters, set as M1 = 500MB. For time limit, it needs to be completed within 2 seconds, i.e. L1 = 2s. The model characteristics are represented by a feature vector, assuming that the model is a deep convolutional neural network with many layers and parameters, and the feature vector can be set as V model，1 = (0.8, 0.6, 0.7), representing the relative values of model depth, parameter quantity, and computational intensity, respectively. The device state vector is assumed to be V DevState，1 = (45℃, 70%), indicating that the current temperature of the device is 45℃ and the CPU load is 70%. The real-time running information vector is assumed to be V RuntimeInfo，1 = (10, 60%, 1.5s), representing the current task queue length of 10, GPU utilization of 60%, and average processing time of completed tasks of 1.5 seconds, respectively. The actual execution time T actual，i = 1.8s, resource utilization ResourceUtilization i = 80%.

[0120] After a period of task execution, data of multiple similar tasks is collected. The Pearson correlation coefficient is used to calculate the correlation between each target factor and task execution efficiency. For example, the correlation between computational complexity and task execution efficiency is calculated as follows: first, calculate the average value of all task computational complexity and the average value of all task execution efficiency Assume that after calculation Then calculate the correlation coefficient R Comp，Efficiency . Assume that after calculation R Comp，Efficiency = 0.6, indicating that there is a strong positive correlation between computational complexity and task execution efficiency.

[0121] Assume that the initial weight value of the computational complexity is W Comp，0 = 0.3, W Mem，0 = 0.2, WLimit，0 = 0.2, W Model，0 = 0.1, W DevState，0 = 0.1, W RuntimeInfo，0 = 0.1. Since the calculation complexity is positively correlated with the task execution efficiency and the correlation coefficient is large, the weight value is increased according to the adjustment strategy. Assuming that the adjustment step size a = 0.05, the new weight value is W Comp，1 = W Comp，0 + a = 0.3 + 0.05 = 0.35.

[0122] Finally, reinforcement learning is performed, and the state space can be represented as S = {800, 500MB, 2s, (0.8, 0.6, 0.7), (45℃, 70%), (10, 60%, 1.5s), 0.35, 0.2, 0.2, 0.1, 0.1, 0.1}. The action space assumes that the current task is assigned to the GPU for execution, i.e., a t = AssignTaskToGPU. The reward function assumes R = λ1 / 1.8 + λ2×0.8 + λ3×StabilityFactor, where λ1, λ2, λ3 are preset weight coefficients, and StabilityFactor is set to 0.8 according to the stability of the device running. Assuming that the reward value is 0.7 after calculation. Through the deep Q network algorithm, the strategy is continuously learned and adjusted, and with the execution of more tasks, the weight value and task allocation strategy are continuously optimized.

[0123] Optionally, according to the task execution of the deep learning task, the priority of the deep learning task in the calculation result is adjusted, including: obtaining the task execution of the deep learning task according to the intermediate state of the deep learning task; and adjusting the priority of the deep learning task according to the influence of the task execution on the task execution.

[0124] In this embodiment, by monitoring the intermediate state and task execution of the deep learning task in real time, the priority of the task can be dynamically adjusted to ensure that high-priority tasks can be processed in a timely manner. For example, when part of the results of a task have been generated and subsequent calculations depend on the completion of other tasks, the priority of the task is appropriately reduced, and tasks that have a dependency relationship with the current task are prioritized, which can reduce the waiting time and improve overall efficiency.

[0125] Optionally, in the smart home system, the intermediate state of the deep learning task includes: "smart home appliance device state monitoring data is being transmitted", "image recognition partial results of the smart security system have been generated", "scene setting instructions of the smart lighting control system are being processed", etc.

[0126] In this embodiment, the current intermediate state of the task can be identified through the information fed back by the sensor and communication module. For example, when the smart home appliance device sends state data to the system, it is identified as the state of "smart home appliance device state monitoring data is being transmitted"; when the smart security system completes a part of image recognition and generates preliminary results, the state is updated to "smart security system image recognition partial results have been generated". The "smart home appliance device state monitoring data is being transmitted" state may cause the task that relies on these data (such as the task of adjusting the energy management strategy according to the state of the household appliance) to be in a waiting state, affecting the execution progress of the task. At the same time, the stability and speed of data transmission will also affect the response time of the task. The "smart security system image recognition partial results have been generated" state means that the subsequent security decision task (such as whether to trigger an alarm, notify the user, etc.) needs to wait for other factors (such as further confirmation of whether it is a false alarm) to continue, occupying a certain system resource. If the image recognition task of the smart security system is in the "partial results have been generated" state, and further confirmation of whether there are other abnormal situations is needed to decide whether to trigger an alarm, the priority of this task can be appropriately reduced. Prioritize tasks that can quickly confirm whether there are other abnormal situations, such as tasks that check whether doors and windows are closed, to reduce waiting time. For the task of "smart home appliance device state monitoring data is being transmitted", if other tasks (such as the task of adjusting the operation mode of the household appliance according to the real-time energy price) rely on these data, the priority can be adjusted according to the progress and remaining time of data transmission. If the data transmission is about to be completed, the priority of the task that relies on the data can be appropriately increased so that the calculation can start immediately after the data transmission is completed.

[0127] Optionally, the allocation method further comprises: adjusting the priority of the deep learning task in the calculation result according to the urgency and importance of the deep learning task.

[0128] In this embodiment, the urgency can be determined according to the degree of influence of the task on the safety and comfort of the home. For example, the task of detecting suspected intrusion by the smart security system has a high degree of urgency and should be given a higher priority. The importance can be determined according to the long-term impact of the task on the quality of life at home. For example, the task of optimizing energy use by the smart energy management system has a high degree of importance and should be given a higher priority. Different priority adjustment rules are set for tasks of different degrees of urgency and importance. For example, for a smart security alarm task that is urgent and important, even if it has a large calculation complexity (such as needing to perform complex image analysis and pattern recognition), resources should be allocated to it for processing in priority; for a task that is not urgent and has low importance, such as a task of periodically updating the software of a smart household appliance, it can be processed when resources are sufficient or delayed.

[0129] In a specific embodiment, for example, a smart home system, a task state monitoring module can be established in the central controller to monitor the intermediate state, urgency and importance of the tasks in real time. According to the monitoring results, the priority of the tasks is dynamically adjusted, and a priority queue is used to manage the tasks. High-priority tasks are placed in the priority processing queue, and low-priority tasks are placed in the ordinary processing queue. When allocating resources, tasks are selected from the priority processing queue for processing first.

[0130] Suppose in a smart home system, multiple tasks are running simultaneously, including image recognition tasks of the intelligent security system, smart home appliance state monitoring tasks, scene setting tasks of the intelligent lighting control system, and optimization tasks of the intelligent energy management system. The image recognition task of the intelligent security system is in the "partial result generated" state, and needs to be further confirmed whether there is a real security threat. At this time, the priority of this task can be reduced, and the task of checking whether the doors and windows are closed is given priority to be processed, so as to determine as soon as possible whether the alarm needs to be triggered. The data of the smart home appliance state monitoring task is being transmitted, and the optimization task of the intelligent energy management system depends on these data. If the data transmission is about to be completed, the priority of the optimization task of the intelligent energy management system can be appropriately increased, so that it can start processing immediately after the data transmission is completed. The scene setting task of the intelligent lighting control system has relatively low urgency and importance, and can be processed when the system resources are sufficient. If the system resources are tight, the execution of the task can be delayed to ensure the smooth progress of high-priority tasks.

[0131] Optionally, according to the adjusted task allocation strategy and the calculation result, the deep learning task is allocated to the target processor, including: determining the priority order of the deep learning task according to the adjusted calculation result; determining the target processor corresponding to the deep learning task according to the adjusted task allocation strategy; and allocating the deep learning task to the target processor according to the priority order.

[0132] In this embodiment, the priority order of the deep learning task is re-determined according to the adjusted calculation result, and the task execution efficiency can be improved by adjusting the task priority and the task allocation strategy, and the high-priority task can be ensured to be processed in time. According to the adjusted task allocation strategy, the target processor corresponding to the deep learning task can be determined. For example, for a computation-intensive task (such as image recognition), it is preferentially allocated to a GPU or NPU for execution to fully utilize the parallel computing capability thereof. For a logical processing task (such as speech recognition), it is preferentially allocated to a CPU for execution to fully utilize the logical processing capability thereof. For a task with high real-time requirement (such as video monitoring), it is preferentially allocated to a processor with low load for execution to ensure that the task can be completed in time. According to the current load of the processor, the task allocation scheme is dynamically adjusted to ensure the load balancing of each processor and avoid resource idling or overload. Finally, the tasks are allocated to the target processor in sequence according to the priority order.

[0133] Optionally, the large model is compressed to the target processor in the following manner: a target model compression algorithm is determined according to the characteristics of the large model; a target processor is determined according to the execution efficiency of the large model on different processors; and the large model is compressed to the target processor based on the target model compression algorithm.

[0134] In combination with Figure 4 As shown in the figure, the present disclosure provides a large model compression method, and the execution subject thereof can also be a central control device of a smart home. The large model compression method comprises the following steps:

[0135] S401. The central control device determines a target model compression algorithm according to the characteristics of the large model.

[0136] S402. The central control device determines a target processor according to the execution efficiency of the large model on different processors.

[0137] S403. The central control device compresses the large model to the target processor based on the target model compression algorithm.

[0138] In this embodiment, a suitable model compression algorithm is selected, which can significantly reduce the number of parameters and the amount of calculation of the large model, so that the large model can run efficiently in a low computing power environment. The execution efficiency of the compressed large model on different processors is analyzed, and the most suitable target processor can be selected to fully utilize the performance advantages of the processor. The compressed large model is allocated to the most suitable processor for execution, which can fully utilize the heterogeneous computing resources in the device and avoid resource idling or overload.

[0139] The following describes the large model compression of an image recognition model in a smart home.

[0140] First, analyze the characteristics of the large model. The structure of the image recognition model may be relatively simple to adapt to limited computing resources. For example, a shallower convolutional neural network structure may be used, reducing the number of convolutional and fully connected layers. The computational demand of the image recognition model is mainly concentrated on real-time analysis of images taken by home cameras, such as recognizing family members, pets, and abnormal situations. Due to the limited computing power of smart home devices, the amount of calculation needs to be controlled within an acceptable range. The image recognition model also needs to consider the memory limitations of smart home devices, trying to reduce the storage of model parameters and intermediate results.

[0141] Then, select the appropriate model compression algorithm. The image recognition model can consider using the knowledge distillation method to extract the knowledge of a larger pre-trained image model and train a smaller model suitable for the smart home environment. For example, a teacher model trained on a large-scale image dataset can be used to transfer its knowledge to a student model trained on a home image dataset. Quantization of the image recognition model is also a suitable choice, converting the model's weights from floating-point numbers to integers, reducing memory usage and computational load. Depending on the computing power and accuracy requirements of smart home devices, the appropriate number of quantization bits can be selected.

[0142] Then, analyze the execution efficiency of the compressed model on different processors. For the compressed image recognition model, if there is a GPU in the device, the compressed image model may have certain performance improvement on the GPU, but due to the high power consumption and cost of the GPU, it may not be able to run for a long time. Analyze the advantages and disadvantages of different processors in processing image tasks. For example, CPU can handle more complex logic and control tasks, while GPU has an advantage in parallel computing. According to the characteristics of the task and the resource situation of the device, reasonably allocate the calculation task of the image model.

[0143] Finally, the compressed model parts are allocated to the most suitable processors for execution. Allocation is made according to the characteristics of different layers or modules of the model. For image recognition models, image preprocessing and simple feature extraction tasks can be allocated to CPUs for execution, while more complex convolution operations and feature fusion tasks can be allocated to GPUs or dedicated image processors. For example, in a home camera, the CPU can first perform preprocessing operations such as noise reduction and cropping on the image, and then transmit the processed image data to the GPU for convolution operations and feature extraction. In a smart home environment, it is necessary to fully utilize the advantages of various heterogeneous computing resources to improve the execution efficiency and performance of the model. For example, use the flexibility and versatility of the CPU to handle complex logic tasks, use the parallel computing capabilities of the GPU or dedicated image processor to handle image tasks, and use the low-power characteristics of the DSP or MCU to handle voice signal preprocessing tasks. At the same time, optimize data transmission and communication mechanisms to reduce data transmission overhead between different processors. For example, use shared memory or high-speed bus technology to improve data transmission speed and efficiency.

[0144] Optionally, the allocation method further includes: monitoring the execution of the deep learning task and the change of the device state in real time, collecting feedback information; determining whether a new task is coming or the device state has changed, if so, returning to the heterogeneous computing resource analysis step to perform task allocation again; if not, continuing to monitor and feedback in real time.

[0145] In combination with Figure 5 The embodiments of the present disclosure provide another deep learning task allocation method, which includes:

[0146] S501, the central control device analyzes the heterogeneous computing resources of each processor in the target device, and obtains an analysis result.

[0147] S502, the central control device determines a target factor according to the requirements of the deep learning task to be allocated and the resource status of the target device.

[0148] S503, the central control device determines a task scheduling function according to the target factor and the weight of the target factor, and calculates the priority of the deep learning task according to the task scheduling function, and obtains a calculation result.

[0149] S504, the central control device allocates the deep learning task to the target processor according to the analysis result and the calculation result; the target processor is provided with a compressed large model for executing the deep learning task.

[0150] S505, the central control device monitors the execution of the deep learning task and the change of the device state in real time, and collects feedback information.

[0151] S506, the central control device determines whether there is a new task or a change in the device state, if yes, S501 is executed, if not, S505 is executed.

[0152] In this embodiment, by monitoring the task execution and device state changes in real time, the task allocation strategy can be dynamically adjusted when a new task arrives or the device state changes, ensuring that the task can be executed on the most suitable processor, enhancing flexibility and adaptability, and improving task execution efficiency.

[0153] In combination Figure 6 As shown in the figure, the embodiment of the present disclosure provides a deep learning task allocation device 600, which includes a processor 601 and a memory 602. Optionally, the device can also include a communication interface 603 and a bus 604. Wherein the processor 601, the communication interface 603, the memory 602 can complete the communication among each other through the bus 604. The communication interface 603 can be used for information transmission. The processor 601 can call the logical instructions in the memory 602 to execute the deep learning task allocation method of the above-mentioned embodiment.

[0154] In addition, the logical instructions in the memory 602 described above can be realized in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium.

[0155] The memory 602 as a kind of computer readable storage medium can be used to store software programs, computer executable programs, such as the program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 601 executes the program instructions / modules stored in the memory 602, thereby executing function application and data processing, i.e. realizing the deep learning task allocation method in the above-mentioned embodiment.

[0156] The memory 602 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function; The data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 602 can include a high-speed random access memory, and can also include a non-volatile memory.

[0157] The embodiment of the present disclosure provides an electronic device, comprising: an electronic device body, and the deep learning task distribution device described above. The deep learning task distribution device is installed in the electronic device body. The installation relationship described herein is not limited to being placed in the electronic device, but also includes installation connection with other components of the electronic device, including but not limited to physical connection, electrical connection or signal transmission connection, etc. Those skilled in the art can understand that the deep learning task distribution device can be adapted to the feasible electronic device body, and thus realize other feasible embodiments.

[0158] The embodiment of the present disclosure provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are configured to execute the deep learning task distribution method.

[0159] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes one or more instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0160] The above description and drawings suffice to fully enable one skilled in the art to practice the embodiments of the present disclosure. Other embodiments can include structural, logical, electrical, process, and other changes. The embodiments are merely representative of possible variations. Individual components and functions are optional unless explicitly required, and the order of operations can be varied. Portions and features of some embodiments can be included in, or substituted for, those of other embodiments. Also, the word "comprising" is used herein to mean the inclusion of an utterance of an item recited in the claim or of any of several components of claims without necessarily excluding other comparable items that are schematically drawn in or that would otherwise fall within the scope of such claim. In addition, the term "comprising" as used in the claims is intended to be synonymous with the term "including" and / or the term "containing" and / or the like. Also, when used in the following claims, the term "comprise" and / or "comprising" and / or "include" and / or "including" and / or "contain" and / or "containing" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "characterized by" and / or "

Claims

1. A method for assigning a deep learning task, characterized in that, The application comprises the following steps: analyzing the heterogeneous computing resources of each processor in the target device to obtain an analysis result; determining a target factor according to the requirements of the deep learning task to be allocated and the resource status of the target device; calculating the priority of the deep learning task to be allocated according to the parameter value corresponding to the target factor and the task scheduling function to obtain a calculation result; allocating the deep learning task to the target processor according to the analysis result and the calculation result; The target processor is provided with a compressed large model for executing the deep learning task.

2. The dispensing method of claim 1, wherein, The analysis of the heterogeneous computing resources of each processor in the target device comprises the following steps: traversing the heterogeneous computing resources of each processor in the target device; determining the performance parameters and characteristics of each processor and analyzing the performance differences of each processor in different deep learning tasks.

3. The dispensing method of claim 1, wherein, The target factor comprises one or more of the following: computational complexity, memory requirement, time limit, model characteristic, device state, and real-time running information. Determining the target factor comprises one or more of the following: determining the computational complexity according to the algorithm and data size of the deep learning task; determining the memory requirement according to the amount of device memory required for executing the deep learning task; determining the time limit required for completing the task according to the requirements and priority of the deep learning task; determining the model characteristic according to the type, size, and structure of the large model in the processor; determining the device state according to the running state information of the target device; determining the real-time running information according to the current load and task execution of the target device.

4. The dispensing method of claim 3, wherein, The task scheduling function is determined according to the following formula: P(t) = exp(w1 × Comp(t)) + arctan(w2 × Mem(t)) + tanh(w3 × Limit(t)) + w4 × Comp(t) × Mem(t) + w5 × Model(task) + w6 × Comp(t) × Limit(t) + w7 × Mem(t) × Limit(t) + f(RuntimeInfo) wherein P(t) is the priority of the deep learning task, w1 is the weight value of the computational complexity, Comp(t) is the parameter value of the computational complexity, w2 is the weight value of the memory requirement, Mem(t) is the parameter value of the memory requirement, w3 is the weight value of the time limit, Limit(t) is the parameter value of the time limit, w4 is the weight value of the interaction term of the computational complexity and the memory requirement, w5 is the weight value of the model characteristic, Model(task) is the parameter value of the model characteristic, w6 is the weight value of the interaction term of the computational complexity and the time limit, w7 is the weight value of the interaction term of the memory requirement and the time limit, and f(RuntimeInfo) is the influence of the real-time running information on the task priority.

5. The method of distributing according to claim 1, wherein, Allocating the deep learning task to the target processor according to the analysis result and the calculation result comprises the following steps: determining the influence degree of the target factor on the task execution efficiency according to the analysis result, and adjusting the task allocation strategy; adjusting the priority of the deep learning task in the calculation result according to the task execution of the deep learning task. According to the adjusted task allocation strategy and the calculation result, the deep learning task is allocated to the target processor.

6. The dispensing method of claim 5, wherein, According to the analysis result, the influence degree of the target factor on the task execution efficiency is determined, and the task allocation strategy is adjusted, including: According to the analysis result, the correlation between the target factor and the task execution efficiency is determined; According to the correlation of the target factor, the weight value of the target factor in the task allocation strategy is preliminarily adjusted; Based on the historical data and the equipment state data of the task execution, reinforcement learning is performed to adjust the preliminarily adjusted weight value again.

7. The dispensing method of claim 5, wherein, According to the adjusted task allocation strategy and the calculation result, the deep learning task is allocated to the target processor, including: According to the adjusted calculation result, the priority order of the deep learning task is determined; According to the adjusted task allocation strategy, the target processor corresponding to the deep learning task is determined; According to the priority order, the deep learning task is allocated to the target processor.

8. The dispensing method according to any one of claims 1 to 7, characterized in that, The large model is compressed to the target processor in the following manner: According to the characteristics of the large model, the target model compression algorithm is determined; According to the execution efficiency of the large model on different processors, the target processor is determined; Based on the target model compression algorithm, the large model is compressed to the target processor.

9. A device for allocating a deep learning task, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the deep learning task allocation method according to any one of claims 1 to 8 when running the program instructions.

10. An electronic device, comprising: Including: Electronic device body; The deep learning task allocation device according to claim 9 is installed in the electronic device body.

Images