Resource allocation method and device, equipment and medium

Abstract

The invention provides a resource allocation method and device, equipment and a medium, and the method comprises the steps: obtaining the basic information of candidate algorithms, the basic information is used for indicating the computing power, memory and time consumption demands of each candidate algorithm in the same business scene, and the candidate algorithms comprise a neural network algorithm and / or a common algorithm; according to the basic information, a resource division strategy corresponding to the candidate algorithm is obtained, and the resource division strategy is used for indicating division of existing NPU resources by each candidate algorithm; and when the service runs, starting candidate algorithms according to the resource division strategy, so that each candidate algorithm runs according to the pre-allocated NPU resources. According to the method, the computing power requirements of the candidate algorithms can be confirmed in an offline mode, the multiple candidate algorithms are deployed to different resources of the NPU by dividing the resources of the NPU, parallel acceleration of a neural network algorithm and / or a common algorithm is achieved, the utilization rate of the NPU resources is increased, and meanwhile loads of other resources are reduced.

Description

Technical Field

[0001] The present application relates to the field of intelligent vehicle technology, and in particular to a resource allocation method, device, equipment and medium. Background Art

[0002] How to make full use of computing power is one of the key issues that the smart cabin and smart driving industries need to solve. Running as many algorithms as possible with limited hardware resources and providing more cost-effective software and hardware solutions are important goals of the industry. Currently, smart cabins and smart driving usually provide NPU to accelerate the reasoning speed of neural networks.

[0003] Smart cabin and smart driving algorithms include neural network algorithms and traditional algorithms. Neural network algorithm models usually use NPU to accelerate the reasoning speed of neural networks. Some neural network algorithm models have low computing power requirements but require exclusive use of NPU. Some algorithms still use CPU or GPU for acceleration. On the one hand, this causes idleness and waste of resources in NPU, and on the other hand, it causes high CPU and GPU load, affecting the real-time performance of GPU rendering. Summary of the invention

[0004] Based on the above problems, the present application provides a resource allocation method, device, equipment and medium.

[0005] In a first aspect, the present application provides a resource allocation method, the method comprising:

[0006] Obtain basic information of candidate algorithms, where the basic information is used to indicate the computing power, memory, and time consumption requirements of each candidate algorithm in the same business scenario, and the candidate algorithms include neural network algorithms and / or common algorithms;

[0007] According to the basic information, a resource partitioning strategy corresponding to the candidate algorithm is obtained, where the resource partitioning strategy is used to indicate how each candidate algorithm partitions existing NPU resources;

[0008] When the service is running, the candidate algorithms are started according to the resource partitioning strategy, so that each candidate algorithm runs according to the pre-allocated NPU resources.

[0009] In a possible implementation manner, acquiring, according to the basic information, a resource partitioning strategy corresponding to the candidate algorithm includes:

[0010] According to the basic information, hardware resource requirement information corresponding to each of the candidate algorithms is obtained, where the hardware resource requirement information includes at least one of register occupancy information, memory occupancy information, and thread occupancy information;

[0011] According to the hardware resource requirement information, corresponding hardware resources are allocated to each candidate algorithm, where the hardware resources belong to the NPU resources;

[0012] According to the basic information, obtaining operation attribute information of each candidate algorithm, the operation attribute information including topological relationship, response time, operation time and execution cycle;

[0013] According to the running attribute information, the starting time point and priority of each candidate algorithm are obtained so that all the candidate algorithms can be completed within the total running time required by the business scenario. The priority is used to indicate the execution priority of the candidate algorithm.

[0014] In a possible implementation manner, after starting the candidate algorithm according to the resource partitioning strategy, the method further includes:

[0015] Acquire state information of the candidate algorithm, where the state information is used to indicate a corresponding running state of the candidate algorithm during this running;

[0016] According to the status information, if the actual response time and / or QoS data in the candidate algorithm does not meet the target algorithm corresponding to the basic information, the resource allocation strategy is adjusted, and the QoS data is used to indicate the actual performance of the candidate algorithm for the preset QoS indicators when the business is running.

[0017] In a possible implementation manner, the adjusting the resource partitioning strategy includes:

[0018] Obtaining hardware resource usage information corresponding to the target algorithm within the actual response time;

[0019] According to the hardware resource usage information, if it is determined that a target candidate algorithm other than the target algorithm occupies the hardware resources of the target algorithm, then according to the operation attribute information, obtaining a first priority corresponding to the target candidate algorithm and a second priority corresponding to the target algorithm, wherein the first priority is higher than the second priority;

[0020] The resource partitioning strategy is adjusted according to the first priority and the second priority.

[0021] In a possible implementation manner, adjusting the resource partitioning strategy according to the first priority and the second priority includes:

[0022] If both the first priority and the second priority are higher than or equal to the preset level, determining whether the continuous time period in which no candidate algorithm is running before the start time point of the target algorithm within the total running time satisfies the running time corresponding to the target candidate algorithm;

[0023] If yes, then changing the start time point of the target candidate algorithm so that the target candidate algorithm is completed before the start time point of the target algorithm;

[0024] If both the first priority and the second priority are lower than the preset level, or only the first priority is higher than or equal to the preset level, the start time point corresponding to the target algorithm is changed so that the target algorithm is started after the target candidate algorithm is finished running.

[0025] In a possible implementation, the method further includes:

[0026] If the continuous time period in which no candidate algorithm runs before the start time point of the target algorithm does not meet the running time corresponding to the target candidate algorithm, or there is no target candidate algorithm other than the target algorithm occupying the hardware resources of the target algorithm;

[0027] Then, the hardware resource allocation for the target algorithm is increased so that the actual response time and / or QoS data of the target algorithm meet the corresponding basic information.

[0028] In a possible implementation, the method further includes:

[0029] During each subsequent business operation, the latest resource allocation strategy is obtained;

[0030] Start the candidate algorithm according to the latest resource partitioning strategy, and repeat the step of obtaining the status information of the candidate algorithm. If the current resource partitioning strategy needs to be adjusted, repeat the step of adjusting the resource partitioning strategy according to the first priority and the second priority.

[0031] In a second aspect, the present application provides a resource allocation device, the device comprising:

[0032] An acquisition module, used to acquire basic information of candidate algorithms, where the basic information is used to indicate the computing power, memory and time consumption requirements of each candidate algorithm in the same business scenario, and the candidate algorithms include neural network algorithms and / or ordinary algorithms;

[0033] A processing module, configured to obtain a resource partitioning strategy corresponding to the candidate algorithm according to the basic information, wherein the resource partitioning strategy is used to indicate how each candidate algorithm partitions existing NPU resources;

[0034] The processing module is further used to start the candidate algorithms according to the resource partitioning strategy when the service is running, so that each candidate algorithm runs according to the pre-allocated NPU resources.

[0035] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in any one of the first aspects.

[0036] In a fourth aspect, the present application provides an electronic device, comprising: at least one processor and a memory; wherein:

[0037] The memory stores computer-executable instructions;

[0038] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method as described in any one of the first aspects.

[0039] The resource allocation method, apparatus, device and medium provided in the present application obtain the basic information of the candidate algorithms to confirm the computing power, memory and time requirements of each candidate algorithm, and the candidate algorithms include neural network algorithms and / or ordinary algorithms. Based on the basic information, a resource partitioning strategy is obtained to indicate the division of existing NPU resources by each candidate algorithm. And when the business is running, the candidate algorithm is started based on the resource partitioning strategy, so that each candidate algorithm runs according to the pre-allocated NPU resources. The method can confirm the computing power requirements of the candidate algorithms in an offline manner, and by dividing the resources of the NPU, multiple candidate algorithms are deployed on different resources of the NPU, thereby realizing parallel acceleration of neural network algorithms and / or ordinary algorithms, improving the utilization rate of NPU resources, and reducing the load on other resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] Figure 1 A system business flow chart for applying the resource allocation method provided in the embodiment of the present application;

[0042] Figure 2 Resource allocation method process provided in the embodiment of the present application Figure 1 ;

[0043] Figure 3 Resource allocation method process provided in the embodiment of the present application Figure 2 ;

[0044] Figure 4 Resource allocation method process provided in the embodiment of the present application Figure 3 ;

[0045] Figure 5 A diagram of a resource allocation device provided by an embodiment of the present invention;

[0046] Figure 6 A hardware schematic diagram of an electronic device provided by an embodiment of the present invention.

[0047] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0049] In the field of smart cabins / smart driving, the algorithms used usually include neural network algorithms and ordinary algorithms. Among them, neural network algorithm models usually use NPU to accelerate the reasoning speed of neural networks. Some neural network algorithm models have low computing power requirements but require exclusive NPU. Ordinary algorithms still use CPU or GPU for acceleration. Therefore, on the one hand, it is easy to cause idleness and waste of resources in NPU, and on the other hand, it is easy to cause high CPU and GPU load, affecting the real-time performance of GPU rendering.

[0050] Specifically, the common computing resource allocation methods in the industry are mainly focused on CPU and GPU: in terms of CPU, parallel processing is achieved by arranging different CPU cores to handle different algorithms; in terms of GPU, for example, on the NVDIA Orin platform, based on the relatively mature GPU scheduling framework MPS (Multi-process service), a driver-level scheduling mechanism can be provided to assist in achieving the goal, but the scheduling mechanism does not provide a corresponding scheduling strategy, nor does it provide a method for designing resource scheduling strategies; in terms of NPU, it is usually used in an exclusive manner, and there is currently no resource allocation and scheduling method for NPU.

[0051] Based on the above problems, the present application proposes a resource allocation method, which obtains the basic information of the candidate algorithm, obtains the resource partitioning strategy corresponding to the candidate algorithm according to the basic information, and starts the candidate algorithm according to the resource partitioning strategy when the business is running, so that each candidate algorithm runs according to the pre-allocated NPU resources. At the same time, during operation, the status information of the candidate algorithm is collected, and the resource scheduling strategy is adjusted according to the status information. This method can fully evaluate the computing power requirements of the algorithm in an offline manner, including neural network algorithms and / or ordinary algorithms, and by dividing the resources of the NPU, multiple neural network algorithms and / or ordinary algorithms are deployed on different resources in the NPU, thereby realizing parallel acceleration of the algorithm. And optimize the resource allocation strategy through dynamic adjustment. This method improves the utilization rate of NPU resources and effectively reduces the load on the GPU.

[0052] The method provided in this application is applicable to various cloud, automotive and mobile hardware platforms (including mobile phones, VR, etc.), and is applicable to NPU hardware of various chip platforms.

[0053] Figure 1 The system business flow chart of the resource allocation method provided in the embodiment of the present application is as follows. Figure 1 As shown, the business process includes an offline scheduling generation phase and an online dynamic adjustment phase.

[0054] Specifically, in the offline scheduling generation stage, the offline scheduling orchestration module obtains the basic information of the candidate algorithm from the algorithm module, wherein the algorithm module contains the basic information of each candidate algorithm; the basic information includes the computing power, memory and time consumption requirements of each candidate algorithm in the same business scenario. Based on the basic information, the offline scheduling orchestration module divides the candidate algorithm into corresponding resource partitioning strategies. The offline scheduling orchestration module outputs the resource partitioning strategy to the scheduling configuration module so that when the business is running, the candidate algorithm is configured with resources according to the resource partitioning strategy.

[0055] Furthermore, in the online dynamic adjustment stage, the scheduling configuration module configures resources for the candidate algorithms according to the existing resource partitioning strategy. When the candidate algorithms start running based on the pre-configured resources, the monitoring module monitors each candidate algorithm online and obtains the status information of the candidate algorithms in real time. When the candidate algorithms finish running, the monitoring module feeds back the acquired status information to the scheduling configuration module, so that the scheduling configuration module adjusts the current resource partitioning strategy according to the status information and updates the resource configuration before the next candidate algorithm runs.

[0056] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0057] Figure 2 Resource allocation method process provided in the embodiment of the present application Figure 1 .like Figure 2 As shown, the method includes:

[0058] S201. Obtain basic information of candidate algorithms, where the basic information is used to indicate the computing power, memory, and time requirements of each candidate algorithm in the same business scenario, and the candidate algorithms include neural network algorithms and / or ordinary algorithms.

[0059] In this step, the candidate algorithm represents the algorithm that needs to be run in the business scenario. The basic information of the candidate algorithm may include the algorithm's running attribute information and hardware resource requirement information. The running attribute information represents the topological relationship, response time, running time (running time under various hardware resource restriction requirements), execution cycle, etc. between algorithms; the hardware resource requirement information represents the algorithm's demand information for resources such as memory, registers, and threads. In addition, the basic information may also include algorithm identification information to confirm or store the candidate algorithm and the corresponding basic information based on the algorithm identification. By obtaining the basic information of the candidate algorithm, a basis can be provided for subsequent resource division and scheduling. At the same time, by understanding the basic information of the candidate algorithm, the resource requirements and performance of the candidate algorithm in a specific environment can be better estimated.

[0060] S202. Obtain a resource partitioning strategy corresponding to the candidate algorithm according to the basic information, where the resource partitioning strategy is used to indicate how each candidate algorithm partitions existing NPU resources.

[0061] In this step, NPU resources include running time, memory, registers, threads, contexts, and acceleration modules. Among them, memory refers to storage resources such as TCM / DRAM / SRAM / Cache, and acceleration modules refer to acceleration hardware such as Scalar / HVX / HMX of Qualcomm NPU. According to the basic information of the candidate algorithm, the corresponding hardware resources are configured offline, including memory, registers, threads, etc. For some candidate algorithms with large resource consumption, the configuration of context or acceleration modules can also be added to support their operation. At the same time, according to the running attribute information of the candidate algorithm, the running attributes of the candidate algorithm can be further set, such as priority or algorithm start time. According to the running attributes, on the one hand, an indication of the running of the candidate algorithm is provided. On the other hand, if a conflict in the use of hardware resources between algorithms still occurs during actual operation, it can be dynamically adjusted according to the pre-set running attributes to avoid the impact of the conflict.

[0062] S203: When the service is running, start the candidate algorithms according to the resource partitioning strategy, so that each candidate algorithm runs according to the pre-allocated NPU resources.

[0063] In this step, the corresponding resources are configured for each candidate algorithm module according to the resource partitioning strategy to support the normal operation of the candidate algorithm. For example, each candidate algorithm can be allocated to a corresponding NPU core or resource pool according to the resource partitioning strategy to ensure that each candidate algorithm obtains the resources it needs.

[0064] Therefore, this method is suitable for scenarios where neural network algorithms run in parallel, or scenarios where ordinary algorithms run in parallel, and also supports scenarios where neural network algorithms and ordinary algorithms run in combination.

[0065] Optionally, when the business is running, the status information of each candidate algorithm can be obtained to understand the performance of the candidate algorithm in actual operation to confirm whether the candidate algorithm meets the needs of the business scenario, so as to further adjust the current resource partitioning strategy. And when the business is running next time, the latest resource partitioning strategy is used to support the operation of the candidate algorithm. In this way, the resource partitioning strategy can be dynamically updated and adjusted to ensure the performance and efficiency of business operation and effectively improve it.

[0066] The resource allocation method provided in this embodiment obtains the basic information of the candidate algorithms to confirm the computing power, memory and time requirements of each candidate algorithm, and the candidate algorithms include neural network algorithms and / or ordinary algorithms. Based on the basic information, a resource partitioning strategy is obtained to indicate the division of existing NPU resources for each candidate algorithm. And when the business is running, the candidate algorithm is started based on the resource partitioning strategy, so that each candidate algorithm runs according to the pre-allocated NPU resources. This method can confirm the computing power requirements of the candidate algorithms in an offline manner, and by dividing the resources of the NPU, multiple candidate algorithms can be deployed on different resources of the NPU, thereby realizing parallel acceleration of neural network algorithms and / or ordinary algorithms, improving the utilization rate of NPU resources, and reducing the load on other resources.

[0067] Figure 3 Resource allocation method process provided in the embodiment of the present application Figure 2 .like Figure 3 As shown, in Figure 2 Based on the embodiment, the process of obtaining the resource partitioning strategy corresponding to the candidate algorithm is described in detail. The method includes:

[0068] S301. Obtain basic information of candidate algorithms, and based on the basic information, obtain hardware resource requirement information corresponding to each of the candidate algorithms.

[0069] The hardware resource requirement information includes at least one of register occupancy information, memory occupancy information and thread occupancy information.

[0070] In this step, the hardware resource requirement information is further obtained from the basic information of the candidate algorithm. The hardware resource requirement information involves aspects such as register occupancy, memory occupancy or thread occupancy. Specifically, register occupancy indicates the number of registers required by the algorithm during operation. Memory occupancy indicates the size of the memory space required for the algorithm to execute. Thread occupancy indicates the number of threads required for the algorithm to execute in parallel. By confirming the above hardware resource requirements, a basis for reasonable planning and utilization of NPU resources can be provided.

[0071] S302. Allocate corresponding hardware resources for each candidate algorithm according to the hardware resource requirement information, where the hardware resources belong to the NPU resources.

[0072] In this step, the available NPU resources are divided to each candidate algorithm according to the hardware resource requirement information of each candidate algorithm. Different algorithms can be assigned to different NPU cores or resource pools to fully utilize hardware resources and meet the requirements of the algorithms.

[0073] For example, if a candidate algorithm has high requirements for registers and memory but low requirements for threads, it can be assigned to a core with more register and memory resources. On the other hand, if a candidate algorithm has high requirements for thread parallelism, it can be assigned to a core with more thread resources. By properly partitioning hardware resources, each algorithm is ensured to get the resources it needs and maximize system performance and utilization.

[0074] S303: Acquire operation attribute information of each candidate algorithm according to the basic information, wherein the operation attribute information includes topological relationship, response time, operation time and execution cycle.

[0075] In this step, the operation attribute information part is further obtained from the basic information of the candidate algorithm to determine the operation characteristics and behavior of the candidate algorithm. Specifically, the topological relationship indicates the dependency and data flow relationship between algorithms. It is very important for parallel execution and scheduling algorithms, mainly because it is necessary to ensure the correct flow of data and the correct collaboration between algorithms. Response time refers to the time required for an algorithm to receive input data and generate output results. This is an important indicator for evaluating algorithm performance and response speed. Shorter response time corresponds to higher performance. Run time represents the time required for the actual execution of the algorithm, which can measure the algorithm execution efficiency and resource consumption. Shorter run time indicates higher efficiency. It should be understood that the algorithm running time is less than the response time and is part of the response time. The execution cycle represents the repeated execution cycle of the algorithm. For some algorithms that need to be executed periodically, the execution cycle is a key parameter. Based on the operation attribute information of each algorithm, the execution of the algorithm can be planned and scheduled to meet the requirements of the business scenario.

[0076] S304. According to the running attribute information, the starting time point and priority of each candidate algorithm are obtained so that all the candidate algorithms can be completed within the total running time required by the business scenario. The priority is used to indicate the execution priority of the candidate algorithm.

[0077] In this step, the start time point indicates when the candidate algorithm starts to execute in the business scenario. Exemplarily, the start time point of each candidate algorithm can be determined based on the operation attribute information, including the dependencies between the candidate algorithms, to ensure the normal flow of data and the sequential execution of the candidate algorithms. Furthermore, according to the attribute information such as the response time, running time and execution cycle of the candidate algorithm, an appropriate priority is assigned to each candidate algorithm, and the candidate algorithm with a higher priority can be executed and scheduled first when a resource conflict occurs.

[0078] The resource allocation method provided in this embodiment obtains the hardware resource requirement information of each candidate algorithm through the basic information of the candidate algorithm, so as to allocate the corresponding hardware resources to each candidate algorithm. And further obtains the operation attribute information of each candidate algorithm based on the basic information, so as to determine the start time point and priority of each candidate algorithm, so as to meet the requirement that all the candidate algorithms are completed within the total operation time required by the business scenario. This method realizes the pre-allocation of algorithm resources in a static manner, and improves the efficiency of algorithm operation and resource utilization.

[0079] Figure 4 Resource allocation method process provided in the embodiment of the present application Figure 3 ,like Figure 4 As shown, in Figure 3 Based on the embodiment, the process of adjusting the resource partitioning strategy is described in detail. The method includes:

[0080] S401. Obtain state information of the candidate algorithm, where the state information is used to indicate a corresponding running state of the candidate algorithm during this running.

[0081] In this step, when the candidate algorithm is running in the current business scenario, the status information of each candidate algorithm is obtained through real-time monitoring. Exemplarily, the current running status of the candidate algorithm can be obtained through a monitoring tool or interface, including but not limited to hardware resource usage information, actual response time, QoS data, etc. By collecting the real-time status information of the candidate algorithm, basic data is provided for subsequent optimization.

[0082] S402. According to the state information, if the candidate algorithms have actual response time and / or QoS data that do not satisfy the target algorithm corresponding to the basic information, then obtain hardware resource usage information corresponding to the target algorithm within the actual response time.

[0083] The QoS data is used to indicate the actual performance of the candidate algorithm for the preset QoS indicators when the service is running.

[0084] In this step, in order to confirm whether the actual operation of the candidate algorithm meets the requirements of the business scenario, it is possible to check whether the actual response time and / or QoS data of the candidate algorithm meet expectations. If not, the non-compliant candidate algorithm is confirmed as the target algorithm, and the hardware resource usage information is obtained to further analyze the problem. Specifically, based on the previously obtained status information, check whether the candidate algorithm meets the response time and / or preset QoS indicator value in the basic information. If not, the specific target algorithm that does not meet the requirements is confirmed, and the hardware resource usage information of the target algorithm within the actual response time is obtained. It should be noted that any one of the actual response time and QoS data can be selected as a standard for measuring whether the candidate algorithm meets the requirements of the business scenario, or the two indicators can be used in combination as a reference standard, which is not limited here.

[0085] QoS (Quality of Service) stands for quality of service, which is used to measure the performance and service quality of an algorithm to meet specific performance requirements or agreed levels. Through QoS, we can focus on the data processing speed and throughput of the algorithm, and confirm whether the algorithm's service quality reaches the expected level based on the actual performance of the algorithm on these indicators, or provide a basis for further optimization. Hardware resource usage information includes the use of hardware resources such as memory and registers by the target algorithm in actual operation.

[0086] S403. According to the hardware resource usage information, if it is determined that a target candidate algorithm other than the target algorithm occupies the hardware resources of the target algorithm, then according to the operation attribute information, obtain a first priority corresponding to the target candidate algorithm and a second priority corresponding to the target algorithm.

[0087] The first priority is higher than the second priority.

[0088] In this step, by analyzing the hardware resource usage information, it is determined whether there are other candidate algorithms occupying the hardware resources required by the target algorithm. If so, the priority information of the target candidate algorithm and the target algorithm is further obtained. It should be understood that the target candidate algorithm indicates the algorithm that occupies the hardware resources of the target algorithm. When resource preemption occurs, the prerequisite is that the priority of the target candidate algorithm that preempts resources is higher than the priority of the target algorithm. In subsequent analysis, the priority can first be used to determine whether the above two algorithms have higher operating requirements, for example, the required response time must be met, or must be executed first, etc.

[0089] S404: If both the first priority and the second priority are higher than or equal to the preset level, determine whether the continuous time period in which no candidate algorithm is running before the start time point of the target algorithm within the total running time satisfies the running time corresponding to the target candidate algorithm.

[0090] In this step, the priority critical value of the algorithm with higher running requirements can be described by setting a preset level. When the priority of the algorithm is greater than or equal to the preset level, it means that the running requirements of the algorithm are relatively high. When resources are preempted, it is not applicable to delay the operation of the algorithm or take other concession measures. Therefore, when the first priority of the target candidate algorithm that preempts resources and the second priority of the target algorithm are both higher than or equal to the preset level, it means that the running requirements of the above two algorithms are relatively high, and it is necessary to further consider setting the target candidate algorithm to run in advance while ensuring that the original start time point of the target algorithm remains unchanged. In order to achieve the above purpose, it is necessary to first confirm whether the continuous time period before the start time point of the target algorithm when no candidate algorithm is running can meet the running time of the target candidate algorithm.

[0091] S405: If yes, then change the start time point of the target candidate algorithm so that the target candidate algorithm is completed before the start time point of the target algorithm.

[0092] S406: If not, increase the allocation of hardware resources to the target algorithm so that the actual response time and / or QoS data of the target algorithm meet the corresponding basic information.

[0093] It should be noted that, if there is enough time, the start time point of the target candidate algorithm can be adjusted to ensure that it completes its operation before the target algorithm is started to avoid hardware resource conflicts. When there is not enough time, the hardware resource allocation of the target algorithm can be increased to meet its actual response time and / or QoS data to meet the basic information requirements. Increasing the allocation of hardware resources includes but is not limited to the allocation of threads, memory, registers, etc.

[0094] S407: If both the first priority and the second priority are lower than the preset level, or only the first priority is higher than or equal to the preset level, change the start time point corresponding to the target algorithm so that the target algorithm is started after the target candidate algorithm is finished running.

[0095] In this step, if the first priority of the target candidate algorithm and the second priority of the target algorithm are both lower than the preset level, it means that the operation requirements of the above two algorithms are relatively low. At this time, you can consider postponing the start time of the target algorithm. Similarly, when only the first priority of the target candidate algorithm is higher than or equal to the preset level, you can also consider postponing the start time of the target algorithm. When changing the start time of the target algorithm, you can set it to start the target algorithm immediately after the target candidate algorithm is finished running according to the actual situation, or choose to run it in a certain time period that meets the running conditions according to the resource usage.

[0096] Exemplarily, if the continuous time period in which no candidate algorithm runs before the start time point of the target algorithm does not meet the running time corresponding to the target candidate algorithm, or there is no target candidate algorithm other than the target algorithm occupying the hardware resources of the target algorithm;

[0097] Then, the hardware resource allocation for the target algorithm is increased so that the actual response time and / or QoS data of the target algorithm meet the corresponding basic information.

[0098] Exemplarily, each time a business is run thereafter, the latest resource partitioning strategy is obtained;

[0099] Start the candidate algorithm according to the latest resource partitioning strategy, and repeat the step of obtaining the status information of the candidate algorithm. If the current resource partitioning strategy needs to be adjusted, repeat the step of adjusting the resource partitioning strategy according to the first priority and the second priority.

[0100] It should be noted that in order to ensure the timely update of the resource partitioning strategy during each subsequent business operation, the latest resource partitioning strategy needs to be obtained to ensure that if adjustments were made during the last operation, the current operation uses the corresponding adjusted resource partitioning strategy. And in each subsequent operation, the status information of the candidate algorithm is monitored and obtained in real time to determine whether to adjust the current resource partitioning strategy. During the adjustment, further optimization is performed based on the priority information, thereby realizing continuous optimization and adjustment of the resource partitioning strategy, ensuring the business scenario requirements and improving the algorithm operation efficiency.

[0101] The resource allocation method provided in this embodiment obtains the status information of the candidate algorithm. When it is confirmed that the actual response time and / or QoS data in the candidate algorithm do not meet the target algorithm corresponding to the basic information, it is confirmed through the hardware resource usage whether there is a target candidate algorithm preempting the target algorithm. If so, the resource division strategy is adjusted according to the priority of the two algorithms; if not, the hardware resource allocation of the target algorithm is further increased. This method realizes the continuous optimization of the resource allocation strategy through dynamic adjustment, ensures the orderly operation of the algorithm, and effectively improves the utilization rate of NPU resources.

[0102] Figure 5 FIG. 1 is a diagram of a resource allocation device provided by an embodiment of the present invention. Figure 5 As shown, the device 50 includes: an acquisition module 501 and a processing module 502.

[0103] An acquisition module 501 is used to acquire basic information of candidate algorithms, where the basic information is used to indicate the computing power, memory and time consumption requirements of each candidate algorithm in the same business scenario, and the candidate algorithms include neural network algorithms and / or common algorithms;

[0104] The processing module 502 is used to obtain a resource partitioning strategy corresponding to the candidate algorithm according to the basic information, where the resource partitioning strategy is used to indicate how each candidate algorithm partitions existing NPU resources;

[0105] The processing module 502 is further configured to start the candidate algorithms according to the resource partitioning strategy when the service is running, so that each candidate algorithm runs according to the pre-allocated NPU resources.

[0106] Figure 6 Schematic diagram of the hardware of the electronic device provided by the embodiment of the present invention. Figure 6 As shown, the electronic device 60 provided in this embodiment includes: at least one processor 601 and a memory 602. The device 60 also includes a communication component 603. The processor 601, the memory 602 and the communication component 603 are connected via a bus 604.

[0107] In a specific implementation process, at least one processor 601 executes the computer-executable instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0108] The specific implementation process of the processor 601 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0109] In the above Figure 6 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0110] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0111] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0112] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described above is implemented.

[0113] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0114] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0115] The division of the units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, which may be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] Each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0118] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0119] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0120] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A resource allocation method, characterized in that: The method comprises: Obtain basic information of candidate algorithms, where the basic information is used to indicate the computing power, memory, and time consumption requirements of each candidate algorithm in the same business scenario, and the candidate algorithms include neural network algorithms and / or common algorithms; According to the basic information, a resource partitioning strategy corresponding to the candidate algorithm is obtained, where the resource partitioning strategy is used to indicate how each candidate algorithm partitions existing NPU resources; When the service is running, the candidate algorithms are started according to the resource partitioning strategy, so that each candidate algorithm runs according to the pre-allocated NPU resources.

2. The method according to claim 1, characterized in that The acquiring, according to the basic information, a resource partitioning strategy corresponding to the candidate algorithm includes: According to the basic information, hardware resource requirement information corresponding to each of the candidate algorithms is obtained, where the hardware resource requirement information includes at least one of register occupancy information, memory occupancy information, and thread occupancy information; According to the hardware resource requirement information, corresponding hardware resources are allocated to each candidate algorithm, where the hardware resources belong to the NPU resources; According to the basic information, obtaining operation attribute information of each candidate algorithm, the operation attribute information including topological relationship, response time, operation time and execution cycle; According to the running attribute information, the starting time point and priority of each candidate algorithm are obtained so that all the candidate algorithms can be completed within the total running time required by the business scenario. The priority is used to indicate the execution priority of the candidate algorithm.

3. The method according to claim 1, characterized in that After starting the candidate algorithm according to the resource partitioning strategy, the method further includes: Acquire state information of the candidate algorithm, where the state information is used to indicate a corresponding running state of the candidate algorithm during this running; According to the status information, if the actual response time and / or QoS data in the candidate algorithm does not meet the target algorithm corresponding to the basic information, the resource allocation strategy is adjusted, and the QoS data is used to indicate the actual performance of the candidate algorithm for the preset QoS indicators when the business is running.

4. The method according to claim 3, characterized in that The adjusting the resource partitioning strategy includes: Obtaining hardware resource usage information corresponding to the target algorithm within the actual response time; According to the hardware resource usage information, if it is determined that a target candidate algorithm other than the target algorithm occupies the hardware resources of the target algorithm, then according to the operation attribute information, obtaining a first priority corresponding to the target candidate algorithm and a second priority corresponding to the target algorithm, wherein the first priority is higher than the second priority; The resource partitioning strategy is adjusted according to the first priority and the second priority.

5. The method according to claim 4, characterized in that The adjusting the resource partitioning strategy according to the first priority and the second priority includes: If both the first priority and the second priority are higher than or equal to the preset level, determining whether the continuous time period in which no candidate algorithm is running before the start time point of the target algorithm meets the running time corresponding to the target candidate algorithm within the total running time; If yes, then changing the start time point of the target candidate algorithm so that the target candidate algorithm is completed before the start time point of the target algorithm; If both the first priority and the second priority are lower than the preset level, or only the first priority is higher than or equal to the preset level, the start time point corresponding to the target algorithm is changed so that the target algorithm is started after the target candidate algorithm is finished running.

6. The method according to claim 5, characterized in that The method further comprises: If the continuous time period in which no candidate algorithm runs before the start time point of the target algorithm does not meet the running time corresponding to the target candidate algorithm, or there is no target candidate algorithm other than the target algorithm occupying the hardware resources of the target algorithm; Then, the hardware resource allocation for the target algorithm is increased so that the actual response time and / or QoS data of the target algorithm meet the corresponding basic information.

7. The method according to claim 6, characterized in that The method further comprises: During each subsequent business operation, the latest resource allocation strategy is obtained; Start the candidate algorithm according to the latest resource partitioning strategy, and repeat the step of obtaining the status information of the candidate algorithm. If the current resource partitioning strategy needs to be adjusted, repeat the step of adjusting the resource partitioning strategy according to the first priority and the second priority.

8. A resource allocation device, characterized in that: The device comprises: An acquisition module, used to acquire basic information of candidate algorithms, where the basic information is used to indicate the computing power, memory and time consumption requirements of each candidate algorithm in the same business scenario, and the candidate algorithms include neural network algorithms and / or ordinary algorithms; A processing module, configured to obtain a resource partitioning strategy corresponding to the candidate algorithm according to the basic information, wherein the resource partitioning strategy is used to indicate how each candidate algorithm partitions existing NPU resources; The processing module is further used to start the candidate algorithms according to the resource partitioning strategy when the service is running, so that each candidate algorithm runs according to the pre-allocated NPU resources.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: at least one processor and memory; wherein, The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 7.

Images