Unlock instant, AI-driven research and patent intelligence for your innovation.

Model allocation method and device, electronic equipment and computer readable storage medium

An allocation method and model technology, applied in computing, resource allocation, program control design, etc., can solve problems such as difficulty in achieving the performance balance of each NPU, and difficulty in fully utilizing the performance of smart boxes, and achieve the effect of improving performance utilization.

Pending Publication Date: 2021-11-26
创新奇智(重庆)科技有限公司
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, at present, the allocation of each algorithm model in the smart box is usually realized by artificial pre-designation, but the accuracy of this method depends heavily on the experience of the allocator, and the allocation results are often difficult to achieve the performance balance of each NPU, so it is difficult to fully Take advantage of the capabilities of the smart box

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Model allocation method and device, electronic equipment and computer readable storage medium
  • Model allocation method and device, electronic equipment and computer readable storage medium
  • Model allocation method and device, electronic equipment and computer readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] In order to achieve a more balanced model distribution in the smart box and make full use of the performance of the smart box, an embodiment of the present application provides a model distribution method. see figure 1 as shown, figure 1 It is a schematic flow chart of the model allocation method provided in the embodiment of this application, including:

[0031] S101: Obtain each model to be allocated.

[0032] In this embodiment of the application, each model to be allocated may be determined by an engineer or a user according to a required business scenario. The model to be allocated may be various known or unknown network models, which are not limited in this embodiment of the present application.

[0033] S102: Obtain the first resource usage index value in each NPU in all allocation combination schemes for the model to be allocated.

[0034] It should be understood that the allocation and combination scheme described in the embodiment of the present applicatio...

Embodiment 2

[0076] On the basis of Embodiment 1, this embodiment takes a more specific model allocation process as an example to further illustrate this application.

[0077] Step 1: Obtain the running time of all the models to be allocated on each NPU of the smart box (that is, the running time of the model to be allocated when running in the NPU) and the memory usage (that is, the time required for the model to be allocated in the NPU) used internal memory).

[0078] Step 2: Obtain all distribution combination schemes of all models to be distributed loaded on each NPU of the smart box.

[0079] Step 3: Calculate the total running time of all models to be allocated corresponding to each NPU under each allocation combination scheme for each model to be allocated, and calculate the performance balance index under each allocation combination scheme.

[0080] For example, suppose there are 4 NPUs numbered 1, 2, 3, and 4 in the smart box, and assuming that under a certain allocation combinat...

Embodiment 3

[0085] Based on the same inventive concept, the embodiment of the present application also provides a model allocation device 300 . see image 3 as shown, image 3 shows the use of figure 1 The model distribution apparatus for the method shown. It should be understood that for the specific functions of the device 300, reference may be made to the foregoing description, and to avoid repetition, detailed descriptions are appropriately omitted here. The device 300 includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in an operating system of the device 300 . specifically:

[0086] see image 3 As shown, the apparatus 300 includes: an acquisition module 301 , a determination module 302 and an allocation module 303 . in:

[0087] The obtaining module 301 is used to obtain each model to be allocated, and to obtain the first resource usage index value in each embedded neural network processor NPU in all...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a model allocation method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring each to-be-allocated model; obtaining a first resource use index value in each NPU in all allocation combination schemes for the to-be-allocated model; wherein the first resource use index value in each NPU represents the resource use condition in each NPU; determining a target allocation combination scheme with the most balanced first resources use index value according to the first resources use index value in each NPU under each allocation combination scheme; and according to the target allocation combination scheme, loading the to-be-allocated models to the corresponding NPUs respectively. Therefore, in the intelligent box loaded with the to-be-distributed model, the performance of each NPU is relatively balanced, so that the performance utilization rate of the intelligent box can be improved.

Description

technical field [0001] The present application relates to the technical field of edge computing, and in particular, to a model distribution method, device, electronic equipment, and computer-readable storage medium. Background technique [0002] With the development of edge computing, NPU (Neural-network Processing Unit, embedded neural network processor) specially used for deep neural network reasoning is more and more widely used. At present, there are various smart boxes equipped with multiple NPUs on the market, through which the parallel deployment of multiple algorithm models can be realized, thereby providing customers with more abundant business processing capabilities. [0003] On a smart box integrated with multiple NPUs (also known as an edge computing box), in order to achieve the best running effect, multiple algorithm models in an algorithm solution need to be evenly distributed on different NPUs. In addition, when the algorithm models of more than two algorit...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/50G06N3/063
CPCG06F9/5027G06N3/063
Inventor 张发恩林国森
Owner 创新奇智(重庆)科技有限公司