A method, device, storage medium, and equipment for deploying a model to a hardware platform

Abstract

This specification discloses a method, apparatus, storage medium, and device for deploying a model to a hardware platform. The method involves determining the network structure search space of the model to be deployed, identifying the target structure constituting the model based on preset performance evaluations of each candidate structure in the network structure search space, deploying the model to be deployed using the target structure on the target hardware platform, using the model to be deployed to calculate sample data, obtaining calculation results, determining a loss value using a preset loss function based on the difference between the calculation results and the labels corresponding to the sample data, and the expected computational performance of the model to be deployed in the network structure search space, and adjusting the structure of the model to be deployed based on the loss value until the computational performance of the model meets preset performance conditions. The structure of the model to be deployed can be adjusted according to the performance requirements of the model to be deployed and the hardware performance of the target hardware platform.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, apparatus, storage medium, and device for deploying a model to a hardware platform. Background Art

[0002] In the embedded application scenarios of AI algorithms, deep learning algorithms emerge in an endless stream and embedded hardware resources are frequently updated. In actual application scenarios, deep learning algorithms need to deploy neural network models to corresponding hardware platforms before they can carry out model training tasks. In order to improve the efficiency of training, improving the compatibility of deep learning algorithms and hardware platforms has become one of the key areas of concern for current researchers.

[0003] At present, a relatively sound prior knowledge base has been formed for the shallow connection between model characteristics and hardware characteristics, such as the requirements of each operator for computing power and storage space. However, there is still a lack of feasible solutions for the deeper problem of optimizing the model structure according to the hardware characteristics of the hardware platform.

[0004] Therefore, the present invention provides a method, apparatus, storage medium, and device for deploying a model to a hardware platform. Summary of the invention

[0005] This specification provides a method and device for deploying a model to a hardware platform to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a method for deploying a model to a hardware platform, including:

[0008] Determine the network structure search space of the model to be deployed;

[0009] Determining a target structure constituting the model to be deployed from among the candidate structures according to a preset performance evaluation of each candidate structure in the network structure search space;

[0010] Deploy the to-be-deployed model consisting of the target structure on a target hardware platform;

[0011] Calculating the sample data using the model to be deployed to obtain a calculation result;

[0012] Determine a loss value using a preset loss function according to a difference between the calculation result and the label corresponding to the sample data and the expected calculation performance of the model to be deployed in the network structure search space;

[0013] The structure of the model to be deployed is adjusted according to the loss value until the computing performance of the model to be deployed meets the preset performance conditions, and the model to be deployed is deployed on the target hardware platform.

[0014] Optionally, adjusting the structure of the model to be deployed according to the loss value until the computing performance of the model to be deployed meets a preset performance condition, and deploying the model to be deployed on the target hardware platform specifically includes:

[0015] Updating the performance evaluation of the target structure in the network structure search space according to the loss value;

[0016] If the update amplitude of the performance evaluation of the target structure is greater than the preset update threshold, the target structure is re-determined in the network structure search space, and after the model to be deployed consisting of the updated target structure is deployed on the target hardware platform, the performance evaluation of the target structure is continued to be calculated and updated until the update amplitude of the performance evaluation of the target structure is no greater than the preset update threshold, thereby completing the deployment of the model to be deployed to the target hardware platform.

[0017] Optionally, according to a preset performance evaluation of each candidate structure in the network structure search space, determining a target structure constituting the model to be deployed from each candidate structure specifically includes:

[0018] Determining the probability of each candidate structure being selected according to a preset performance evaluation of each candidate structure in the network structure search space;

[0019] According to the probability of being selected, a target structure constituting the model to be deployed is determined from among the candidate structures.

[0020] Optionally, after determining the loss value, it also includes:

[0021] The parameters of each operator in the target structure are updated according to the loss value.

[0022] Optionally, the expected computing performance specifically includes: at least one of expected latency, expected storage consumption, and expected computing power consumption.

[0023] Optionally, preset loss functions, including:

[0024] Establish a loss function that determines the loss value based on the difference between the calculation result and the label corresponding to the sample data, the expected delay, the expected storage consumption and the expected computing power consumption, wherein the smaller the hardware delay difference between the hardware devices in the target hardware platform, the greater the weight of the expected delay; the smaller the storage resource difference between the hardware devices in the target hardware platform, the greater the weight of the expected storage consumption; the smaller the computing power resource difference between the hardware devices in the target hardware platform, the greater the weight of the expected computing power consumption.

[0025] Optionally, before deploying the to-be-deployed model of the target structure on the target hardware platform, the method further includes:

[0026] Determining the compatibility of the model to be deployed and the hardware platform to be selected according to the required performance of the model to be deployed and the hardware characteristics of the hardware platform to be selected;

[0027] When the degree of adaptation is higher than a preset adaptation threshold, the to-be-selected hardware platform is used as a target hardware platform.

[0028] This specification provides a device for deploying a model to a hardware platform, including:

[0029] Determine the module and the network structure search space of the model to be deployed;

[0030] A search module, which determines a target structure constituting the model to be deployed from among the candidate structures according to a preset performance evaluation of each candidate structure in the network structure search space;

[0031] A deployment module deploys the to-be-deployed model consisting of the target structure on a target hardware platform;

[0032] A calculation module, which uses the model to be deployed to calculate the sample data to obtain a calculation result;

[0033] A loss module, which determines a loss value using a preset loss function according to a difference between the calculation result and the label corresponding to the sample data and the expected computing performance of the model to be deployed in the network structure search space;

[0034] The adjustment module adjusts the structure of the model to be deployed according to the loss value until the computing performance of the model to be deployed meets the preset performance conditions, and deploys the model to be deployed on the target hardware platform.

[0035] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for deploying a model to a hardware platform is implemented.

[0036] This specification provides a device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method of deploying a model to a hardware platform when executing the program.

[0037] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0038] It can be seen from the above method that this method can adjust the structure of the model to be deployed according to the performance requirements of the model to be deployed and the hardware performance of the target hardware platform, and optimize the structure of the model to be deployed in the process of deploying the model to be deployed to the target hardware platform, so that the model to be deployed can achieve better computing effect after being deployed on the target hardware platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0040] Figure 1 A flowchart of a method for deploying a model to a hardware platform in this specification;

[0041] Figure 2 A schematic diagram of a process for adjusting the structure of a model to be deployed according to a loss value in this specification;

[0042] Figure 3 This is a flow chart of determining the target hardware platform according to the adaptability in this specification;

[0043] Figure 4 A schematic diagram of a device for deploying a model to a hardware platform provided in this specification;

[0044] Figure 5 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0047] Figure 1 The following is a flow chart of a method for deploying a model to a hardware platform in this specification. The method for deploying a model to a hardware platform specifically includes the following steps:

[0048] S100: Determine the network structure search space of the model to be deployed.

[0049] In order to improve the compatibility between the model to be deployed and the hardware platform, before deploying the model to be deployed to the hardware platform, the network structure search space of the model to be deployed can be determined in advance.

[0050] Specifically, the user may predetermine the scale and architecture of the model to be deployed, and determine the network structure search space of the model to be deployed according to the scale and architecture of the model to be deployed and the computing power of the target hardware platform.

[0051] Among them, the process of determining the network structure search space can be to collect existing typical lightweight network structures in advance, encapsulate the reusable substructures of each model layer in the above typical lightweight network structure, obtain each initial operator, and expand each initial operator to obtain several candidate operators with different input parameters corresponding to an initial operator, and use each candidate operator to establish a general operator library.

[0052] Subsequently, the number of model layers and the types of model layers in the network structure search space can be determined based on the predetermined architecture of the model to be deployed, and based on the computing performance of a single hardware in the target hardware platform and the scale of the parameters of the predetermined model to be deployed, several operator combinations can be set for each model layer, and possible connection methods can be set for each model layer, and each model layer can be connected to obtain the network structure search space of the model to be deployed.

[0053] Among them, the types of model layers may include convolutional layers, pooling layers, activation layers, etc.

[0054] In one or more embodiments of the present specification, the model to be deployed is a multi-source information navigation and positioning model, and the vehicle can be positioned according to each information source in the vehicle equipped with the multi-source information navigation and positioning model, wherein the above-mentioned information sources may include inertial navigation information, image matching positioning information and configuration navigation and positioning information. The data of the above-mentioned information sources are input into the multi-source information navigation and positioning model, and the positioning information of the vehicle can be obtained in real time.

[0055] S102: According to a preset performance evaluation of each candidate structure in the network structure search space, determine a target structure constituting the model to be deployed from each candidate structure.

[0056] After generating the network structure search space, the performance evaluation corresponding to each path in the network structure search space can be randomly generated, wherein a path represents a candidate structure of the model to be deployed composed of various candidate operators, and the candidate structure with better performance evaluation is determined as the target structure of the model to be deployed according to the performance evaluation of each candidate structure, and the candidate operator in the target structure is the target operator. Among them, the performance evaluation represents the computing performance of the model to be deployed of a candidate structure on the target hardware platform, which can be expressed in various forms such as scoring and grading, and this manual does not limit it here.

[0057] According to the performance evaluation of each candidate structure, the probability of each candidate structure being selected can be obtained. The higher the performance evaluation of a candidate structure, the higher the corresponding probability of being selected of the candidate structure. The sum of the probabilities of being selected of each candidate structure is 1. Therefore, according to the probability of being selected of each candidate structure in the network structure search space, the target structure constituting the model to be deployed can be determined from each candidate structure.

[0058] In one or more embodiments of the present specification, a preset performance evaluation of each candidate structure may be randomly generated.

[0059] It should be noted that after generating the network structure search space, since the performance evaluation is randomly generated, without adjustment, the initial performance evaluation of a candidate structure does not truly reflect the performance of the candidate structure's to-be-deployed model after deployment.

[0060] S104: deploying the to-be-deployed model of the target structure on a target hardware platform.

[0061] After the target structure is determined, each target operator in the target structure can be deployed in the target hardware platform to form the model to be deployed.

[0062] Specifically, when generating the initial operator and obtaining each candidate operator based on the initial operator, the computing performance requirements of each initial operator and candidate operator can be determined and recorded based on the input parameters and operation type of each candidate operator or initial operator. Therefore, when deploying each target operator in the target hardware platform, the hardware device that matches each target operator can be determined in each hardware device of the target hardware platform based on the computing requirements of each target operator and the hardware characteristics of each hardware device, so as to deploy the target operator.

[0063] The computing requirements of the target operator may include requirements for storage resources, requirements for computing power resources, requirements for communication speed, etc.

[0064] S106: Calculate the sample data using the model to be deployed to obtain a calculation result.

[0065] When encapsulating the reusable substructures of each model layer in step S100, the interface and interface calling module of each initial operator can be determined according to the type of model layer corresponding to each initial operator, such as convolution layer, pooling layer, activation layer, etc.

[0066] Therefore, after the deployment of the target operators is completed, each target operator can be connected through its own interface and interface call module to form a complete model to be deployed. By using each target operator to calculate the sample data, the calculation result of the model output can be obtained.

[0067] S108: Determine a loss value using a preset loss function according to a difference between the calculation result and the label corresponding to the sample data and an expected calculation performance of the model to be deployed in the network structure search space.

[0068] After obtaining the calculation results of the model output, the preset loss function can be used to substitute the calculation results, the labels corresponding to the sample data of each operator, and the expected computing performance of the model to be deployed in the network structure search space to determine the loss value.

[0069] In one or more embodiments of the present specification, the weight of each candidate structure is determined based on the performance evaluation of each candidate structure in the network structure search space, and the expected computing performance of the model to be deployed in the network structure search space is determined based on the computing performance of the model to be deployed of each candidate structure after deployment and the weight corresponding to each candidate structure.

[0070] Specifically, the computing performance corresponding to the candidate structure with the best performance evaluation can be determined according to the performance evaluation of each candidate structure in the network structure search space, as the expected computing performance under the network structure search space; the weight value of each candidate structure can also be determined according to the performance evaluation of each candidate structure in the network structure search space, and the computing performance of each candidate structure and the weight value corresponding to each candidate structure can be weighted to obtain the expected performance under the network structure search space.

[0071] A random number generation algorithm can be used to assign a true value to each candidate structure as an initial performance evaluation, and then a softmax function is used to convert the randomly generated true value into a decimal between 0 and 1, representing the weight of each candidate structure:

[0072]

[0073] in, and They respectively represent the true values ​​assigned by the random number function to the i-th and j-th candidate structures. represents the weight of the i-th candidate structure after calculation, It also represents the network structure parameters.

[0074] In one or more embodiments of the present specification, the computing performance of each candidate operator after deployment may include at least one of the expected latency, expected storage consumption, and expected computing power consumption of the candidate operator. Correspondingly, the expected computing performance of the network structure search space may include at least one of the expected latency, expected storage consumption, and expected computing power consumption of the network structure search space.

[0075] S110: Adjust the structure of the model to be deployed according to the loss value until the computing performance of the model to be deployed meets the preset performance conditions, and deploy the model to be deployed on the target hardware platform.

[0076] After determining the loss value, the performance evaluation of the target structure in the network structure search space can be adjusted according to the loss value. Thus, when the target structure is redetermined according to the network structure search space, the adjusted structure of the model to be deployed can be obtained. Thus, the process of deploying the model - calculating - adjusting the structure of the model to be deployed is repeated, and the structure of the model to be deployed that meets the preset performance conditions can be obtained.

[0077] like Figure 1 A method for deploying a model to a hardware platform is shown, which can adjust the structure of the model to be deployed according to the performance requirements of the model to be deployed and the hardware performance of the target hardware platform, and optimize the structure of the model to be deployed in the process of deploying the model to be deployed to the target hardware platform, so that the model to be deployed can achieve better computing effect after being deployed on the target hardware platform.

[0078] In addition, in Figure 1 In the step S110 shown, the performance evaluation of the target structure in the network structure search space is updated according to the loss value. If the update amplitude of the performance evaluation of the target structure is greater than the preset update threshold, the target structure is re-determined in the network structure search space, and the model to be deployed consisting of the updated target structure is deployed on the target hardware platform. The performance evaluation of the target structure is then continued to be calculated and updated until the update amplitude of the performance evaluation of the target structure is no greater than the preset update threshold, thereby completing the deployment of the model to be deployed to the target hardware platform.

[0079] like Figure 2 As shown, after the loss value is determined, step S200 can be executed to update the performance evaluation of the target structure according to the gradient of the loss value relative to the target structure.

[0080] After the performance evaluation of each target operator is updated, step S202 is executed to determine the update amplitude of the performance evaluation of this target structure. If the update amplitude is greater than the preset update threshold, it means that the network structure search space is still not stable, and the structure of the model to be deployed cannot be determined based on this - which operators the model to be deployed is composed of. Steps S102-S108 are continued to be executed to re-structure the target structure in the updated network structure search space, deploy the updated model to be deployed of the target structure on the target hardware platform, and continue to calculate the sample data using the deployed model to be deployed. The loss value is re-determined based on the difference between the updated calculation result and the label corresponding to the sample data and the expected computing performance of the model to be deployed under the network structure search space, and the preset loss function.

[0081] It should be noted that, due to changes in the performance evaluation of the target structure, the expected computing performance of the model to be deployed in the network structure search space will also change.

[0082] Therefore, steps S200, S202, S102-S108 are executed in a loop, and the performance evaluation of the target structure in the network structure search space is continuously updated according to the re-determined loss value until the update amplitude of the performance evaluation is no greater than the preset update threshold. At this time, it is considered that the network structure search space is stable, and the target operators in the current target structure are the operators that ultimately constitute the model to be deployed, and step S204 is executed to complete the deployment of the model to be deployed in the target hardware platform.

[0083] In addition, in Figure 1 In step S102 shown, the probability of each candidate structure being selected is determined according to the preset performance evaluation of each candidate structure in the network structure search space, and the target structure constituting the model to be deployed is determined among the candidate structures according to the probability of being selected.

[0084] Specifically, the weight of each candidate structure in the above step S108 can be used to take the weight corresponding to a candidate structure as the probability of being selected corresponding to the candidate structure.

[0085] In addition, in Figure 1 After step S108 shown, the parameters of each target operator in the target structure are updated according to the loss value.

[0086] After obtaining the calculation results corresponding to the sample data, the parameters of each target operator can also be updated. Thus, the target operator after parameter update can extract certain feature information in the sample data. The "difference between the calculation results and the labels corresponding to the sample data" in the loss function can have a stronger guiding significance. The model to be deployed obtained according to the network structure search space can also have a stronger processing capability for the task type corresponding to the sample data.

[0087] In this way, the training process of the model to be deployed and the deployment process of the model to be deployed can be collaboratively optimized, so that the model to be deployed that is finally deployed has stronger processing capabilities for the task types corresponding to the sample data.

[0088] In one or more embodiments of the present specification, the expected computing performance specifically includes: at least one of expected latency, expected storage consumption, and expected computing power consumption.

[0089] Due to the discontinuous gradient characteristics of hardware resource characteristic indicators, the resource characteristic indicators of the hardware platform cannot be directly introduced as guidance during the model training process. The method provided in this specification makes a differentiable representation of the expected computing performance, which is the most concerned composite hardware resource characteristics in the engineering project, thereby indirectly introducing the hardware resource characteristic indicators into the model training process, enhancing the correlation between the model to be deployed and the target hardware platform, and also introducing the hardware resource characteristic indicators to provide support for the inference acceleration of the model to be deployed.

[0090] Specifically, the expected delay can be determined according to the following formula:

[0091]

[0092] in, Represents the expected delay of the subdivision operator module in the model network; Represents the candidate operator of the i-th model layer under the j-th path in the model, that is, the j-th candidate structure; Represents candidate operators The probability of being selected; Represents candidate operators The inference delay is composed of the sending delay and the propagation delay, where the sending delay = data frame length / sending rate, and the propagation delay = channel length / the propagation speed of electromagnetic waves on the channel. The data frame length is determined according to the structure of each candidate operator, and the sending rate and channel length are the average parameters of each hardware in the target hardware platform.

[0093] It should be noted that It can be determined based on the probability of each candidate structure being selected.

[0094] The expected storage consumption can be determined according to the following formula:

[0095]

[0096]

[0097] During inference, the storage consumption of operators in the model It consists of three parts: the input data storage overhead of the operator , the storage cost of the output result and operator parameter storage overhead , can be determined according to the structure of each candidate operator. So by analogy with the expected delay, we can get the expected storage consumption of the network structure search space .

[0098] The expected computing power consumption can be determined according to the following formula:

[0099]

[0100] The computing power requirement for each candidate operator can be determined based on the structure of each candidate operator. By analogy with the expected latency, the expected storage consumption of the network structure search space is obtained. .

[0101] In one or more embodiments of the present specification, when presetting the loss function, a loss function is established to determine the loss value based on the difference corresponding to the calculation results, the expected delay, the expected storage consumption, and the expected computing power consumption, wherein the smaller the difference in hardware delay of each hardware device in the target hardware platform, the greater the weight of the expected delay; the smaller the difference in storage resources of each hardware device in the target hardware platform, the greater the weight of the expected storage consumption; and the smaller the difference in computing power resources of each hardware device in the target hardware platform, the greater the weight of the expected computing power consumption.

[0102] Specifically, the loss function can be The build is as follows:

[0103]

[0104]

[0105] in, represents the cross entropy loss function, represents the weight decay loss, , , and Represents the weight decay loss and the weight parameters of each expected computing performance indicator, which can be set by the user. represents the true probability distribution of the labels of the sample data, Represents the predicted probability distribution of the label of the sample data. Since the loss function Each sub-item in the formula is continuously differentiable and can be read by looking up the table. So far, it is also differentiable after weighting. Therefore, when using the loss function When updating the parameters of each candidate operator and the performance evaluation of the target structure, the continuity of the updated gradient can be guaranteed, ensuring the normal progress of the updating process.

[0106] Furthermore, in order to balance the guiding ability of each expected computing performance indicator, the loss function middle , , and Optimize and transform the network model structure optimization problem into a multi-objective optimization problem, and set the user-adjustable parameter P:

[0107]

[0108] The loss function can be changed by dynamically adjusting P The model accuracy and hardware inference latency can guide the optimization of the network structure, so that the optimized model can adapt to the actual inference latency and accuracy requirements of different application scenarios. Specifically, when the value of parameter P increases, it means that the guidance ability of inference latency is enhanced, and the parameter size and inference latency of the model tend to decrease sharply, while the corresponding model accuracy can remain relatively stable within a certain range of P value changes, and the decline is relatively slow.

[0109] This specification defines the hardware characteristics of all hardware devices in the target hardware platform as , . (i=2,3,4) represents the sending rate corresponding to the expected latency in the expected computing performance, the hardware storage space corresponding to the expected storage consumption, and the hardware computing power unit corresponding to the expected computing power consumption (specifically, it can be TOPS (Tera Operations Per Second, the number of operations that the processor can perform per second) or TFLOPS (TeraFloating Point Operations Per Second, the number of floating-point operations that the processor can perform per second) and other common computing power units, which are not limited in this manual). As for the target hardware platform, the hardware platform includes several hardware devices, and each hardware device is often not of the same model, so There are often differences in the values. The value range of is not uniform, so it is difficult to effectively introduce hardware features by directly assigning weights to each expected computing performance indicator. In order to reduce the impact of data differences between different indicators on the final optimization, each hardware indicator is normalized in this manual.

[0110]

[0111] in, represents the normalized result. The max() and min() in the formula refer to the values ​​of each hardware device in the target platform. The maximum and minimum values ​​of hardware characteristics. This operation can classify the expected dimensions of hardware characteristic indicators such as computing capacity and storage into a unified level. Furthermore, the weight distribution value of each expected computing performance is obtained. :

[0112]

[0113]

[0114]

[0115] Based on the outstanding advantages of information entropy in quantitatively measuring distribution characteristics, especially in uniformity, diversity and convergence, this manual uses information entropy to solve decision conflicts between multiple objectives, and automatically assigns weights to each expected computing performance, and solves the information entropy E of each hardware indicator. i and the weights of each expected computational performance.

[0116] In addition, in Figure 1 In step S100 shown, for any model layer in the model to be deployed, the model layer is split into candidate operators executed serially or in parallel according to the parameter scale of the model layer and the hardware characteristics of each hardware device in the target hardware platform.

[0117] The network structure search space is composed of candidate operators, and each model layer of the model to be deployed can be split according to the model to be deployed. For example, different blocks are used to schedule the convolution layer to the operator that performs the convolution calculation, or the loop expansion is used to schedule multiple parallel calculation operators at the same time. Specifically, the splitting method can be set by the user according to the hardware characteristics of the hardware devices in the target hardware platform and the scale of the model to be deployed. The user can set the search parameters, which may include the block size of the loop block, the parallelism of the loop expansion, etc.

[0118] In one or more embodiments of this specification, in the Figure 1Before step S100 shown, the compatibility between the model to be deployed and the hardware platform to be selected can also be determined based on the required performance of the model to be deployed and the hardware characteristics of the hardware platform to be selected. When the compatibility is higher than a preset adaptation threshold, the hardware platform to be selected is used as the target hardware platform.

[0119] As mentioned above, when determining the network structure space of the model to be deployed, the hardware characteristics of the target hardware platform can be referred to. Therefore, before determining the network structure search space of the model to be deployed, the target hardware platform needs to be determined. Specifically, the adaptability of the model to be deployed and the selected hardware platform can be determined based on the latency adaptability and storage adaptability:

[0120]

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127] Indicates the compatibility between the model to be deployed and the hardware platform to be selected. and represents the weight, Indicates the latency adaptability between the model to be deployed and the hardware platform to be selected. represents the minimum theoretical delay of model inference, Indicates the maximum delay threshold required by the engineering indicator. represents the penalty factor for the prior reasoning delay ratio, express The maximum value in represents the minimum delay of the system, represents the minimum storage delay of the model, Indicates the total amount of data to be processed calculated by the model resource demand. Indicates the storage unit capacity of the hardware device. Represents the system clock period, Represents the minimum computational delay of the model, Indicates the basic operation category, Indicates the number of operations corresponding to the basic operation category. represents the minimum communication delay of the model, Indicates the total amount of data to be processed calculated by the model resource demand. Indicates the fastest memory access rate of the storage unit. Indicates the storage resource compatibility index between the model and the hardware device. Indicates the storage unit capacity required by the model, Indicates the storage unit capacity of the hardware device. Indicates the penalty factor of the a priori storage resource ratio.

[0128] Specifically, the process of determining the target hardware platform based on the adaptability is as follows: Figure 3 As shown, first, according to the required performance of the model to be deployed and the hardware characteristics of the hardware platform to be selected, the minimum delay of communication bandwidth, the minimum delay of operation and the minimum delay of memory scheduling are calculated respectively, and the minimum theoretical delay of model reasoning is obtained by calculating the maximum value of the minimum delay of communication bandwidth, the minimum delay of operation and the minimum delay of memory scheduling and the sum of the minimum system delay. If the minimum theoretical delay of model reasoning is not less than the preset project delay index, the fitness of the hardware platform to be selected and the model to be deployed is directly determined to be 0; on the other hand, it is determined whether the total model storage overhead is less than the preset target hardware total storage. If not, similarly, the fitness of the hardware platform to be selected and the model to be deployed is directly determined to be 0; when the total model storage overhead is less than the preset target hardware total storage and the minimum theoretical delay of model reasoning is less than the preset project delay index, the fitness of the hardware platform to be selected and the model to be deployed is determined according to the above formula, and whether the hardware platform to be selected and the model to be deployed are compatible is determined according to the determined fitness and the preset fitness threshold.

[0129] Therefore, before deploying the model to be deployed to the target hardware platform, it can be determined whether the model to be deployed can be deployed to the target hardware platform without losing performance, providing guidance for model deployment.

[0130] The above is a method for deploying a model to a hardware platform provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for deploying a model to a hardware platform, such as Figure 4 shown.

[0131] Figure 4 A schematic diagram of a device for deploying a model to a hardware platform provided in this specification specifically includes:

[0132] Determine module 400, determine the network structure search space of the model to be deployed;

[0133] A search module 402, which determines a target structure constituting the model to be deployed from among the candidate structures according to a preset performance evaluation of each candidate structure in the network structure search space;

[0134] A deployment module 404 deploys the to-be-deployed model consisting of the target structure on a target hardware platform;

[0135] A calculation module 406 calculates the sample data using the model to be deployed to obtain a calculation result;

[0136] The loss module 408 determines a loss value using a preset loss function according to the difference between the calculation result and the label corresponding to the sample data and the expected calculation performance of the model to be deployed in the network structure search space;

[0137] The adjustment module 410 adjusts the structure of the model to be deployed according to the loss value until the computing performance of the model to be deployed meets the preset performance conditions, and deploys the model to be deployed on the target hardware platform.

[0138] Optionally, the adjustment module 410 is specifically used to: update the performance evaluation of the target structure in the network structure search space according to the loss value; if the update amplitude of the performance evaluation of the target structure is greater than a preset update threshold, redetermine the target structure in the network structure search space, deploy the model to be deployed consisting of the updated target structure on the target hardware platform, and continue to calculate and update the performance evaluation of the target structure until the update amplitude of the performance evaluation of the target structure is no greater than the preset update threshold, thereby completing the deployment of the model to be deployed to the target hardware platform.

[0139] Optionally, the search module 402 is specifically used to: determine the probability of each candidate structure being selected according to a preset performance evaluation of each candidate structure in the network structure search space, and determine the target structure constituting the model to be deployed from each candidate structure according to the probability of being selected.

[0140] Optionally, the adjustment module 410 is further used to: update the parameters of each operator in the target structure according to the loss value.

[0141] Optionally, the expected computing performance specifically includes: at least one of expected latency, expected storage consumption, and expected computing power consumption.

[0142] Optionally, the loss module 408 is also used to: establish a loss function that determines the loss value based on the difference between the calculation result and the label corresponding to the sample data, the expected delay, the expected storage consumption and the expected computing power consumption, wherein the smaller the hardware delay difference between the hardware devices in the target hardware platform, the greater the weight of the expected delay; the smaller the storage resource difference between the hardware devices in the target hardware platform, the greater the weight of the expected storage consumption; and the smaller the computing power resource difference between the hardware devices in the target hardware platform, the greater the weight of the expected computing power consumption.

[0143] Optionally, the deployment module 404 is also used to: determine the compatibility of the model to be deployed and the hardware platform to be selected based on the required performance of the model to be deployed and the hardware characteristics of the hardware platform to be selected; when the compatibility is higher than a preset adaptation threshold, use the hardware platform to be selected as the target hardware platform.

[0144] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides methods for deploying models to hardware platforms.

[0145] This manual also provides Figure 5 The schematic structure diagram of the electronic device shown in FIG. Figure 5 As mentioned above, at the hardware level, the equipment for deploying the model to the hardware platform includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The method of deploying the model to the hardware platform. Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0146] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0147] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0148] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0149] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0150] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0155] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0156] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0157] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0158] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0160] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0161] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A method for deploying a model to a hardware platform, characterized in that: The method comprises: Determine a network structure search space of a model to be deployed, wherein the network structure search space is obtained by connecting each model layer by determining the number of model layers and the type of model layers in the network structure search space according to a predetermined architecture of the model to be deployed, and by setting a number of operator combinations for each model layer according to the computing performance of a single hardware in a target hardware platform and the predetermined scale of parameters of the model to be deployed; and setting an implementable connection method for each model layer; Determining a target structure constituting the model to be deployed from among the candidate structures according to a preset performance evaluation of each candidate structure in the network structure search space; According to the required performance of the model to be deployed and the hardware characteristics of the hardware platform to be selected, determine the compatibility of the model to be deployed and the hardware platform to be selected, wherein the compatibility is determined according to the delay adaptability and the storage adaptability, and the delay adaptability is determined by the minimum communication bandwidth delay, the minimum operation delay and the minimum memory scheduling delay calculated according to the required performance of the model to be deployed and the hardware characteristics of the hardware platform to be selected, and when the compatibility is higher than a preset adaptation threshold, the hardware platform to be selected is used as the target hardware platform; Deploy the to-be-deployed model of the target structure on a target hardware platform; Calculating the sample data using the model to be deployed to obtain a calculation result; Determine a loss value using a preset loss function according to a difference between the calculation result and the label corresponding to the sample data and the expected calculation performance of the model to be deployed in the network structure search space; The structure of the model to be deployed is adjusted according to the loss value until the computing performance of the model to be deployed meets the preset performance conditions, and the model to be deployed is deployed on the target hardware platform.

2. The method according to claim 1, characterized in that Adjusting the structure of the model to be deployed according to the loss value until the computing performance of the model to be deployed meets the preset performance conditions, and deploying the model to be deployed on the target hardware platform, specifically includes: Updating the performance evaluation of the target structure in the network structure search space according to the loss value; If the update amplitude of the performance evaluation of the target structure is greater than the preset update threshold, the target structure is re-determined in the network structure search space, and after the model to be deployed consisting of the updated target structure is deployed on the target hardware platform, the performance evaluation of the target structure is continued to be calculated and updated until the update amplitude of the performance evaluation of the target structure is no greater than the preset update threshold, thereby completing the deployment of the model to be deployed to the target hardware platform.

3. The method according to claim 1, characterized in that According to the preset performance evaluation of each candidate structure in the network structure search space, determining the target structure constituting the model to be deployed from each candidate structure specifically includes: Determining the probability of each candidate structure being selected according to a preset performance evaluation of each candidate structure in the network structure search space; According to the probability of being selected, a target structure constituting the model to be deployed is determined from among the candidate structures.

4. The method according to claim 1, characterized in that After determining the loss value, it also includes: The parameters of each target operator in the target structure are updated according to the loss value.

5. The method according to claim 1, characterized in that The expected computing performance specifically includes: at least one of expected latency, expected storage consumption, and expected computing power consumption.

6. The method according to claim 5, characterized in that Preset loss function, including: Establish a loss function that determines the loss value based on the difference between the calculation result and the label corresponding to the sample data, the expected delay, the expected storage consumption and the expected computing power consumption, wherein the smaller the hardware delay difference between the hardware devices in the target hardware platform, the greater the weight of the expected delay; the smaller the storage resource difference between the hardware devices in the target hardware platform, the greater the weight of the expected storage consumption; the smaller the computing power resource difference between the hardware devices in the target hardware platform, the greater the weight of the expected computing power consumption.

7. A device for deploying a model to a hardware platform, characterized in that: include: A determination module is used to determine a network structure search space of a model to be deployed, wherein the network structure search space is obtained by connecting each model layer by determining the number of model layers and the type of model layers in the network structure search space according to a predetermined architecture of the model to be deployed, and by setting a plurality of operator combinations for each model layer according to the computing performance of a single hardware in a target hardware platform and the predetermined scale of parameters of the model to be deployed; and setting an implementable connection mode for each model layer; A search module, which determines a target structure constituting the model to be deployed from among the candidate structures according to a preset performance evaluation of each candidate structure in the network structure search space; A deployment module determines the compatibility of the model to be deployed and the hardware platform to be selected according to the required performance of the model to be deployed and the hardware characteristics of the hardware platform to be selected, wherein the compatibility is determined according to the delay adaptability and the storage adaptability, and the delay adaptability is determined according to the minimum communication bandwidth delay, the minimum operation delay and the minimum memory scheduling delay calculated according to the required performance of the model to be deployed and the hardware characteristics of the hardware platform to be selected. When the compatibility is higher than a preset adaptation threshold, the hardware platform to be selected is used as the target hardware platform; the model to be deployed consisting of the target structure is deployed on the target hardware platform; A calculation module, which uses the model to be deployed to calculate the sample data to obtain a calculation result; A loss module, which determines a loss value using a preset loss function according to a difference between the calculation result and the label corresponding to the sample data and the expected computing performance of the model to be deployed in the network structure search space; The adjustment module adjusts the structure of the model to be deployed according to the loss value until the computing performance of the model to be deployed meets the preset performance conditions, and deploys the model to be deployed on the target hardware platform.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 6 is implemented.

Images