Model deployment method and device, storage medium and electronic equipment

Abstract

The invention discloses a model deployment method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring an initial deployment strategy group for a service model, wherein the initial deployment strategy group comprises two deployment strategies; inputting the feature code of each deployment strategy in the initial deployment strategy group into a pre-trained proxy model, and determining performance distribution information of each deployment strategy on the processing equipment; adjusting at least one item of input by utilizing preset label distribution, and obtaining an adjusted deployment strategy group under the condition that the difference between the output performance distribution information and the label distribution meets a set requirement; and determining a target deployment strategy in the adjusted deployment strategy group, and deploying the service model based on the target deployment strategy. According to the scheme, the time loss for exploring the model deployment strategy is reduced, and the model deployment efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a model deployment method, device, storage medium and electronic device. Background Art

[0002] With the rapid development of deep learning technology, neural network models have achieved remarkable results in many fields. However, how to effectively deploy neural network models in actual production environments faces many challenges. Model deployment not only needs to consider the performance of the model, but also needs to take into account multiple factors such as computing resources, storage space, inference speed, and compatibility of the deployment environment.

[0003] Currently, Neural Architecture Search (NAS) technology is commonly used to explore the model architecture, and Mapping Space Explore (MSE) technology is used to explore the configuration between the model and the processing device, so as to find the optimal model deployment strategy and deploy it.

[0004] However, since the architectural parameters of the model and the configuration parameters between it and the processing device are usually discrete data, in the process of exploring the model architecture and configuration through the above method, a large number of parameter combinations need to be explored to determine different model deployment strategies, and the actual model performance brought by each parameter combination needs to be tested to find the optimal model deployment strategy. This results in the model deployment process not only occupying a large amount of computing power resources, but also taking a long time and being inefficient. Summary of the invention

[0005] This specification provides a model deployment method, device, storage medium and electronic device to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions: A model deployment method, the method comprising: Acquire an initial deployment strategy group for a business model, the initial deployment strategy group including two deployment strategies, each deployment strategy including architecture information of the business model and configuration mapping information between the business model and a configuration of a processing device; Inputting feature codes of each deployment strategy in the initial deployment strategy group into a pre-trained proxy model to determine performance distribution information of each deployment strategy on the processing device; At least one of the inputs is adjusted using a preset label distribution, and when a difference between the output performance distribution information and the label distribution meets a set requirement, an adjusted deployment strategy group is obtained; A target deployment strategy is determined in the adjusted deployment strategy group, and the business model is deployed based on the target deployment strategy.

[0007] Optionally, for each deployment strategy, the architecture information included in the deployment strategy includes: topology information corresponding to the business model and operation type information corresponding to each network node in the business model.

[0008] Optionally, adjusting at least one of the inputs using a preset label distribution specifically includes: With the goal of maximizing the deviation between the performance distribution information output by the proxy model and the label distribution information and making the size relationship represented by the performance distribution information output by the proxy model the same as the small relationship represented by the label distribution, one of the inputs is adjusted.

[0009] Optionally, the two deployment strategies include: a first deployment strategy and a second deployment strategy; the preset label distribution includes: a first label distribution and a second label distribution; wherein the magnitude relationship between the performances of the two deployment strategies on the processing device represented by the first label distribution is opposite to the magnitude relationship between the performances of the two deployment strategies on the processing device represented by the second label distribution; At least one of the inputs is adjusted using a preset label distribution, specifically including: Using a preset label distribution, the first deployment strategy and the second deployment strategy are adjusted for a plurality of adjustment cycles, wherein the label distributions corresponding to two adjacent adjustment cycles are different; For each adjustment period, if the label distribution corresponding to the adjustment period is the first label distribution, adjusting the first deployment strategy in the adjustment period by using the first label distribution until the next adjustment period is reached; In the next adjustment cycle, the second deployment strategy in the next adjustment cycle is adjusted using the second label distribution.

[0010] Optionally, adjusting at least one of the inputs using a preset label distribution specifically includes: Determining probability distribution data corresponding to the at least one input, and determining gradient information of the performance distribution information relative to the initial deployment strategy group; Using the preset label distribution, adjusting the probability distribution data according to the gradient information to obtain adjusted probability distribution data; Based on the adjusted probability distribution data, the at least one input after adjustment is determined.

[0011] Optionally, adjusting at least one of the inputs using a preset label distribution specifically includes: Determining gradient information of the performance distribution information relative to initial configuration mapping information included in the initial deployment strategy group, and determining configuration constraint information corresponding to the initial mapping configuration information; Under the constraints of the configuration constraint information, the initial configuration mapping information is adjusted according to the gradient information using the preset label distribution, and the configuration constraint is updated based on the adjusted mapping configuration information.

[0012] Optionally, training the proxy model specifically includes: Obtain each historical deployment strategy, and construct multiple historical strategy groups based on each historical deployment strategy; wherein each historical strategy group includes two historical deployment strategies; Determine the actual performance of each historical deployment strategy on the processing device, and determine actual performance distribution information corresponding to each historical deployment strategy group according to the actual performance; For each historical strategy group, input the historical strategy group into the proxy model to be trained, so as to determine the prediction performance distribution information corresponding to the historical strategy group through the proxy model; The loss value of the proxy model is determined according to the predicted performance distribution information and the actual performance distribution information corresponding to the historical strategy group, and the proxy model is trained according to the loss value.

[0013] This specification provides a model deployment device, including: An acquisition module is used to acquire an initial deployment strategy group for a business model, wherein the initial deployment strategy group includes two deployment strategies, each of which includes architecture information of the business model and configuration mapping information between the business model and the configuration of a processing device; A determination module, inputting feature codes of each deployment strategy in the initial deployment strategy group into a pre-trained proxy model, and determining performance distribution information of each deployment strategy on the processing device; An adjustment module, adjusting at least one of the inputs using a preset label distribution, and obtaining an adjusted deployment strategy group when a difference between the output performance distribution information and the label distribution meets a set requirement; The deployment module determines a target deployment strategy in the adjusted deployment strategy group and deploys the business model based on the target deployment strategy.

[0014] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned model deployment method is implemented.

[0015] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned model deployment method when executing the program.

[0016] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects: In the model deployment method provided in this specification, an initial deployment strategy group for a business model is obtained, the initial deployment strategy group includes two deployment strategies, each deployment strategy contains architecture information corresponding to the business model deployed by the deployment strategy and configuration mapping information between the business model and the configuration of the processing device; the initial deployment strategy group is input into a pre-trained proxy model to determine performance distribution information between the deployment strategies; at least one deployment strategy in the initial deployment strategy group is adjusted according to the performance distribution information, and the above steps are repeated until a preset termination condition is met to determine a target deployment strategy group; a target deployment strategy is determined in the target deployment strategy group, and the business model is deployed based on the target deployment strategy.

[0017] It can be seen from the above method that when deploying the model, this scheme can use the proxy model to determine the comparison relationship between the performance brought by multiple deployment strategies in a set of deployment strategies, and make several adjustments to the deployment strategies based on the comparison relationship, so as to find the optimal deployment strategy in the final deployment strategy group to deploy the model. The entire process does not require testing the actual model performance brought by each model deployment strategy separately, which effectively reduces the resource demand and time loss in the process of exploring the model deployment strategy, and further improves the efficiency of model deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings: Figure 1 A flowchart of a model deployment method provided in this specification; Figure 2 A schematic diagram of a topological structure of a business model provided in this specification; Figure 3 A schematic diagram of characteristic coding of a type of architecture information provided in this specification; Figure 4 A schematic diagram of the hardware structure of a processing device provided in this specification; Figure 5 A schematic diagram of the model structure of an agent model provided in this specification; Figure 6A schematic diagram of the training process of a performance comparison model provided in this specification; Figure 7 A schematic diagram of a construction process of sample data provided in this specification; Figure 8 A schematic diagram of a process for determining a target deployment strategy provided in this specification; Fig. 9 A schematic diagram of a process for adjusting architecture information provided in this specification; Fig.10 A schematic diagram of a process for adjusting configuration mapping information provided in this specification; Fig.11 A schematic diagram of a projection process provided in this specification; Fig.12 A schematic diagram of a model deployment device provided in this specification; Fig.13 A method corresponding to the Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0020] NAS technology aims to automate the network architecture design process to reduce its dependence on expert experience, and it constitutes an important part of Automatic Deep Learning (AutoDL). Among them, NAS usually involves three aspects: search space, search strategy, and evaluation strategy. Current search strategies usually adopt heuristic search strategies, such as reinforcement learning, evolutionary algorithms, Bayesian deployment, etc.; evaluation strategies usually adopt proxy data sets, proxy training, proxy models, etc. In terms of architecture evaluation, proxy models are currently mostly used for evaluation acceleration.

[0021] For MSE technology, the hardware architecture of the processing device determines how to efficiently perform computationally intensive tasks such as convolutional neural networks (CNNs). The key architectural components of the accelerator include processing elements (PEs), on-chip buffers, and control units. Processing elements are responsible for parallel computing operations, caches are used to store intermediate results and reduce external memory accesses, and control units are responsible for scheduling operations. Effective MSE requires considering how to map the algorithm's computational steps to these hardware resources to minimize execution time, energy consumption, and storage bandwidth.

[0022] However, both mapping space search and neural network architecture search are complex combinatorial deployment problems, and their search spaces are very large. The neural architecture search space is usually discrete, non-convex, and the space scale is extremely large. At the same time, due to the discreteness and constraints of hardware resources, the performance surface of the mapping space is also usually non-convex and discrete. Therefore, the coupling between the neural network architecture parameters and the mapping space parameters will significantly increase the difficulty of space search. Existing black-box deployment (zero-order deployment) methods (such as genetic algorithms, reinforcement learning algorithms, etc.) are inefficient when searching the mapping space, and it is difficult to effectively find the global optimal mapping strategy.

[0023] A surrogate model or agent is a model used to simulate, approximate, or replace a complex, expensive, or infeasible real system or process. Common surrogate models can represent data space as regression models (numerical regression), classification models, or relational models, where relational models can be list-based, binary relational comparison-based, etc. The selection of appropriate surrogate models and methods depends on the specific problem and available data. Surrogate models play an important role in space exploration and deployment, thereby effectively exploring and deploying complex systems and design spaces.

[0024] If the modeling of the differentiable proxy model is regarded as an approximate representation or proxy representation of the original architecture space, then the architecture can be directly deployed based on the proxy model. However, the key difficulty and technical bottleneck are: the architecture space is a discrete space. Even if the differentiable proxy model can realize the original space, the discrete architecture variables still cannot be deployed based on differentiability. The current differentiable architecture search method mainly relies on the construction and training of differentiable super networks, which requires a lot of computing power, video memory resources and time. At the same time, the final performance of its deployment search is also limited.

[0025] Based on this, this specification provides a model deployment method. The following is a detailed description of the technical solutions provided by each embodiment of this specification in conjunction with the accompanying drawings.

[0026] Figure 1This is a flow chart of a model deployment method provided in this specification, which includes the following steps: S101: Acquire an initial deployment strategy group for a business model, wherein the initial deployment strategy group includes two deployment strategies, each of which includes architecture information of the business model and configuration mapping information between the business model and configuration of a processing device.

[0027] In this specification, the execution entity used to implement a model deployment method can be a designated device such as a server. For the sake of ease of description, the following will only take the server as the execution entity as an example to illustrate a model deployment method provided in this specification.

[0028] In practical applications, the model architecture of a business model may generally include several optional parameter combinations, wherein the architecture information corresponding to each parameter combination may include the topological structure information corresponding to the topological structure of the business model (i.e., the connection structure between the network nodes of the business model) and the operation type information corresponding to each network node in the business model (such as convolution (conv), pooling (pool), activation, up / down sampling, input (Input), output (Output), etc.).

[0029] In this specification, the above-mentioned business model can be a neural network model used to execute different businesses, such as graphic recognition model, large language model, risk control model, navigation model, information recommendation model, etc., and of course it can also include neural network models used to execute other businesses. This specification does not make specific limitations on this.

[0030] For ease of understanding, this specification provides a schematic diagram of the topological structure of a business model, such as Figure 2 shown.

[0031] in, Figure 2 (a) shows all the optional topologies of the business model, including the possible connections between each network node. Figure 2 (b) in the figure shows one of the topological structures. The operation types corresponding to each network node include: conv1x1, conv3x3, Max pool3x3, Input and Output.

[0032] In addition, in the process of deploying the service model to the processing device, there may be multiple mapping relationships between the service model and the configuration of the processing device. These mapping relationships are used to indicate how the service model and the network element are configured and executed on the processing device.

[0033] From the above content, it can be concluded that in the process of deploying the business model, the model deployment strategy can be jointly determined by the model architecture (topology structure and operation type) and the mapping relationship between the processing device configuration, that is, for each model deployment strategy, the model deployment strategy can include the architecture information corresponding to the business model deployed by this deployment strategy and the configuration mapping information between the business model and the configuration of the processing device.

[0034] In the actual process of exploring the model deployment strategy, the server can randomly select two model deployment strategies, and build an initial deployment strategy group based on the two model deployment strategies as the initial value for exploring the model deployment strategy.

[0035] Specifically, for each model deployment strategy in the initial deployment strategy group, the server can feature encode the topological structure information of the business model in the model deployment strategy to obtain the coding representation T corresponding to the topological structure information, and encode the operation type information corresponding to each network node of the business model in the model deployment strategy to obtain the feature representation O corresponding to the operation type information.

[0036] For ease of understanding, this specification provides a schematic diagram of characteristic coding of architecture information, such as Figure 3 shown.

[0037] in, Figure 3 (a) in FIG. 2 shows the feature encoding of the operation type corresponding to (a) in FIG. 2. Figure 3 (b) shows the Figure 2 (b) in the figure corresponds to the feature encoding of the topological structure.

[0038] In the process of deploying the service model to the processing device, there may be multiple mapping relationships between the service model and the configuration of the processing device. These mapping relationships are used to indicate how the service model and the network element are configured and executed on the processing device.

[0039] In this specification, the processing device may be a neural processing unit (NPU), and of course, may also be other processing devices such as a CPU, a GPU, etc., and this specification does not make any specific limitations on this.

[0040] Taking the NPU as an example, its hardware structure is as follows: Figure 4 shown.

[0041] Figure 4 A schematic diagram of the hardware structure of a processing device provided in this specification.

[0042] Among them, the key architectural components of NPU include PEs, cache and control unit. The processing unit is responsible for parallel computing operations, the cache is used to store intermediate results and reduce external memory access, and the control unit is responsible for scheduling operations.

[0043] Based on the above structure, the mapping relationship between the service model and the NPU configuration may include: Buffer Allocation: Buffer allocation determines how inputs, outputs, and weights are distributed to on-chip caches to reduce memory bandwidth requirements.

[0044] For example, for an on-chip L2 cache, the number of cache buffers per level is 16, so the parameter examples are: .

[0045] Tiling Size: The tiling strategy defines how the computation is handled in the accelerator cache and determines the granularity of parallel computation.

[0046] For example, with on-chip L2 cache and DRAM as the third-level storage, the 7-layer loop bound of CNN atomic operations is: ,

[0047] The number of tile divisions for each layer of loop bound is the number of storage levels.

[0048] Loop Ordering: defines the order of traversing input data, convolution kernels, and output feature maps, affecting cache hit rate and data reuse.

[0049] For example, with on-chip L2 cache and DRAM as the third-level storage, the 7-layer loop reorder sequence of CNN atomic operations is described as:

[0050] Among them, Q (Output Channels), P (Input Channels), N (Batch Size), C (Height / Width), S (Kernel Height), K (Kernel Width), R (Kernel Depth).

[0051] The server may encode the mapping relationship between the service model and the configuration of the processing device, thereby obtaining a feature code X corresponding to the configuration mapping information.

[0052] Thus, the server can obtain the feature code O corresponding to the topology information, the feature code X corresponding to the operation type information, and the feature code T corresponding to the configuration mapping information. Each combination of [O, T, X] corresponds to a model deployment strategy. The above feature codes can be one-hot coding at the cell level.

[0053] S102: Inputting feature codes of each deployment strategy in the initial deployment strategy group into a pre-trained proxy model to determine performance distribution information of each deployment strategy on the processing device.

[0054] After determining the initial deployment strategy group, the server can input the initial deployment strategy group into the pre-trained proxy model, thereby determining the performance distribution information between the model deployment strategies through the proxy model. The performance distribution information is used to characterize the comparison relationship between the performance of each model deployment strategy on the processing device.

[0055] In the present specification, the above-mentioned proxy model can be a binary proxy model, and the performance distribution information output by it can include the probability distribution corresponding to the performance size of each model deployment strategy in the initial deployment strategy group on the processing device, and the probability distribution represents the comparison relationship between the performance of the business models deployed by the above-mentioned deployment strategies on the processing device.

[0056] It should be noted that the proxy model does not output the specific performance data of the deployment strategy on the processing device. Its role is to compare the performance of the input deployment strategy binary on the processing device.

[0057] For example, for the model deployment strategy in the initial deployment strategy group and model deployment strategies , the performance distribution information output by the performance comparison model can be expressed as: , that is, the performance of B is greater than that of A, and the difference between B and A is / .

[0058] The model structure of the above performance comparison model can be as follows Figure 5 shown.

[0059] Figure 5 A schematic diagram of the model structure of an agent model provided in this specification.

[0060] Among them, the spatial point feature extraction layer can be an end-to-end deep learning model such as LSTM, GRU, MLP, etc., and the proxy model is a differentiable model. The paired spatial point codes (spatial point 1, spatial point 2) are respectively obtained through the feature extraction layer to obtain feature layers, and the feature layers are stacked and input into the prediction branch. The prediction branch outputs a binary vector, which is normalized to obtain a binary probability distribution vector.

[0061] The server can use the Kullback-Leibler (KL) divergence to measure the binary distribution distance of the task according to the binary probability distribution vector and binary relationship label corresponding to each indicator, and finally implement supervised training of the proxy model. For ease of understanding, this manual provides a schematic diagram of the training process of the performance comparison model, such as Figure 6 shown.

[0062] The training process of the performance comparison model may include the following steps: S201: Acquire each historical deployment strategy, and construct multiple historical strategy groups based on each historical deployment strategy; wherein each historical strategy group includes two historical deployment strategies.

[0063] The server can sample small-scale datasets from the collaborative space of neural network architectures and mapping schemes, evaluate validation of spatial points, and produce data pairs for binary relationship training.

[0064] S202: Determine the actual performance of each historical deployment strategy on the processing device, and determine actual performance distribution information corresponding to each historical deployment strategy group according to the actual performance.

[0065] The server can further determine the actual performance of the business model deployed by each historical deployment strategy collected on the processing device, and determine the actual performance distribution information corresponding to each historical deployment strategy group according to the actual performance as the label of the sample data. The sample data construction process is as follows: Figure 7 shown.

[0066] The actual performance distribution information may be determined by comparing information such as hardware-related indicators and model indicators after the business model is deployed through each historical model deployment strategy.

[0067] S203: for each historical strategy group, input the historical strategy group into the proxy model to be trained, so as to determine the prediction performance distribution information corresponding to the historical strategy group through the proxy model; S204: Determine a loss value of the proxy model according to the predicted performance distribution information and actual performance distribution information corresponding to the historical strategy group, and train the proxy model according to the loss value.

[0068] Among them, the above loss value can be determined by the KL loss function.

[0069] S103: Using a preset label distribution to adjust at least one of the inputs, and when the difference between the output performance distribution information and the label distribution meets the set requirements, an adjusted deployment strategy group is obtained.

[0070] In practical applications, the two model deployment strategies in the initial deployment strategy group may include a first deployment strategy and a second deployment strategy. The server may use a preset label distribution to make multiple adjustments to the two inputs to the proxy model, and obtain an adjusted deployment strategy group when the difference between the output performance distribution information and the label distribution meets the set requirements. The set requirements met by the difference between the output performance distribution information and the label distribution may include: the difference between the two reaches a preset difference.

[0071] In an embodiment provided in the present specification, the above-mentioned label distribution may include only one label distribution. In this case, the server may adjust one of the inputs with the goal of maximizing the deviation between the performance distribution information output by the proxy model and the label distribution information and making the size relationship between the two inputs represented by the performance distribution information output by the proxy model the same as the size relationship between the two inputs represented by the label distribution.

[0072] For example, for the tuple [A, B] of the first deployment strategy A and the second deployment strategy B, and the preset label distribution P[0.4, 0.6], the server can use the probability distribution as a reference and make multiple adjustments to the first deployment strategy A while fixing the second deployment strategy B. During this process, the probability of A corresponding to performance is always less than that of B, but as the difference between the two gradually increases, the output performance distribution information With label distribution The differences between them are gradually increasing.

[0073] In another embodiment provided in the present specification, the above-mentioned label distribution may include a label distribution and a second label distribution, and the relationship between the performance of the two deployment strategies represented by the first label distribution on the processing device is opposite to the relationship between the performance of the two deployment strategies represented by the second label distribution on the processing device.

[0074] The server may use the preset label distribution to adjust the first deployment strategy and the second deployment strategy for multiple adjustment cycles, wherein the label distributions corresponding to two adjacent adjustment cycles are different.

[0075] For each adjustment period, if the label distribution corresponding to the adjustment period is the first label distribution, the server can use the first label distribution to adjust the first deployment strategy in the adjustment period until the next adjustment period is reached. In the next adjustment period, the server can use the second label distribution to adjust the second deployment strategy in the next adjustment period.

[0076] When the difference between the performance distribution information and the label distribution meets the set requirements and reaches a preset adjustment period, an adjusted deployment strategy group is obtained.

[0077] For ease of understanding, this specification provides a schematic diagram of the process of determining a target deployment strategy, such as Figure 8 shown.

[0078] Among them, the server can use the proxy model as a mapping operation from features to outputs. Since the gradient information of the output of the entire model to the input is known, it can achieve iterative generation of optimal / dominant input features, and realize architecture search and mapping solution search. Figure 8 As shown in Figure 1, the server can randomly select two model deployment strategies as initial points and encode them as A and B respectively as binary relation model input. Two different label distributions are used for supervised training. and They are respectively expressed as the adjustment cycles of training A and B, which can be designed to be performed alternately and in stages. That is: : The first label distribution P1[0.4,0.6], fix B, iteratively adjust A so that A is less than B; : The second label distribution P2 [0.6, 0.4], that is, fix A and iteratively adjust B so that B is less than A.

[0079] In the later stages of exploration, the server can also be de-encoded, trained, and verified for hardware deployment to obtain the final performance and deployment metrics, and selected according to certain preference strategies.

[0080] In the process of adjusting the model deployment strategy, the server can determine the probability distribution data corresponding to at least one input, and determine the gradient information of the performance distribution information relative to the initial deployment strategy group; use the preset label distribution to adjust the probability distribution data according to the gradient information to obtain the adjusted probability distribution data. Then, based on the adjusted probability distribution data, determine at least one adjusted input (i.e., reconvert the adjusted probability distribution data into discrete feature encoding).

[0081] It should be noted that since the proxy model is a differentiable proxy model, it has learned the mapping relationship from input features to output during the training process. Therefore, after the training is completed, the proxy model can directly provide the gradient information of its output relative to the input.

[0082] Specifically, for the topological structure of a neural network, the connection relationship between nodes in the network is usually a binary choice problem, that is, either there is a connection between nodes or there is no connection. In traditional architecture search, the choice of connection relationship is discrete, so it is difficult to search through gradient deployment. Based on this, the server can parameterize these discrete connection relationships into a continuous probability representation through a reparameterized sampling method. In this way, the selection of topological structure information can determine the possibility of connection by controlling the parameters of the probability representation. Through the deployment search of this parameter, the selection process of the connection relationship can be smoothed, so that deployment can be performed through gradient information during the search process.

[0083] In addition to the topological connection relationship between network nodes, nodes also need to select the optimal operation from multiple candidate operation types, such as convolution kernels of different sizes or different types of pooling operations. The selection of these operations is usually discrete, and traditional architecture search methods are difficult to directly deploy them continuously. To solve this problem, the server can re-parameterize the candidate operation type information. By converting the operation type information corresponding to the network node from discrete options to a probability-based representation, it can be deployed through gradient information during the search process. For ease of understanding, this manual provides a schematic diagram of the process of adjusting the architecture information, such as Fig. 9 shown.

[0084] Among them, for the probability representation corresponding to the architecture information (ie, the continuous space variable (α, β)), the server can update it by reparameterizing the sampling.

[0085] As for the mapping relationship between the business model and the configuration of the processing device, the server can determine the gradient information of the performance distribution information relative to the initial configuration mapping information contained in the initial deployment strategy group, and determine the configuration constraint information corresponding to the initial mapping configuration information; under the constraint of the configuration constraint information, the preset label distribution is used to adjust the initial configuration mapping information according to the gradient information, and the configuration constraint is updated based on the adjusted mapping configuration information. For ease of understanding, this specification provides a schematic diagram of the process of adjusting the configuration mapping information, such as Fig.10 shown.

[0086] The server may explore the configuration mapping information by using a projected gradient method, and the process may include the following steps: Initialization parameters: Choose an initial parameter value, usually within the feasible region; Calculate gradient: Calculate the gradient of the objective function with respect to the parameters; Gradient-based parameter update and projection; The above steps are then repeated until the stopping condition is met, for example: the maximum number of iterations is reached or the gradient is small enough.

[0087] The above projection process is as follows Fig.11 As shown, the rectangular wireframe represents the feasible region of the space parameters: Parameter@(t+1) = parameter@(t) - learning rate * gradient.

[0088] Projecting to the feasible domain ensures that the parameters still meet the constraints after updating the parameters. If the updated parameters do not meet the constraints, it is necessary to change the projection distance limit or change the spatial direction of the projection and re-project until the feasible domain is reached.

[0089] In the process of exploring the deployment strategy, the loss function corresponding to each exploration can also be the KL divergence loss, and the loss value can be expressed as:

[0090] Among them, the loss value represents the distance measure between the binary distribution Q output by the proxy model and the label distribution P.

[0091] S104: Determine a target deployment strategy in the adjusted deployment strategy group, and deploy the business model based on the target deployment strategy.

[0092] After determining the target deployment strategy group, the target deployment strategy group is bound to contain an optimal model deployment strategy. The server can use the model deployment strategy as the target deployment strategy and deploy it to the processing device based on the topology information contained in the target deployment strategy, the operation type information of each network node, and the configuration mapping information between the processing device, so as to execute services based on the business model (such as graphic recognition, information recommendation, anomaly detection, etc.).

[0093] From the above method, it can be seen that the model deployment method provided by this solution uses the proxy model combined with the gradient deployment method, so that the deployment direction has gradient information and the search is efficient; The original landscape was remodeled using a binary relationship agent model to avoid being trapped in a local optimal area during deployment. During the exploration phase, there is basically no need for actual point verification (based on the real environment, backend operating environment), which improves the exploration efficiency and further improves the model deployment efficiency.

[0094] The above are one or more implementation model deployment methods of this specification. Based on the same idea, this specification also provides a corresponding model deployment device, such as Fig.12 shown.

[0095] Fig.12 A schematic diagram of a model deployment device provided for this specification includes: An acquisition module 1201 is used to acquire an initial deployment strategy group for a business model, wherein the initial deployment strategy group includes two deployment strategies, each deployment strategy includes architecture information of the business model and configuration mapping information between the business model and the configuration of the processing device; A determination module 1202 is used to input the feature code of each deployment strategy in the initial deployment strategy group into a pre-trained proxy model to determine the performance distribution information of each deployment strategy on the processing device; An adjustment module 1203, configured to adjust at least one of the inputs using a preset label distribution, and obtain an adjusted deployment strategy group when a difference between the output performance distribution information and the label distribution meets a set requirement; The deployment module 1204 is used to determine a target deployment strategy in the adjusted deployment strategy group, and deploy the business model based on the target deployment strategy.

[0096] Optionally, for each deployment strategy, the architecture information included in the deployment strategy includes: topology information corresponding to the business model and operation type information corresponding to each network node in the business model.

[0097] Optionally, the adjustment module 1203 is specifically used to adjust one of the inputs with the goal of maximizing the deviation between the performance distribution information output by the proxy model and the label distribution information and making the size relationship represented by the performance distribution information output by the proxy model identical to the small relationship represented by the label distribution.

[0098] Optionally, the two deployment strategies include: a first deployment strategy and a second deployment strategy; the preset label distribution includes: a first label distribution and a second label distribution; wherein the magnitude relationship between the performances of the two deployment strategies on the processing device represented by the first label distribution is opposite to the magnitude relationship between the performances of the two deployment strategies on the processing device represented by the second label distribution; The adjustment module 1203 is specifically used to adjust the first deployment strategy and the second deployment strategy for multiple adjustment cycles using a preset label distribution, wherein the label distributions corresponding to two adjacent adjustment cycles are different; for each adjustment cycle, if the label distribution corresponding to the adjustment cycle is the first label distribution, the first deployment strategy in the adjustment cycle is adjusted using the first label distribution until the next adjustment cycle is reached; in the next adjustment cycle, the second deployment strategy in the next adjustment cycle is adjusted using the second label distribution.

[0099] Optionally, the adjustment module 1203 is specifically used to determine the probability distribution data corresponding to the at least one input, and to determine the gradient information of the performance distribution information relative to the initial deployment strategy group; using the preset label distribution, adjust the probability distribution data according to the gradient information to obtain adjusted probability distribution data; based on the adjusted probability distribution data, determine the at least one input after adjustment.

[0100] Optionally, the adjustment module 1203 is specifically used to determine the gradient information of the performance distribution information relative to the initial configuration mapping information contained in the initial deployment strategy group, and determine the configuration constraint information corresponding to the initial mapping configuration information; under the constraint of the configuration constraint information, using the preset label distribution, adjust the initial configuration mapping information according to the gradient information, and update the configuration constraint based on the adjusted mapping configuration information.

[0101] Optionally, the device also includes: a training module 1205, which is used to obtain each historical deployment strategy and construct multiple historical strategy groups based on each historical deployment strategy; wherein each historical strategy group contains two historical deployment strategies; determine the actual performance of each historical deployment strategy on the processing device, and determine the actual performance distribution information corresponding to each historical deployment strategy group based on the actual performance; for each historical strategy group, input the historical strategy group into the proxy model to be trained, so as to determine the predicted performance distribution information corresponding to the historical strategy group through the proxy model; determine the loss value of the proxy model based on the predicted performance distribution information and the actual performance distribution information corresponding to the historical strategy group, and train the proxy model based on the loss value.

[0102] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A model deployment method provided.

[0103] This manual also provides Fig.13 The one shown corresponds to Figure 1 A schematic diagram of the electronic device. Fig.13 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 3 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0104] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0105] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0106] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0107] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0108] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] In a typical configuration, a processing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0113] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0114] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a processing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0116] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0118] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0119] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A model deployment method, characterized in that: The method comprises: Acquire an initial deployment strategy group for a business model, the initial deployment strategy group including two deployment strategies, each deployment strategy including architecture information of the business model and configuration mapping information between the business model and a configuration of a processing device; Inputting feature codes of each deployment strategy in the initial deployment strategy group into a pre-trained proxy model to determine performance distribution information of each deployment strategy on the processing device; At least one of the inputs is adjusted using a preset label distribution, and when a difference between the output performance distribution information and the label distribution meets a set requirement, an adjusted deployment strategy group is obtained; A target deployment strategy is determined in the adjusted deployment strategy group, and the business model is deployed based on the target deployment strategy.

2. The method according to claim 1, characterized in that For each deployment strategy, the architecture information included in the deployment strategy includes: topology information corresponding to the business model and operation type information corresponding to each network node in the business model.

3. The method according to claim 1, characterized in that At least one of the inputs is adjusted using a preset label distribution, specifically including: With the goal of maximizing the deviation between the performance distribution information output by the proxy model and the label distribution information and making the size relationship represented by the performance distribution information output by the proxy model the same as the small relationship represented by the label distribution, one of the inputs is adjusted.

4. The method according to claim 3, characterized in that The two deployment strategies include: a first deployment strategy and a second deployment strategy; the preset label distribution includes: a first label distribution and a second label distribution; wherein the magnitude relationship between the performances of the two deployment strategies on the processing device represented by the first label distribution is opposite to the magnitude relationship between the performances of the two deployment strategies on the processing device represented by the second label distribution; At least one of the inputs is adjusted using a preset label distribution, specifically including: Using a preset label distribution, the first deployment strategy and the second deployment strategy are adjusted for a plurality of adjustment cycles, wherein the label distributions corresponding to two adjacent adjustment cycles are different; For each adjustment period, if the label distribution corresponding to the adjustment period is the first label distribution, adjusting the first deployment strategy in the adjustment period by using the first label distribution until the next adjustment period is reached; In the next adjustment cycle, the second deployment strategy in the next adjustment cycle is adjusted using the second label distribution.

5. The method according to claim 1, characterized in that At least one of the inputs is adjusted using a preset label distribution, specifically including: Determining probability distribution data corresponding to the at least one input, and determining gradient information of the performance distribution information relative to the initial deployment strategy group; Using the preset label distribution, adjusting the probability distribution data according to the gradient information to obtain adjusted probability distribution data; Based on the adjusted probability distribution data, the at least one input after adjustment is determined.

6. The method according to claim 1, characterized in that At least one of the inputs is adjusted using a preset label distribution, specifically including: Determining gradient information of the performance distribution information relative to initial configuration mapping information included in the initial deployment strategy group, and determining configuration constraint information corresponding to the initial mapping configuration information; Under the constraints of the configuration constraint information, the initial configuration mapping information is adjusted according to the gradient information using the preset label distribution, and the configuration constraint is updated based on the adjusted mapping configuration information.

7. The method according to claim 1, characterized in that Training the agent model specifically includes: Obtain each historical deployment strategy, and construct multiple historical strategy groups based on each historical deployment strategy; wherein each historical strategy group includes two historical deployment strategies; Determine the actual performance of each historical deployment strategy on the processing device, and determine actual performance distribution information corresponding to each historical deployment strategy group according to the actual performance; For each historical strategy group, input the historical strategy group into the proxy model to be trained, so as to determine the prediction performance distribution information corresponding to the historical strategy group through the proxy model; The loss value of the proxy model is determined according to the predicted performance distribution information and the actual performance distribution information corresponding to the historical strategy group, and the proxy model is trained according to the loss value.

8. A model deployment device, characterized in that: include: An acquisition module is used to acquire an initial deployment strategy group for a business model, wherein the initial deployment strategy group includes two deployment strategies, each of which includes architecture information of the business model and configuration mapping information between the business model and the configuration of a processing device; A determination module, inputting feature codes of each deployment strategy in the initial deployment strategy group into a pre-trained proxy model, and determining performance distribution information of each deployment strategy on the processing device; An adjustment module, adjusting at least one of the inputs using a preset label distribution, and obtaining an adjusted deployment strategy group when a difference between the output performance distribution information and the label distribution meets a set requirement; The deployment module determines a target deployment strategy in the adjusted deployment strategy group and deploys the business model based on the target deployment strategy.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor implements the method described in any one of claims 1 to 7 when executing the program.

Images