Lightweight method for neural network, electronic device and readable storage medium

Abstract

The present application provides a lightweight method, an electronic device and a readable storage medium, and relates to the field of terminals. The method is applied to an electronic device, and the method includes: obtaining a first neural network, obtaining a second neural network based on the first neural network, the parameter amount of the second neural network is less than the parameter amount of the first neural network, the weight of the second neural network is determined based on the weight of the first neural network, and training the second neural network. The solution provided by the present application can reduce the parameter amount of the first neural network, that is, reduce its complexity, save the storage space required for the lightweight first neural network, and improve its running speed and efficiency.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a lightweight method for a neural network, an electronic device, and a readable storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, neural network models have been widely used in computer vision fields such as target classification and target recognition. For example, in the camera perception module, the subject detection network can perceive the attributes of the current object being photographed, such as the category of the object being photographed, including human body, animal, plant or text. In the application process of neural network models, the accuracy of neural network models is constantly improving. Correspondingly, the complexity of neural network models is also gradually increasing, the required storage space and running time are also increasing, and the operating efficiency is low. Summary of the invention

[0003] The present application provides a method for lightweighting a neural network, an electronic device, and a readable storage medium. The solution provided by the present application can reduce the number of parameters of the lightweight neural network, that is, reduce its complexity, save the storage space required for the lightweight neural network, and improve the running speed and efficiency of the lightweight neural network.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, a lightweight method for a neural network is provided, the method comprising: obtaining a first neural network; obtaining a second neural network based on the first neural network, wherein the parameter amount of the second neural network is smaller than the parameter amount of the first neural network, and the weight of the second neural network is determined based on the weight of the first neural network; and training the second neural network.

[0006] Based on the above solution, the number of parameters of the second neural network is smaller than that of the first neural network, that is, the complexity of the second neural network is lower than that of the first neural network. Deploying the trained second neural network to the terminal side can save storage space, and the running speed and efficiency of the second neural network on the terminal side will also be improved. The first neural network can be a neural network that has been trained to convergence and has met the accuracy requirements. Therefore, the weight of the second neural network is determined based on the weight of the first neural network, which can make it more likely that the accuracy, performance or power consumption of the trained second neural network on the terminal side will meet the requirements.

[0007] In a possible implementation, the first neural network includes N network blocks, N is greater than or equal to 1; obtaining the second neural network based on the first neural network includes: sampling from the N network blocks to obtain M neural sub-networks, M is less than or equal to N, the weight of each neural sub-network is determined based on the weight of the network block corresponding to the neural sub-network, and the parameter amount of at least one neural sub-network is less than the parameter amount of the network block corresponding to the neural sub-network; training the second neural network includes: training the M neural sub-networks, when training the M neural sub-networks, training each of the M neural sub-networks together with the third neural network in turn, the third neural network being a collection of other network blocks in the first neural network except the network block corresponding to the trained neural sub-network.

[0008] Based on the above scheme, sampling conditions can be set to sample M neural sub-networks from N network blocks. The M neural sub-networks are the second neural network, so the speed of obtaining the second neural network is faster. And any sampled neural sub-network is trained together with the network block, which can reduce the amount of training compared to training each neural sub-network from scratch. The first neural network can be a network sampled for the first time, that is, a neural network trained to convergence before lightweight processing, or a network sampled again after one or more rounds of training. During the training process, the weights of the trained network will be updated. Therefore, the weights of the sampled neural sub-network are determined based on the weights of the sampled network blocks. Compared with the scheme of training each neural sub-network from scratch and updating the weights during the training process, the trained neural sub-network is more likely to meet the accuracy, performance or power consumption requirements after being deployed on the end side.

[0009] In a possible implementation manner, the M neural sub-networks obtained by sampling meet the following conditions: the number of layers of at least one neural sub-network among the M neural sub-networks is less than or equal to the number of layers of the network block corresponding to the neural sub-network; the number of channels of at least one neural sub-network among the M neural sub-networks is less than or equal to the number of channels of the network block corresponding to the neural sub-network; the ratio of the input resolution of the M neural sub-networks to the input resolution of the first neural network is less than 1; the number of floating-point operations per second FLOPs of the M neural sub-networks is less than the FLOPs of the first neural network.

[0010] Based on the above scheme, during the sampling process, the number of parameters of the first neural network can be reduced by reducing the number of layers and channels of at least one network block to simplify the structure of the first neural network. The number of parameters of the first neural network can also be reduced by reducing the input resolution, so that the FLOPs of the M neural sub-networks obtained by sampling are smaller than those of the first neural network.

[0011] One possible implementation is that if the number of channels of a network block is greater than or equal to a multiple of 32, the number of channels of the neural sub-network corresponding to the network block is a multiple of 32, which can meet the support of certain hardware (such as Qualcomm hardware) for the number of channels; if the number of channels of a network block is less than 32, the number of channels of the neural sub-network corresponding to the network block is equal to the number of channels of the network block.

[0012] In one possible implementation, M is greater than or equal to 2, the M neural sub-networks include a first neural sub-network and a second neural sub-network, and training the M neural sub-networks includes: in one iteration of training, inputting training samples of a batch size into the first neural sub-network and the set of other network blocks in the first neural network except the network block corresponding to the first neural sub-network for training, and inputting training samples of the same batch size into the second neural sub-network and the set of other network blocks in the first neural network except the network block corresponding to the second neural sub-network for training.

[0013] In a possible implementation, the method also includes: constructing a reinforcement learning module, the reinforcement learning module is used to generate a strategy for re-sampling M neural sub-networks from the trained N network blocks, the state space of the reinforcement learning module includes the trained neural sub-network, the accuracy corresponding to the neural sub-network, and the inference delay corresponding to the neural sub-network; sampling M neural sub-networks from N network blocks includes: sampling M neural sub-networks from the trained N network blocks based on the sampling strategy generated by the reinforcement learning module.

[0014] Based on the above scheme, a reinforcement learning module can be constructed according to the M neural sub-networks after training, the accuracy of the M neural sub-networks after training, and the inference delay of the M neural sub-networks after training. The strategy of sampling neural sub-networks is generated by the reinforcement learning module, which can reduce the amount of training.

[0015] In a possible implementation, before building the reinforcement learning module, the method also includes: after training for multiple epochs, deploying the trained M neural sub-networks to the end side; obtaining the inference latency of the trained M neural sub-networks from the end side; and evaluating the accuracy of the trained M neural sub-networks.

[0016] Based on the above scheme, when the evaluation finds that the inference latency and accuracy of the M neural sub-networks after training meet the requirements, the M trained neural sub-networks can be deployed to the end side for use. When the evaluation finds that the inference latency and / or accuracy of the M neural sub-networks after training do not meet the requirements, the next sampling can be performed based on the sampling strategy and the above sampling conditions to obtain M neural sub-networks and continue training.

[0017] In one possible implementation, the reinforcement learning module includes an actor network and a critic network. Both the actor network and the critic network adopt a fully connected network structure. The actor network is used to map the state space to the action space, and the critic network is used to map the state space to the value.

[0018] Based on the above scheme, the state space is in vector form, so a fully connected network structure is adopted, which can make the Actor network and Critic network in the reinforcement learning module converge more easily during the training process.

[0019] In a possible implementation, the number of layers of the fully connected network structure is 3.

[0020] In one possible implementation, the reinforcement learning module uses a proximal strategy to optimize the PPO algorithm.

[0021] In a possible implementation manner, the method further includes: deploying the trained M neural sub-networks to the end side when the accuracy and inference latency of the trained M neural sub-networks meet the requirements.

[0022] In a possible implementation, the method further includes: when the accuracy and / or inference delay of the trained M neural sub-networks do not meet the requirements, constructing the above-mentioned state space based on the accuracy and inference delay of the trained M neural sub-networks.

[0023] Based on the above scheme, if the accuracy and inference delay of the M neural sub-networks obtained by sampling (for example, the first time, or the M neural sub-networks obtained by sampling in the previous few times) do not meet the requirements after training, the state space of the reinforcement learning module can be constructed based on the obtained accuracy data and inference delay data, which can be used to generate a sampling strategy for the reinforcement learning module to reduce the amount of training.

[0024] In a second aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of processing in the first aspect or any one of the implementations of the first aspect when executing the computer program.

[0025] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the steps of processing in the first aspect or any one of the methods in the first aspect.

[0026] In a fourth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an electronic device, the electronic device executes the steps of processing in the first aspect or any one of the methods in the first aspect.

[0027] Among them, the beneficial effects of the second to fourth aspects can refer to the first aspect and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is an application scenario diagram provided by an embodiment of the present application;

[0029] Figure 2 is a flowchart of a lightweight method 200 for a neural network provided in an embodiment of the present application;

[0030] Figure 3 is a flowchart of a lightweight method 300 for a neural network provided in an embodiment of the present application;

[0031] Figure 4 is a schematic diagram of a neural sub-network obtained by sampling provided in an embodiment of the present application;

[0032] Figure 5 is a schematic diagram of training a neural sub-network in an iterative process provided by an embodiment of the present application;

[0033] Figure 6 is a structural block diagram of a device 600 for training a neural network provided in an embodiment of the present application;

[0034] Figure 7 It is a structural diagram of an electronic device 700 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0036] In the description of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or its similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solution of the embodiment of the present application, in the embodiment of the present application, the words "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the differences.

[0037] Figure 1 It is an application scenario diagram provided by an embodiment of the present application.

[0038] like Figure 1As shown, the device for training the neural network takes the server 10 as an example, the device for deploying the trained neural network takes the terminal 20 as an example, the neural network deployed on the terminal 20 takes the subject detection network as an example, and the subject detection network performs relevant operations in the hardware abstraction layer (HAL). The server 10 can obtain a training set from the terminal 20, and the training set includes multiple training samples, such as photos. The server 10 can train the subject detection network based on the training set. After the training is completed, the server can evaluate the accuracy of the subject detection network by itself, that is, evaluate the accuracy of the subject detection network in identifying the attributes of the photographed object in the picture, and can deploy the trained subject detection network to the terminal 20 to test the performance of the subject detection network, such as testing the inference delay. Testing the inference delay at the terminal 20 can be achieved through hardware in the loop. The server can deploy the trained subject detection network converted into a high-pass format to the terminal 20. After the terminal 20 tests the performance, the performance data obtained by the test is transmitted to the server 10. If the performance and accuracy of the trained subject detection network meet the requirements, the trained subject detection network can be directly deployed to the terminal 20 for use. If the performance or accuracy of the trained subject detection network does not meet the requirements, the server 10 continues to train the subject detection network and continues to evaluate the performance and accuracy after training until the performance and accuracy of the trained subject detection network meet the requirements. After meeting the requirements, the trained subject detection network is deployed to the terminal 20 for use.

[0039] It is understandable that the equipment for training neural networks is not limited to servers, but can also be cloud servers, etc. Terminals include but are not limited to mobile phones, tablet computers, laptops, PDAs, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), smart homes, etc. The embodiments of the present application do not limit the equipment for training and deploying neural networks.

[0040] In order to improve the accuracy, the subject detection network is gradually complicated, so the subject detection network occupies a large storage space of the terminal 20, and the speed and efficiency of the subject detection network running on the terminal 20 are relatively low.

[0041] In view of this, the present application provides a lightweight method for a neural network, which reduces the number of parameters of the neural network, reduces its complexity, saves the storage space occupied by the neural network in the terminal, and improves the operating speed and efficiency of the neural network.

[0042] Figure 2 A schematic flow chart of a lightweight method 200 for a neural network provided in an embodiment of the present application. Figure 2 The method 200 shown includes the following steps:

[0043] S201, obtaining neural network #1, or determining neural network #1.

[0044] Exemplarily, neural network #1 is an initial neural network, that is, a neural network that needs to be lightweight. The structure of neural network #1 can be determined according to a paper or experience. For example, the algorithm of the initial neural network can be determined by investigating the paper. The initial neural network with a determined structure is recorded as neural network #0. Neural network #0 is trained according to a training sample set. The training sample set can also be called a data set. The training sample set includes multiple training samples. Inputting all training samples in the training sample set into neural network #0 for training once is called training an epoch. Inputting a batch of training samples in the training sample set into neural network #0 for training once is called training an iteration. An epoch includes multiple iterations. The number of iterations included in an epoch is determined based on the batch of training samples in the training sample set. The number of training samples contained in a batch is called batch size. Taking pictures as an example of training samples, if the training sample set contains 2,000 pictures, the 2,000 pictures are divided into 4 batches, and the batch size of each batch is 500. It takes 4 iterations to complete an epoch.

[0045] When neural network #0 is trained to convergence, its training results are saved after each epoch, such as evaluation indicators such as loss or accuracy (ACC), and the value of the neural network parameter corresponding to the best training result in all rounds of training is determined, and the value of the parameter is used as the weight of neural network #1.

[0046] S202, obtaining neural network #2 based on neural network #1, the parameter amount of neural network #2 is less than the parameter amount of neural network #1, and the weight of neural network #2 is determined based on the weight of neural network #1.

[0047] In one possible implementation, neural network #1 includes N network blocks, and some or all of the network blocks in neural network #1 are pruned in an irregular manner to obtain neural network #2, and the weights of the parameters of neural network #2 are the same as the weights of the parameters in neural network #1.

[0048] Exemplarily, the number of layers of some or all network blocks in neural network #1 can be trimmed. Taking trimming the number of layers of a certain network block as an example, if the network block #1 included in neural network #1 has 6 layers, the number of layers of network block #1 can be trimmed to 4 layers, and the number of layers of other network blocks remains unchanged to obtain neural network #2. The weights of all parameters that can correspond to neural network #2 and neural network #1 are the same. In this example, the network block #1 after the number of layers is trimmed included in neural network #2 is recorded as network block #1.0, and the network Block #1 includes layer #1, layer #2, layer #3, layer #4, layer #5 and layer #6, and network block #1.0 includes layer #1, layer #2, layer #3 and layer #4. The parameters and weights of layer #1, layer #2, layer #3 and layer #4 in network block #1.0 are the same as the parameters and weights of layer #1, layer #2, layer #3 and layer #4 in network block #1. The other network blocks in neural network #2 except network block #1.0 inherit the parameters and weights of other network blocks in neural network #1 except network block #1.

[0049] In addition to the above examples, the number of channels of some or all network blocks in neural network #1 can be cut to obtain neural network #2; or the number of layers of some network blocks in neural network #1 can be cut, and the number of channels of other network blocks can be cut to obtain neural network #2. Similarly, in these two methods, the weight of the parameter in neural network #2 inherits the weight of the parameter in neural network #1, that is, if the parameter in neural network #2 can correspond to the parameter in neural network #1, the weight of the parameter is the same as the weight of the parameter in neural network #1.

[0050] Another possible implementation is to prune part or all of the network blocks in neural network #1 in a regular manner to obtain neural network #2, that is, when pruning, each pruned network block is pruned in the same manner, and the weights of the parameters of neural network #2 are the same as the weights of the parameters of neural network #1 corresponding to the parameters.

[0051] Exemplarily, neural network #1 includes 4 network blocks, of which 2 network blocks are pruned, namely network block #1 and network block #2. Network block #1 pruned the number of layers, network block #2 also pruned the number of layers, and the number of pruned layers was also the same, that is, if network block #1 has 6 layers and network block #2 has 5 layers, network block #1 pruned from 6 layers to 4 layers, then network block #2 pruned from 5 layers to 3 layers. Alternatively, network block #1 pruned the number of channels, network block #2 also pruned the number of channels, network block #1 and network block #2 may have different numbers of channels, the pruned network block #1 and network block #2 may have the same number of channels, or the number of pruned channels of network block #1 and network block #2 may be the same. Alternatively, network block #1 and network block #2 can simultaneously trim the number of layers and channels. The number of layers and channels to be trimmed for each network block can be set, or the target number of layers and target number of channels after each network block is trimmed can be set. There may also be other regular trimming methods, which are not listed here one by one. The regular trimming method means that after certain conditions are set, the same trimming method can be adopted for each network block.

[0052] In addition, neural network #2 can be obtained by reducing the input resolution of neural network #1, or in other words, the above two implementations can be combined to reduce the input resolution of neural network #1 while trimming neural network #1 to obtain neural network #2.

[0053] S203, train neural network #2.

[0054] Exemplarily, after obtaining neural network #2 in the above manner, neural network #2 is trained based on the training sample set, and the network structure of neural network #2 and neural network #1, or the weight of parameters, etc. are dynamically adjusted during the training process. After training one or more epochs, the accuracy and performance of the trained neural network #2 are evaluated. If the evaluation finds that the accuracy and performance of the trained neural network #2 meet the requirements, the trained neural network #2 can be deployed in the terminal application as the lightweight neural network #1. If the evaluation finds that the accuracy and performance of the trained neural network #2 do not meet the requirements, it is necessary to re-acquire neural network #3 from the trained and updated neural network #1 and train neural network #3. The specific training process and the accuracy and performance evaluation process can refer to the relevant description of neural network #2, which will not be repeated. Specifically, how to train neural network #1 and neural network #2 based on the training sample set at the same time, how to evaluate the accuracy and performance of the obtained neural network #2 after training, and how to obtain a new neural network from the trained and updated neural network #1 when the evaluation accuracy and performance are not satisfied, etc. are described in detail by method 300.

[0055] Based on the above scheme, the number of parameters of the lightweight neural network (e.g., neural network #2) is smaller than that of neural network #1, that is, the complexity of the lightweight neural network is lower than that of neural network #1. Deploying the lightweight neural network #1 to the terminal after training can save storage space, and its running speed and efficiency in the terminal will also be improved. Neural network #1 is a neural network that has been trained to convergence and meets the accuracy requirements. Therefore, the weight of the lightweight neural network #1 is determined based on the weight of neural network #1, which can make it more likely that the accuracy, performance or power consumption of the lightweight neural network #1 after training will meet the requirements when running in the terminal.

[0056] The following method 300 is used to exemplarily describe how to obtain neural network #2 by trimming rules, how to train neural network #2 after obtaining it, and subsequent related operations. In method 300, neural network #2 takes a set of M neural sub-networks as an example.

[0057] Figure 3 It is a flowchart of a lightweight method 300 for a neural network provided in an embodiment of the present application. Figure 3 The method 300 shown may be performed on a device capable of training a neural network, such as a server or a cloud server, and includes the following steps:

[0058] S301, obtaining neural network #1, where neural network #1 includes N network blocks, where N is an integer greater than or equal to 1.

[0059] S301 is similar to S201, and the relevant description of S201 may be referred to and will not be repeated here.

[0060] Optionally, the method 300 includes:

[0061] S302, define sampling conditions.

[0062] A possible implementation method is to define or preconfigure sampling conditions, and subsequently sample M neural sub-networks from neural network #1 based on the sampling conditions, where M is an integer greater than or equal to 1 and less than or equal to N.

[0063] Exemplarily, the sampling conditions include:

[0064] Condition #1: The number of layers of the neural sub-network is less than or equal to the number of layers of the network block corresponding to the neural sub-network.

[0065] For example, if neural sub-network #1 is sampled from network block #1, the number of layers of neural sub-network #1 is less than or equal to the number of layers of network block #1. The network block corresponding to the neural sub-network in the embodiment of the present application can be understood as the neural sub-network sampled from the network block.

[0066] Condition #2: The number of channels of a neural sub-network is less than or equal to the number of channels of the network block corresponding to the neural sub-network.

[0067] For example, if neural sub-network #1 is sampled from network block #1, the number of channels of neural sub-network #1 is less than or equal to the number of channels of network block #1.

[0068] Optionally, if the number of channels of a network block is greater than or equal to a multiple of 32, the number of channels of the neural sub-network corresponding to the network block is a multiple of 32; if the number of channels of a network block is less than 32, the number of channels of the neural sub-network corresponding to the network block is equal to the number of channels of the network block.

[0069] For example, neural sub-network #1 is sampled from network block #1. If the number of channels of network block #1 is greater than or equal to 64, the number of channels of neural sub-network #1 is 64. If the number of channels of network block #1 is 24, the number of channels of neural sub-network #1 is 24.

[0070] Condition #3: The input resolution of the M neural sub-networks is lower than the input resolution of the neural network #1, that is, if the input resolution of the M neural sub-networks is input resolution #1, and the input resolution of the neural network #1 is input resolution #2, then (input resolution #1 / input resolution #2) < 1.

[0071] For example, (input resolution #1 / input resolution #2) = 50%, or (input resolution #1 / input resolution #2) = 80%, or the range of (input resolution #1 / input resolution #2) is 50% to 80%, or (input resolution #1 / input resolution #2) can also be other ratios.

[0072] Condition #4: The number of floating point operations per second (FLOPS) of the M neural sub-networks is less than the FLOPs of neural network #1, that is, if the FLOPs of the M neural sub-networks is N1, and the FLOPs of neural network #1 is N2, then (N1 / N2) < 1.

[0073] For example, (N1 / N2)=30%, or (N1 / N2)=80%, or (N1 / N2) is in the range of 30% to 80%, or (N1 / N2) may be other ratios.

[0074] S303, determining whether it is the first sampling.

[0075] Exemplarily, after obtaining neural network #1, determine whether it is the first time to sample M neural sub-networks from neural network #1. If it is the first sampling, execute S304; if it is not the first sampling, execute S304'. It can be understood that if it is the first sampling, the sampled neural network #1 is the original neural network, that is, it is sampled for the first time after training to convergence; if it is not the first sampling, the sampled neural network #1 is the updated neural network #1 trained on the basis of the original neural network and the sampled M neural sub-networks.

[0076] S304, based on the sampling conditions, M neural sub-networks are sampled from neural network #1, the weight of each neural sub-network is determined based on the weight of the network block corresponding to the neural sub-network, and the parameter amount of at least one neural sub-network is smaller than the parameter amount of the network block corresponding to the neural sub-network.

[0077] Exemplarily, the sampled M neural subnetworks satisfy the following conditions: the number of layers of any neural subnetwork among the M neural subnetworks is less than or equal to the number of layers of the network block corresponding to the neural subnetwork; the number of channels of any neural subnetwork among the M neural subnetworks is less than or equal to the number of channels of the network block corresponding to the neural subnetwork; the ratio of the input resolution of the M neural subnetworks to the input resolution of neural network #1 is less than 1; the FLOPs of the M neural subnetworks are less than the FLOPs of neural network #1.

[0078] When the number of channels of a network block is greater than or equal to a multiple of 32, the number of channels of the neural sub-network corresponding to the network block is a multiple of 32, which can meet the support of certain hardware (such as Qualcomm hardware) for the number of channels; when the number of channels of a network block is less than 32, the number of channels of the neural sub-network corresponding to the network block is equal to the number of channels of the network block.

[0079] Exemplarily, the data format of the parameters of any neural subnetwork in the M neural subnetworks can be a dictionary (Dict). Taking neural subnetwork #1 as an example, the dictionary of the parameters of neural subnetwork #1 is, for example, {'depth': List[int], 'channels': List[int], 'resolution': List[int]}, where List[int] represents a list storing int values, and the length of the list depends on the number of network blocks corresponding to neural subnetwork #1, 'depth' represents the number of layers or depth, 'channels' represents the number of channels, and 'resolution' represents the resolution. The number of neural subnetworks M obtained by each sampling can be used as a hyperparameter.

[0080] Below through Figure 4 The example illustrates the process of sampling M neural sub-networks from neural network #1.

[0081] like Figure 4 As shown, M neural sub-networks are obtained based on N network blocks in neural network #1. During the sampling process, a neural sub-network can be sampled from each network block, in which case N = M; or a neural sub-network can be sampled from multiple network blocks, in which case N > M ( Figure 4 (not shown); or multiple neural sub-networks can be sampled from a network block, in which case N<M ( Figure 4 not shown); or, in the sampling process, the above three sampling methods can be combined, for example, for a partial network block, each network block in the partial network block is sampled to obtain a neural sub-network, for a partial network block, each network block in the partial network block is sampled to obtain multiple neural sub-networks, for a partial network block, multiple network blocks in the partial network block are sampled to obtain a neural sub-network. The embodiments of the present application do not limit the specific sampling method.

[0082] S305, training the M neural sub-networks obtained by the above sampling.

[0083] A possible implementation method is that when training M neural sub-networks, each of the M neural sub-networks is trained in turn together with neural network #3, where neural network #3 is a collection of other network blocks in neural network #1 except the network blocks corresponding to the trained neural sub-network.

[0084] For example, M is greater than or equal to 2, and the M neural subnetworks include neural subnetwork #1 and neural subnetwork #2. In one iteration training, batchsize training samples are input into neural subnetwork #1 and neural network #1 except the network block corresponding to neural subnetwork #1 for collective training, and the same training samples are input into neural subnetwork #2 and neural network #1 except the network block corresponding to neural subnetwork #2 for collective training. When the M neural subnetworks include 3 or more neural subnetworks, the training process is analogous.

[0085] In each iteration training process, forward and backward calculations are performed on each neural sub-network. In the calculation process, instead of building a network that conforms to the neural sub-network structure from scratch, forward and backward calculations are performed only on the neural sub-network part sampled from neural network #1 during the forward and backward execution. For example, the above calculations can be implemented through special processing of pytorch, that is, the neural sub-network can inherit the weights of neural network #1, instead of training each neural sub-network separately from scratch. In each iteration training, the parameters of the neural sub-network with the lowest loss are saved.

[0086] Below through Figure 5 The example illustrates the process of training M neural sub-networks.

[0087] like Figure 5 As shown, the training samples of batchsize are training sample group #1, N=M, neural sub-network #1 is sampled from network block #1, neural sub-network #2 is sampled from network block #2, ..., and so on, neural sub-network #M is sampled from network block #N. In one iteration training process, training sample group #1 is input into neural network #1.1. Compared with neural network #1, neural network #1.1 includes neural sub-network #1, neural network #1 includes network block #1, and neural network #1.1 and other network blocks of neural network #1 are the same; after training neural network #1.1 with training sample group #1, training sample group #1 is input into neural network #1.2. Compared with neural network #1, neural network #1.2 includes neural sub-network #2, neural network #1 includes network block #2, and other network blocks of neural network #1.2 and neural network #1 are the same; After group #1 has trained neural network #1.2, it inputs training sample group #1 into neural network #1.3. Compared with neural network #1, neural network #1.3 contains neural sub-network #3, neural network #1 contains network block #3, and neural network #1.3 and other network blocks of neural network #1 are the same; ..., and so on, until training sample group #1 is input into neural network #1.N. Compared with neural network #1, neural network #1.N contains neural sub-network #M, neural network #1 contains network block #N, and other network blocks of neural network #1.N and neural network #1 are the same. At this point, one iteration training is completed.

[0088] In each iteration training process, the network structure, parameters and weights of neural network #1 can be adjusted dynamically. For example, in an iteration training process, the structure, parameters and weights of neural network #1 can be adjusted according to the structures, parameters and weights of neural network #1.1, neural network #1.2, neural network #1.3, ..., neural network #1.N after training and updating. Taking the adjustment of the structure, parameters and weights of neural network #1 according to neural network #1.1 as an example, after training neural network #1.1, the structure, parameters and weights of neural sub-network #1 are updated. Then, the structure, parameters and weights of network block #1 in neural network #1 need to be updated according to the updated structure, parameters and weights of neural sub-network #1. The adjustment of neural network #1 according to neural network #1.2, neural network #1.3, ..., neural network #1.N can be deduced by analogy. When sampling the neural sub-network again in the future, it is not sampled from the original neural network #1, but from the neural network #1 after training and updating.

[0089] For the case where N is less than M or N is greater than M, please refer to the training process when N=M, and no further details will be listed.

[0090] use Figure 5 The training method shown can reduce the amount of training compared to training each neural sub-network from scratch.

[0091] S306, evaluating whether the accuracy and performance of the M neural sub-networks after training meet the requirements, if they do, executing S307, if not, executing S307'.

[0092] Exemplarily, the performance is the inference latency. After training for multiple epochs, the trained M neural sub-networks are deployed to the terminal, the inference latency of the trained M neural sub-networks is obtained from the terminal, and the accuracy of the trained M neural sub-networks is evaluated.

[0093] It can be understood that meeting the requirements means that both the inference delay and the accuracy meet the requirements. If either the inference delay or the accuracy does not meet the requirements, the M neural sub-networks after training do not meet the requirements. For example, both the inference delay and the accuracy have preset thresholds. If the inference delay is less than or equal to the preset threshold #1, the performance meets the requirements. If the accuracy is greater than or equal to the preset threshold #2, the accuracy meets the requirements.

[0094] Optionally, while sampling by method 300 (including first sampling and non-first sampling, non-first sampling is the process of sampling according to the sampling strategy generated by the reinforcement learning module), a neural sub-network can also be obtained by random sampling. The randomly sampled neural sub-network can be trained by regular or irregular cropping, and the accuracy and performance of the randomly sampled neural sub-network after training can be evaluated later. The training process can refer to Figure 5 It should be noted that the neural sub-network obtained by random sampling also needs to meet the sampling conditions described in S302, and the number of neural sub-networks obtained by random sampling is, for example, M1, 1≤M1≤M / 3.

[0095] It can be understood that the neural sub-network obtained by random sampling is mainly used to evaluate the accuracy and performance after training to construct or update the state space so that the accuracy and performance of the neural sub-network obtained by sampling according to the sampling strategy generated by the reinforcement learning module are better.

[0096] S307, deploying the trained M neural sub-networks to the terminal.

[0097] Exemplarily, the accuracy and inference latency of the M trained neural sub-networks meet the requirements, and the trained M neural sub-networks are deployed to the terminal for use.

[0098] S307', constructing a state space based on the accuracy data and performance data of the trained M neural sub-networks.

[0099] Exemplarily, a dictionary of the accuracy and performance of the evaluated neural sub-network is constructed as the state space of the reinforcement learning module. The evaluated neural sub-network includes M neural sub-networks after training, and may also include M1 randomly sampled neural sub-networks after training, and may also include neural network #1 updated during the training process.

[0100] Optionally, the dictionary also includes the accuracy and performance of randomly sampled neural sub-networks.

[0101] Optionally, the dictionary also includes the accuracy and performance of Neural Network #1 after updates during the training process.

[0102] The method for obtaining the accuracy and performance of the trained M1 neural sub-networks and the updated neural network #1 can refer to the method for obtaining the accuracy and performance of the trained M neural sub-networks, which will not be repeated here.

[0103] For example, the state space of reinforcement learning is defined as S = {'architecture': List[Dict], 'accracy': List[float], 'latency': List[float]}. Among them, 'architecture' represents the network structure of the neural subnetwork being evaluated, 'accracy' represents the accuracy data of the neural subnetwork being evaluated, and 'latency' represents the inference latency data of the neural subnetwork being evaluated. There are multiple neural subnetworks involved in the evaluation, so a list is used to store the network structure, accuracy data, and inference latency data of each neural subnetwork. List[Dict] represents the list of network structures of each neural subnetwork using Dict (dictionary), and List[float] represents a list of float values. The length of the list depends on the number of neural subnetworks being evaluated. For example, List[float] corresponding to 'accracy' is [0.8, 0.6, 0.8, 0.4, 0.9, 0.2].

[0104] S308, construct a reinforcement learning module, which is used to generate a strategy for re-sampling M neural sub-networks from the trained and updated neural network #1. The state space of the reinforcement learning module includes the network structure of the evaluated neural sub-network, the accuracy data corresponding to the neural sub-network, and the inference delay data corresponding to the neural sub-network.

[0105] In one possible implementation, the reinforcement learning module includes an actor network and a critic network. Both the actor network and the critic network use a fully connected network structure (multi-layer perception, MLP). The actor network is used to map the state space to the action space, and the critic network is used to map the state space to the value. The use of MLP can make it easier for the actor network and the critic network in the reinforcement learning module to converge during the training process.

[0106] Exemplarily, the constructed reinforcement learning module adopts the proximal policy optimization (PPO) algorithm, the number of MLP layers used by the Actor network and the Critic network is 3, the Actor network contains the data expression of PPO, the Actor network outputs a vector, which is the action, and the vector is, for example, the Dict of the neural sub-network. After the Actor network outputs the action, the forward calculation of the reinforcement learning module is completed, and the Value output by the Critic network uses the PPO update formula to update the PPO parameters. After the parameters are updated, the backward calculation of the reinforcement learning module is completed. In this process, the Dict output by the Actor network is the sampling strategy, for example, the Dict is {'depth': List[int], 'channels': List[int], 'resolution': List[int]}, and in the subsequent sampling process, the neural sub-network is obtained according to the Dict. For example, when sampling in S304', sampling is performed based on the sampling strategy generated by the reinforcement learning module, and the number of neural sub-networks sampled according to the sampling strategy is M.

[0107] S304', sampling to obtain M neural sub-networks based on a sampling strategy, where the sampling strategy is generated based on a reinforcement learning module.

[0108] Exemplarily, if it is determined that it is not the first time to sample from neural network #1, that is, the nth sampling, n is greater than 1, the sampled network is the neural network #1 updated during the training process of the M neural sub-networks obtained based on the n-1th sampling.

[0109] After executing S304', continue with S305~S306. If the accuracy and performance of the M neural sub-networks used this time meet the requirements, execute S307. If the accuracy and performance do not meet the requirements, execute S307'~S308. However, if in the subsequent cyclic training, when executing S307', the state space has been constructed, then S307' should be described as updating the state space based on the accuracy data and performance data of the trained M neural sub-networks. When executing S308, the reinforcement learning module has been constructed and should be described as updating the reinforcement learning module. Subsequently, a cycle of sampling neural sub-networks, training neural sub-networks, evaluating the accuracy and performance of neural sub-networks, updating the state space, and updating the reinforcement learning module is performed until the accuracy and performance of the sampled M neural sub-networks meet the requirements.

[0110] The next sampling plan is based on the sampling strategy generated by the reinforcement learning module. Since the sampling strategy is generated based on the accuracy and performance of the sampled and trained neural sub-network, it has a clear sampling tendency and can reduce the amount of training to a certain extent.

[0111] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values ​​or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0112] Figure 6 1 is a structural block diagram of a device 600 for training a neural network provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0113] The device 600 is, for example, an electronic device, and the device 600 includes a digital signal processing module, referring to Figure 6 , the device 600 includes: an acquisition unit 601 and a training unit 602.

[0114] Among them, the acquisition unit 601 is used to acquire the first neural network; and acquire the second neural network based on the first neural network, the parameter amount of the second neural network is smaller than the parameter amount of the first neural network, and the weight of the second neural network is determined based on the weight of the first neural network; the training unit 602 is used to train the second neural network.

[0115] In a possible implementation, the first neural network includes N network blocks, N is greater than or equal to 1; the acquisition unit 601 is specifically used to sample M neural sub-networks from the N network blocks, M is less than or equal to N, the weight of each neural sub-network is determined based on the weight of the network block corresponding to the neural sub-network, and the parameter amount of at least one neural sub-network is less than the parameter amount of the network block corresponding to the neural sub-network; the training unit 602 is specifically used to train the M neural sub-networks, and when training the M neural sub-networks, each neural sub-network in the M neural sub-networks is trained in turn together with the third neural network, and the third neural network is a collection of other network blocks in the first neural network except the network block corresponding to the trained neural sub-network.

[0116] In a possible implementation manner, the M neural sub-networks obtained by sampling meet the following conditions: the number of layers of at least one neural sub-network among the M neural sub-networks is less than or equal to the number of layers of the network block corresponding to the neural sub-network; the number of channels of at least one neural sub-network among the M neural sub-networks is less than or equal to the number of channels of the network block corresponding to the neural sub-network; the ratio of the input resolution of the M neural sub-networks to the input resolution of the first neural network is less than 1; the number of floating-point operations per second FLOPs of the M neural sub-networks is less than the FLOPs of the first neural network.

[0117] One possible implementation is that if the number of channels of a network block is greater than or equal to a multiple of 32, the number of channels of the neural sub-network corresponding to the network block is a multiple of 32, which can meet the support of certain hardware (such as Qualcomm hardware) for the number of channels; if the number of channels of a network block is less than 32, the number of channels of the neural sub-network corresponding to the network block is equal to the number of channels of the network block.

[0118] In one possible implementation, M is greater than or equal to 2, and the M neural sub-networks include a first neural sub-network and a second neural sub-network. In one iteration of training, training samples of a batch size are input into the first neural sub-network and the collective training of other network blocks in the first neural network except the network block corresponding to the first neural sub-network, and training samples of the same batch size are input into the second neural sub-network and the collective training of other network blocks in the first neural network except the network block corresponding to the second neural sub-network.

[0119] In one possible implementation, the device 600 also includes a construction unit, which is used to construct a reinforcement learning module, which is used to generate a strategy for re-sampling M neural sub-networks from the trained N network blocks, and the state space of the reinforcement learning module includes the trained neural sub-network, the accuracy corresponding to the neural sub-network, and the inference delay corresponding to the neural sub-network; the acquisition unit 601 is specifically used to: sample M neural sub-networks from the trained N network blocks based on the sampling strategy generated by the reinforcement learning module.

[0120] In a possible implementation, the device 600 also includes a transceiver unit and an evaluation unit. Before building the reinforcement learning module, after training for multiple epochs, the trained M neural sub-networks are deployed to the end side. The transceiver unit is used to obtain the inference delay of the trained M neural sub-networks from the end side; the evaluation unit is used to evaluate the accuracy of the trained M neural sub-networks.

[0121] In a possible implementation, the reinforcement learning module includes an Actor network and a Critic network. Both the Actor network and the Critic network adopt a fully connected network structure. The Actor network is used to map the state space to the action space, and the Critic network is used to map the state space to the value.

[0122] In a possible implementation, the number of layers of the fully connected network structure is 3.

[0123] In one possible implementation, the reinforcement learning module uses a proximal strategy optimization PPO algorithm.

[0124] A possible implementation method is to deploy the trained M neural sub-networks to the end side when the accuracy and inference latency of the trained M neural sub-networks meet the requirements.

[0125] In a possible implementation, when the accuracy and / or inference delay of the M trained neural sub-networks do not meet the requirements, the construction unit is also used to construct the above-mentioned state space based on the accuracy and inference delay of the M trained neural sub-networks.

[0126] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions, execution steps, interpretation of terms, associations and technical effects brought about can be found in the method embodiment part and will not be repeated here.

[0127] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. For example, a "module" can be a software program, a hardware circuit, or a combination of the two to implement the above-mentioned functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a proprietary processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a merged logic circuit, and / or other suitable components that support the described functions.

[0128] Therefore, the modules of each example described in the embodiments of the present application can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0129] In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the protection scope of this application. The specific working process of the units and modules in the above system can refer to the corresponding process in the above method embodiment, which will not be repeated here.

[0130] It should be understood that the hardware systems, devices and electronic devices in the embodiments of the present application can execute the lightweight methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0131] Figure 7 7 is a schematic diagram of the structure of an electronic device 700 provided in an embodiment of the present application. The electronic device 700 includes a processor 701. The electronic device 700 can implement the technical solutions shown above, for example, the method 300.

[0132] The electronic device 700 also includes a transceiver 702, which is used to accept the control of the processor 701 and support the electronic device to execute the technical solution shown above. For example, the transceiver 702 can receive the inference delay data and accuracy data mentioned above, or the transceiver 702 can obtain a training sample set from the terminal side.

[0133] Optionally, the electronic device 700 may further include: a storage medium 703 .

[0134] It should be noted that the processor 701 can be implemented using the following circuits or devices: one or more field programmable gate arrays (FPGA), programmable logic devices (PLD), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits that can perform the various functions described throughout this application.

[0135] The embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored; when the computer-readable storage medium is run on an electronic device, the electronic device executes the method shown above. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) methods. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0136] The embodiment of the present application also provides a computer program product including computer instructions, which, when executed on an electronic device, enables the electronic device to execute the technical solution shown above.

[0137] The devices, electronic devices, computer storage media, and computer program products provided in the above-mentioned embodiments of the present application are all used to execute the methods provided above (for example, method 300). Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the methods provided above, and will not be repeated here.

[0138] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application. According to the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, some steps in each embodiment of the above method may be unnecessary, or some new steps may be added. Or a combination of any two or any multiple embodiments of the above. Such modifications, changes or combined solutions also fall within the scope of the embodiments of the present application.

[0139] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

[0140] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0141] It should also be understood that in the embodiments of the present application, there may be a time interval between the related operations after the condition is met and the moment when the condition is met, such as "when...", "under...", "if...", or other similar conditional restriction statements. The embodiments of the present application do not limit the execution of the related operations immediately after the condition is met.

[0142] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0143] It should also be understood that in the embodiments of the present application, "pre-configuration", "pre-setting" and "pre-definition" can be achieved by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in a device for training a neural network (for example, a server), and the present application does not limit its specific implementation method.

[0144] It should also be understood that the division of the methods, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various methods, categories, situations and embodiments can be combined without contradiction.

[0145] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A lightweight method for a neural network, characterized in that: The method comprises: Acquire a first neural network, where the first neural network is used to identify attributes of an object in a picture, and the first neural network includes N network blocks, where N is greater than or equal to 1; Sampling M neural sub-networks from the N network blocks, the M neural sub-networks constituting a second neural network, the M being less than or equal to the N, the weight of each neural sub-network being determined based on the weight of the network block corresponding to the neural sub-network, and the parameter amount of at least one neural sub-network being less than the parameter amount of the network block corresponding to the neural sub-network; The process of training the second neural network includes: training the M neural sub-networks, and when training the M neural sub-networks, training each of the M neural sub-networks in turn together with a third neural network, wherein the third neural network is a collection of other network blocks in the first neural network except the network blocks corresponding to the trained neural sub-networks.

2. The method according to claim 1, characterized in that: The M neural sub-networks obtained by sampling satisfy the following conditions: The number of layers of at least one neural sub-network among the M neural sub-networks is less than or equal to the number of layers of the network block corresponding to the neural sub-network; The number of channels of at least one neural sub-network among the M neural sub-networks is less than or equal to the number of channels of the network block corresponding to the neural sub-network; The ratio of the input resolution of the M neural sub-networks to the input resolution of the first neural network is less than 1; The number of floating point operations per second (FLOPs) of the M neural sub-networks is less than the FLOPs of the first neural network.

3. The method according to claim 1, characterized in that: If the number of channels of the network block is greater than or equal to a multiple of 32, the number of channels of the neural sub-network corresponding to the network block is a multiple of 32; if the number of channels of the network block is less than 32, the number of channels of the neural sub-network corresponding to the network block is equal to the number of channels of the network block.

4. The method according to any one of claims 1 to 3, characterized in that The M is greater than or equal to 2, the M neural sub-networks include a first neural sub-network and a second neural sub-network, and the training of the M neural sub-networks includes: In one iterative training, batch-sized training samples are input into the first neural sub-network and other network blocks in the first neural network except the network block corresponding to the first neural sub-network for collective training, and the same batch-sized training samples are input into the second neural sub-network and other network blocks in the first neural network except the network block corresponding to the second neural sub-network for collective training.

5. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Constructing a reinforcement learning module, the reinforcement learning module is used to generate a strategy for resampling M neural sub-networks from the trained N network blocks, the state space of the reinforcement learning module includes the trained neural sub-networks, the accuracy corresponding to the neural sub-networks, and the inference delay corresponding to the neural sub-networks; The sampling from the N network blocks to obtain the M neural sub-networks includes: The M neural sub-networks are obtained by sampling from the N network blocks after training based on the sampling strategy generated by the reinforcement learning module.

6. The method according to claim 5, characterized in that Before constructing the reinforcement learning module, the method further includes: After multiple rounds of training, the trained M neural sub-networks are deployed to the end side; Obtaining the inference latency of the M trained neural sub-networks from the end side; Evaluate the accuracy of the M neural sub-networks after training.

7. The method according to claim 5, characterized in that The reinforcement learning module includes an actor network and an evaluator network. Both the actor network and the evaluator network adopt a fully connected network structure. The actor network is used to map the state space to the action space, and the evaluator network is used to map the state space to the value.

8. The method according to claim 7, characterized in that The number of layers of the fully connected network structure is 3.

9. The method according to claim 6, characterized in that The reinforcement learning module adopts the proximal strategy optimization PPO algorithm.

10. The method according to claim 1, characterized in that The method further comprises: When the accuracy and inference latency of the trained M neural sub-networks meet the requirements, the trained M neural sub-networks are deployed to the end side.

11. The method according to claim 6, characterized in that The method further comprises: When the accuracy and / or inference delay of the M neural sub-networks after training do not meet the requirements, the state space is constructed based on the accuracy and inference delay of the M neural sub-networks after training.

12. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 11.

14. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 11.

Images