Neural network structure searching method and device, server and storage medium
By deploying pre-trained supernetworks on terminal devices and using lookup tables to match the optimal subnetworks, the problem of the inability to optimize neural network structure search technology on different terminal devices is solved, achieving the best trade-off between model accuracy and inference latency, and improving the user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DUXIAOMAN TECH (BEIJING) CO LTD
- Filing Date
- 2023-02-21
- Publication Date
- 2026-06-02
Smart Images
Figure CN116151309B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of neural network technology, and in particular to a method, apparatus, server, and computer-readable storage medium for searching neural network structures. Background Technology
[0002] With the development of science and technology, the design of neural network structures has shifted from manual design based on experience to automatic design based on tasks, thereby achieving efficient neural network model design, improving model accuracy, and enhancing the automation level of algorithm production. Early neural network structure search techniques were based on reinforcement learning and evolutionary algorithms. The drawback of these methods is their extremely high computational cost, which severely limits their application prospects.
[0003] Currently, in the field of neural network architecture search technology, it is usually constructed and operated based on Cell-type search space, MobileNet-type search space, etc. The computational cost of this technical solution is much lower than that of neural network architecture search technology based on reinforcement learning and evolutionary algorithms, but it also cannot achieve targeted optimization for different terminal devices. Summary of the Invention
[0004] In view of the above-mentioned problems existing in the prior art, the present disclosure provides a neural network architecture search method, apparatus, server and computer-readable storage medium to solve the problem of not being able to apply its suitable sub-model on different terminal devices.
[0005] To address the aforementioned issues, the technical solution provided by this disclosure is: a neural network architecture search method, comprising: deploying a pre-trained supernetwork to a user terminal and testing the inference latency of the supernetwork;
[0006] Based on the user's waiting time threshold and the inference latency of the hypernetwork on the user terminal and the test terminal, the user's corresponding waiting time on the test terminal is determined according to a linear relationship.
[0007] The optimal sub-network corresponding to the waiting time is matched from the lookup table; wherein, the lookup table is generated in advance in the test terminal after calling the optimal sub-network under each inference delay test of the supernetwork;
[0008] The network weights corresponding to the optimal sub-network are inherited from the supernetwork, and the sub-network is deployed.
[0009] Furthermore, the method for generating the lookup table includes:
[0010] Deploy the pre-trained supernetwork to the test terminal;
[0011] The specified delay interval is divided according to the specified step size to obtain several delays to be tested;
[0012] Among the subnetworks of the supernetwork, the subnetwork that achieves the highest accuracy in meeting the test delay is identified as the optimal subnetwork.
[0013] The optimal subnetwork corresponding to each of the tested delays is statistically analyzed, and a lookup table is generated.
[0014] Further, determining the sub-network among the sub-networks of the supernetwork that has the highest accuracy in satisfying the delay to be tested includes:
[0015] Several subnetworks are cyclically sampled from the supernetwork and their inference delays are measured respectively until several subnetworks that satisfy the test delay are obtained, forming a set of subnetworks.
[0016] Adjust the features of the subnetworks in the subnetwork set, remeasure the inference latency, and add the subnetworks that meet the test latency to the subnetwork set;
[0017] Test the inference accuracy of each subnetwork in the set of subnetworks, and extract the subnetwork with the highest accuracy.
[0018] Furthermore, adjusting the features of the subnetworks in the subnetwork set includes:
[0019] The coding features of the subnetworks in the subnetwork set are randomly adjusted according to a first probability, and / or the coding features of several random subnetworks in the subnetwork set are combined according to a second probability.
[0020] Furthermore, before dividing the specified delay interval according to the specified step size, the method further includes:
[0021] The inference latency of the supernetwork in the test terminal is determined to be a, and the inference latency of the smallest subnetwork in the supernetwork is determined to be b.
[0022] The delay interval between a and b is taken as the specified delay interval.
[0023] Further, determining the user's waiting time in the test terminal based on a linear relationship, according to the user's waiting time threshold and the inference latency of the hypernetwork in the user terminal and the test terminal, includes:
[0024] The user's waiting time threshold is determined to be h, the inference latency of the supernetwork in the test terminal is a, and the inference latency of the smallest subnetwork of the supernetwork is b.
[0025] In the user terminal, the inference latency of the supernetwork is a', and the inference latency of the minimum subnetwork is b'.
[0026] The inference delay that is closest to ((ba) / (b'-a'))*(h-a')+a in the lookup table is determined as the waiting time.
[0027] Furthermore, the training method for the hypernetwork includes:
[0028] The hypernetwork is used to train and infer on the sample set to obtain the output results.
[0029] Based on the correct labels of the sample set and the loss function generated from the output results, the model parameters are updated as feedback, serving as the initial training supernetwork;
[0030] The largest subnetwork and several other subnetworks are sampled from the initial training supernetwork and trained according to the knowledge distillation method;
[0031] The output of the largest subnetwork is used as a pseudo-label, and the model parameters are updated based on the loss function generated by the pseudo-label and the output of the other subnetworks.
[0032] Furthermore, the hypernetwork is: the MobileNet search space, or the ViT search space.
[0033] Embodiments of this disclosure also provide a neural network architecture search device, comprising:
[0034] The latency testing unit is used to deploy the pre-trained hypernetwork to the user terminal and test the inference latency of the hypernetwork.
[0035] The time determination unit is used to determine the user's corresponding waiting time in the test terminal based on the user's waiting time threshold and the inference delay of the hypernetwork in the user terminal and the test terminal, according to a linear relationship.
[0036] A network matching unit is used to match the optimal sub-network corresponding to the waiting time from a lookup table; wherein, the lookup table is generated in advance in the test terminal by calling the optimal sub-network under each inference delay test of the super network;
[0037] The subnet deployment unit is used to inherit the network weights corresponding to the optimal subnet from the supernet and to deploy the subnet.
[0038] Embodiments of this disclosure also provide a server, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the neural network structure search method described in any of the preceding claims.
[0039] Embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the neural network structure search method described in any of the preceding claims.
[0040] The beneficial effect of this disclosure is that, under the premise of pushing the same neural network model to all users, the best trade-off between model accuracy and inference latency for a specific terminal device can be achieved, thereby deploying the most suitable sub-model on the terminal device and improving the experience of users using different terminal devices. Attached Figure Description
[0041] Figure 1 This is a flowchart illustrating a neural network structure search method according to an embodiment of the present disclosure;
[0042] Figure 2 This is a schematic diagram of the structure of a MobileNet search space according to an embodiment of the present disclosure;
[0043] Figure 3 This is a schematic diagram of the structure of a lightweight ViT search space according to an embodiment of the present disclosure;
[0044] Figure 4 This is a flowchart illustrating a hypernetwork training method according to an embodiment of the present disclosure;
[0045] Figure 5 This is a flowchart illustrating a lookup table generation method according to an embodiment of the present disclosure;
[0046] Figure 6 This is a flowchart illustrating a sub-network extraction method according to an embodiment of the present disclosure;
[0047] Figure 7 This is a schematic diagram of the structure of a neural network structure search device according to an embodiment of the present disclosure;
[0048] Figure 8 This is a schematic diagram of the hardware structure of a server according to an embodiment of the present disclosure. Detailed Implementation
[0049] Various embodiments and features of this application are described herein with reference to the accompanying drawings.
[0050] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.
[0051] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0052] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.
[0053] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.
[0054] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.
[0055] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.
[0056] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.
[0057] An embodiment of the first aspect of this disclosure provides a method for searching neural network structures.
[0058] Figure 1 A flowchart illustrating the neural network structure search method according to an embodiment of the first aspect is provided.
[0059] like Figure 1 As shown, the neural network structure search method includes:
[0060] S101, deploy the pre-trained supernetwork to the user terminal and test the inference latency of the supernetwork.
[0061] In this embodiment, to ensure the target model (neural network model) can run smoothly on the terminal device, the model size will not be too large. Therefore, the hypernetwork needs to be designed as a lightweight neural network model. Preferably, the hypernetwork uses the MobileNet search space or a lightweight ViT search space.
[0062] Taking the MobileNet search space as an example, such as Figure 2 The diagram shown illustrates the MobileNet search space structure, which is a MobileNet supernetwork built upon MobileNet modules. The model consists of several MobileNet modules connected in series. The kernel size, channel expansion rate, and number of basic channels of each MobileNet module are dynamically changing, as are the number of MobileNet modules (i.e., the number of model layers).
[0063] Taking the lightweight ViT search space as an example, such as Figure 3 The diagram illustrates the ViT search space structure. The ViT model consists of several MobileNet modules in the first half and several Transformer modules in the second half. For the ViT supernetwork, the MobileNet supernetwork in the first half is the same as described above and will not be repeated. In the second half of the Transformer supernetwork, the encoding dimension, number of attention heads, and fully connected expansion rate of each Transformer module are dynamically changing, as are the number of MobileNet modules and the number of Transformer modules (i.e., the number of model layers).
[0064] In practical implementation, the MobileNet search space is suitable for scenarios with medium or relatively small amounts of data, while the lightweight ViT search space is suitable for scenarios with large amounts of data.
[0065] Furthermore, to ensure that the supernetwork and its subnetworks meet user requirements, the supernetwork needs to be trained preferentially, such as... Figure 4 As shown, the specific training steps are as follows:
[0066] S1011, the hypernetwork is called to train the sample set and the output result is obtained.
[0067] S1012, based on the correct labels of the sample set and the loss function generated from the output results, performs feedback updates on the model parameters and serves as the initial training supernetwork.
[0068] S1013 samples the largest subnetwork and several other subnetworks from the initial training supernetwork and trains them using the knowledge distillation method.
[0069] S1014 uses the output of the largest subnetwork as a pseudo-label, and updates the model parameters based on the loss function generated by the pseudo-label and the output of other subnetworks.
[0070] By adopting this training method, the parameters of the neural network model can be continuously updated, further ensuring that the entire supernetwork and all possible subnetworks sampled from the supernetwork have matching accuracy.
[0071] S102, based on the user's waiting time threshold and the inference latency of the hypernetwork at the user terminal and the test terminal, determine the user's corresponding waiting time in the test terminal according to the linear relationship.
[0072] In this embodiment of the disclosure, in order to search for the most suitable sub-network for different terminal devices, it is necessary to determine the waiting time of the user in the test terminal. The specific steps are as follows:
[0073] Set the user's waiting time threshold as h. In the test terminal, the inference latency of the supernetwork is a, and the inference latency of the minimum subnetwork of the supernetwork is b. Correspondingly, in the user terminal, the inference latency of the supernetwork is a', and the inference latency of the minimum subnetwork is b'. Then, determine the inference latency in the lookup table that is closest to ((ba) / (b'-a'))*(h-a')+a, and use it as the waiting time.
[0074] In practice, to ensure that it can meet the needs of different users, when setting the user's waiting time threshold, the waiting time threshold can be determined according to the different user characteristics to find the appropriate waiting time threshold for each user.
[0075] S103, Match the optimal subnetwork corresponding to the waiting time from the lookup table.
[0076] In this embodiment of the disclosure, the lookup table is generated in advance by calling the optimal sub-network under each inference latency test in the test terminal.
[0077] Among them, such as Figure 5 As shown, the specific steps for generating the lookup table are as follows:
[0078] S1031, deploy the pre-trained supernetwork to the test terminal.
[0079] In practice, a pre-trained supernetwork is deployed on any test terminal, and relevant data is randomly input. The inference time 'a' is obtained by inputting the data into the supernetwork, and the inference time 'b' is obtained by inputting the data into the sampled smallest subnetwork.
[0080] In order to ensure that the most suitable sub-model is configured on the terminal device, the test terminal is not specified when deploying the pre-trained supernetwork. The pre-trained supernetwork can be deployed on any test terminal.
[0081] S1032, divide the specified delay interval according to the specified step size to obtain several delays to be tested.
[0082] In specific implementation, the inference time in step S1031 is combined with the step size set to s. The delay interval between a and b obtained in step S1031 is divided according to the set step size s: (ba) / s, 2*(ba) / s, ...
[0083] S1033, among the subnetworks of the supernetwork, the subnetwork with the highest accuracy in meeting the test delay is selected as the optimal subnetwork.
[0084] In practice, for each specific delay (the delay to be tested), after testing, the subnetwork corresponding to the delay with the highest accuracy is the optimal subnetwork.
[0085] S1034, Calculate the optimal sub-network corresponding to each delay to be tested, and generate a lookup table.
[0086] In practice, specific encoding rules are used to uniquely encode the sub-networks, forming a lookup table. The sub-networks referred to here primarily are the optimal sub-networks under various inference delays in the hypernetwork test.
[0087] In this embodiment of the disclosure, among the subnetworks of the determined supernetwork, the subnetwork that achieves the highest accuracy in meeting the test delay is selected as the optimal subnetwork, such as... Figure 6 As shown, the specific steps are as follows:
[0088] S201. Several subnetworks are sampled from the supernetwork and their inference delays are measured respectively until several subnetworks that meet the test delay are obtained, forming a set of subnetworks.
[0089] In the specific implementation, the number of subnetworks n to be sampled is initialized, and the subnetworks are sampled from the supernetwork in a loop and their inference delay is measured until m subnetworks that meet the inference delay requirements are sampled and formed into a subnetwork set.
[0090] S202, adjust the features of the subnetworks in the subnetwork set, remeasure the inference delay, and add the subnetworks that meet the test delay to the subnetwork set.
[0091] In practice, the coding features of the subnetworks in the subnetwork set are randomly adjusted according to a first probability, and / or the coding features of several random subnetworks in the subnetwork set are combined according to a second probability.
[0092] For example, according to a set probability p, some coding features of the sub-networks in the set are randomly changed (such as changing the number of layers, changing the number of channels, etc.), and their inference delay is remeasured. If the inference delay requirement is met, it is put into the set; otherwise, the process is repeated.
[0093] To ensure the accuracy of the measurement, the coding features of two random subnetworks in the set can be combined to form a new subnetwork according to a set probability p. The inference delay is then measured. If the inference delay requirement is met, the subnetwork is added to the set; otherwise, the process is repeated.
[0094] S203 tests the accuracy of each subnetwork in the subnetwork set and extracts the subnetwork with the highest accuracy.
[0095] In practice, in order to obtain all subnetworks that meet the inference delay requirement, step S203 is repeated m times to obtain all subnetworks that meet the inference delay requirement. The accuracy of these subnetworks is tested, and the subnetwork with the highest accuracy is selected as the optimal subnetwork under the inference delay requirement.
[0096] The specified delay interval mainly refers to determining the inference delay of the supernetwork in the test terminal as 'a', the inference delay of the smallest subnetwork in the supernetwork as 'b', and using the delay interval between 'a' and 'b' as the specified delay interval.
[0097] S104: Inherit the network weights corresponding to the optimal subnetwork from the supernetwork and deploy the subnetwork.
[0098] In practice, a software installation package is pushed to the user. This package includes supernetwork weights, a lookup table of key-value pairs (delay, optimal subnetwork encoding), and a delay that does not affect the user experience. Then, the supernetwork model is invoked in the background to test the supernetwork's delay a'. The minimum subnetwork is sampled from the supernetwork, and the delay b' of the minimum subnetwork is tested (if it satisfies a'). <h<b’)。
[0099] Finally, the corresponding sub-network code with a delay of ((ba) / (b'-a'))*(h-a')+a is obtained from the lookup table. This sub-network is the optimal sub-network on the user terminal.
[0100] In actual operation, when a user needs to use this task scenario, the activation request is first invoked, and then the corresponding weights are inherited from the supernetwork according to the encoding of the best subnetwork, the subnetwork is deployed and provides services to the user.
[0101] In this embodiment of the disclosure, under the premise of pushing the same neural network model to all users, the optimal trade-off between model accuracy and inference latency for a specific terminal device can be achieved, thereby deploying the most suitable sub-model on that terminal device and improving the experience of users using different terminal devices.
[0102] Embodiments of the second aspect of this disclosure provide a neural network structure search apparatus.
[0103] Figure 7A schematic diagram of the neural network structure search device according to an embodiment of this disclosure is shown below. Figure 5 As shown, the device includes:
[0104] The latency testing unit 301 is used to deploy the pre-trained hypernetwork to the user terminal and test the inference latency of the hypernetwork.
[0105] The time determination unit 302 is used to determine the user's corresponding waiting time in the test terminal based on the user's waiting time threshold and the inference delay of the hypernetwork in the user terminal and the test terminal, according to a linear relationship.
[0106] The network matching unit 303 is used to match the optimal sub-network corresponding to the waiting time from the lookup table; wherein, the lookup table is generated in advance in the test terminal after calling the optimal sub-network under each inference delay test of the super network.
[0107] Subnet deployment unit 304 is used to inherit the network weights corresponding to the optimal subnet from the supernet and to deploy the subnet.
[0108] In this embodiment of the disclosure, under the premise of pushing the same neural network model to all users, the optimal trade-off between model accuracy and inference latency for a specific terminal device can be achieved, thereby deploying the most suitable sub-model on that terminal device and improving the experience of users using different terminal devices.
[0109] An embodiment of the third aspect of this disclosure provides a server that includes at least one of the above-described components.
[0110] In some embodiments, such as Figure 8 The illustrated schematic diagram shows a hardware structure of a server according to a disclosed embodiment. The server further includes a memory 402 and a processor 401. The memory 402 stores a computer program, and the processor 401 implements the methods provided in any embodiment of this disclosure when executing the computer program in the memory 402. Exemplarily, the server computer program steps are as follows: S11 to S14:
[0111] S11, deploy the pre-trained supernetwork to the user terminal and test the inference latency of the supernetwork.
[0112] S12, based on the user's waiting time threshold and the inference latency of the hypernetwork at the user terminal and the test terminal, determine the user's corresponding waiting time in the test terminal according to the linear relationship.
[0113] S13, Match the optimal sub-network corresponding to the waiting time from the lookup table; wherein, the lookup table is generated in advance in the test terminal after calling the supernetwork to test the optimal sub-network under each inference delay.
[0114] S14: Inherit the network weights corresponding to the optimal subnetwork from the supernetwork and deploy the subnetwork.
[0115] The processor generates the lookup table stored in memory by executing the following computer program:
[0116] Deploy the pre-trained supernetwork to the test terminal;
[0117] The specified delay interval is divided according to the specified step size to obtain several delays to be tested;
[0118] Among the subnetworks of the supernetwork, the subnetwork that achieves the highest accuracy in meeting the test delay is selected as the optimal subnetwork.
[0119] The optimal subnetwork corresponding to each delay to be tested is statistically analyzed, and a lookup table is generated.
[0120] When the processor selects the subnetwork from the determined supernetwork stored in memory that has the highest accuracy in satisfying the delay to be tested, it also executes the following computer program:
[0121] Several subnetworks are sampled cyclically from the supernetwork and their inference delays are measured respectively until several subnetworks that meet the test delay are obtained, forming a set of subnetworks.
[0122] Adjust the features of the subnetworks in the subnetwork set, remeasure the inference latency, and add the subnetworks that meet the latency to be tested to the subnetwork set;
[0123] Test the inference accuracy of each subnetwork in the subnetwork set, and extract the subnetwork with the highest accuracy.
[0124] Before dividing a specified delay interval into specified steps, the processor executes the following computer program stored in execution memory:
[0125] The inference latency of the supernetwork in the test terminal is determined to be 'a', and the inference latency of the smallest subnetwork in the supernetwork is determined to be 'b'.
[0126] Use the delay interval between a and b as the specified delay interval.
[0127] In some embodiments, the server further includes an input device 403 and an output device 404;
[0128] The processor 401, memory 402, input device 403, and output device 404 in the server can be connected via a bus or other means.
[0129] The memory 402, as a non-transitory computer-readable storage medium, can be used to store software programs and computer-executable programs. The processor 401 executes various functional applications and data processing of the server by running the software programs and instructions stored in the memory 402, thereby implementing the positioning method of the above-described method embodiment.
[0130] The memory 402 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device.
[0131] In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, and this remote memory may be connected to the terminal device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0132] The input device 403 may include receiving input numeric or character information, and generating key signal inputs related to user settings and function control of the server.
[0133] The output device 404 may include a display screen or other display device.
[0134] In this embodiment of the disclosure, under the premise of promoting the same neural network model to all users, the optimal trade-off between model accuracy and inference latency for a specific terminal device can be achieved, thereby deploying the most suitable sub-model on that terminal device and improving the experience of users using different terminal devices.
[0135] An embodiment of the fourth aspect of this disclosure provides a computer-readable storage medium, specifically a computer-readable medium storing a computer program that, when executed by a processor, implements the method steps provided in any embodiment of this disclosure, including the following steps S21 to S24:
[0136] S21, Deploy the pre-trained supernetwork to the user terminal and test the inference latency of the supernetwork.
[0137] S22, based on the user's waiting time threshold and the inference latency of the hypernetwork at the user terminal and the test terminal, determine the user's corresponding waiting time in the test terminal according to the linear relationship.
[0138] S23, Match the optimal sub-network corresponding to the waiting time from the lookup table; wherein, the lookup table is generated in advance in the test terminal after calling the super-network to test the optimal sub-network under each inference delay.
[0139] S24. Inherit the network weights corresponding to the optimal subnetwork from the supernetwork and deploy the subnetwork.
[0140] When a computer program is executed by the processor to generate a lookup table, the processor specifically performs the following steps:
[0141] Deploy the pre-trained supernetwork to the test terminal;
[0142] The specified delay interval is divided according to the specified step size to obtain several delays to be tested;
[0143] Among the subnetworks of the supernetwork, the subnetwork that achieves the highest accuracy in meeting the test delay is selected as the optimal subnetwork.
[0144] The optimal subnetwork corresponding to each delay to be tested is statistically analyzed, and a lookup table is generated.
[0145] When a computer program is executed by a processor to determine the subnetwork of the supernetwork that has the highest accuracy in satisfying the test latency, the processor specifically performs the following steps:
[0146] Several subnetworks are sampled cyclically from the supernetwork and their inference delays are measured respectively until several subnetworks that meet the test delay are obtained, forming a set of subnetworks.
[0147] Adjust the features of the subnetworks in the subnetwork set, remeasure the inference latency, and add the subnetworks that meet the latency to be tested to the subnetwork set;
[0148] Test the accuracy of each subnetwork in the set of subnetworks, and extract the subnetwork with the highest accuracy.
[0149] Before a computer program is executed by a processor, it performs the following steps before dividing a specified delay interval into specified steps:
[0150] The inference latency of the supernetwork in the test terminal is determined to be 'a', and the inference latency of the smallest subnetwork in the supernetwork is determined to be 'b'.
[0151] Use the delay interval between a and b as the specified delay interval.
[0152] Optionally, when executed by a computer processor, the computer-executable instructions can also be used to execute the technical solutions of the positioning methods provided in any embodiment of this disclosure.
[0153] In this embodiment of the disclosure, under the premise of pushing the same neural network model to all users, the optimal trade-off between model accuracy and inference latency for a specific terminal device can be achieved, thereby deploying the most suitable sub-model on that terminal device and improving the experience of users using different terminal devices.
[0154] The apparatus provided in this disclosure can execute the processes and steps of the above method embodiments, and also has functional modules corresponding to the above method embodiments, which can perform corresponding operations and have corresponding technical effects. To avoid repetition, it will not be described again here.
[0155] The above embodiments are merely exemplary embodiments of this disclosure and are not intended to limit this disclosure. The scope of protection of this disclosure is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this disclosure within its substance and scope, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this disclosure.
Claims
1. A method for searching neural network structures, characterized in that, include: The pre-trained supernetwork is deployed to the user terminal, and the inference latency of the supernetwork is tested. Based on the user's waiting time threshold and the inference latency of the hypernetwork on the user terminal and the test terminal, the user's corresponding waiting time on the test terminal is determined according to a linear relationship. The optimal sub-network corresponding to the waiting time is matched from the lookup table; wherein, the lookup table is generated in advance in the test terminal after calling the optimal sub-network under each inference delay test of the supernetwork; Inherit the network weights corresponding to the optimal sub-network from the supernetwork, and then deploy the sub-network; The step of determining the user's waiting time in the test terminal based on a linear relationship, according to the user's waiting time threshold and the inference latency of the hypernetwork in the user terminal and the test terminal, includes: The user's waiting time threshold is determined to be h, the inference latency of the supernetwork in the test terminal is a, and the inference latency of the smallest subnetwork of the supernetwork is b. In the user terminal, the inference latency of the supernetwork is a', and the inference latency of the minimum subnetwork is b'. The inference delay that is closest to ((ba) / (b'-a'))*(h-a') +a in the lookup table is determined as the waiting time.
2. The method as described in claim 1, characterized in that, The method for generating the lookup table includes: Deploy the pre-trained supernetwork to the test terminal; The specified delay interval is divided according to the specified step size to obtain several delays to be tested; Among the subnetworks of the supernetwork, the subnetwork that achieves the highest accuracy in meeting the test delay is identified as the optimal subnetwork. The optimal subnetwork corresponding to each of the tested delays is statistically analyzed, and a lookup table is generated.
3. The method as described in claim 2, characterized in that, The subnetwork that best satisfies the required accuracy for the delay under test among the subnetworks of the supernetwork includes: Several subnetworks are cyclically sampled from the supernetwork and their inference delays are measured respectively until several subnetworks that satisfy the test delay are obtained, forming a set of subnetworks. Adjust the features of the subnetworks in the subnetwork set, remeasure the inference latency, and add the subnetworks that meet the test latency to the subnetwork set; Test the inference accuracy of each subnetwork in the set of subnetworks, and extract the subnetwork with the highest accuracy.
4. The method as described in claim 3, characterized in that, The adjustment of the features of the subnetworks in the subnetwork set includes: The coding features of the subnetworks in the subnetwork set are randomly adjusted according to a first probability, and / or the coding features of several random subnetworks in the subnetwork set are combined according to a second probability.
5. The method as described in claim 2, characterized in that, Before dividing the specified delay interval according to the specified step size, the method further includes: The inference latency of the supernetwork in the test terminal is determined to be a, and the inference latency of the smallest subnetwork in the supernetwork is determined to be b. The delay interval between a and b is taken as the specified delay interval.
6. The method as described in claim 1, characterized in that, The training method for the hypernetwork includes: The hypernetwork is used to train and infer on the sample set to obtain the output results. Based on the correct labels of the sample set and the loss function generated from the output results, the model parameters are updated as feedback, serving as the initial training supernetwork; The largest subnetwork and several other subnetworks are sampled from the initial training supernetwork and trained according to the knowledge distillation method; The output of the largest subnetwork is used as a pseudo-label, and the model parameters are updated based on the loss function generated by the pseudo-label and the output of the other subnetworks.
7. A neural network structure search device, characterized in that, include: The latency testing unit is used to deploy the pre-trained hypernetwork to the user terminal and test the inference latency of the hypernetwork. The time determination unit is used to determine the user's corresponding waiting time in the test terminal based on the user's waiting time threshold and the inference delay of the hypernetwork in the user terminal and the test terminal, according to a linear relationship. A network matching unit is used to match the optimal sub-network corresponding to the waiting time from a lookup table; wherein, the lookup table is generated in advance in the test terminal by calling the optimal sub-network under each inference delay test of the super network; The subnet deployment unit is used to inherit the network weights corresponding to the optimal subnet from the supernet and to deploy the subnet. The time determination unit is also used to determine the user's waiting time threshold as h, the inference delay of the supernetwork in the test terminal as a, and the inference delay of the smallest subnetwork of the supernetwork as b. In the user terminal, the inference latency of the supernetwork is a', and the inference latency of the minimum subnetwork is b'. The inference delay that is closest to ((ba) / (b'-a'))*(h-a') +a in the lookup table is determined as the waiting time.
8. A server, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the neural network structure search method as described in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the neural network structure search method as described in any one of claims 1-6.