Neural network model deployment method and apparatus, computer device, and storage medium

Abstract

The present application relates to the field of model deployment, and particularly relates to a neural network model deployment method and device, computer equipment and storage medium. The method comprises the following steps: obtaining initial operator data of an initial neural network model; performing splitting processing on at least one initial operator in the initial operator data to obtain splitting operator data; performing aggregation processing on the splitting operators in the splitting operator data to generate a plurality of candidate neural network models; performing neural structure search processing on the plurality of candidate neural network models according to the memory amount of a to-be-deployed device to obtain a target neural network model; and deploying the target neural network model on the to-be-deployed device. The present application greatly reduces the difficulty of deploying the neural network model. Meanwhile, the running performance of the neural network model on the to-be-deployed device is improved.

Description

Technical Field

[0001] This invention relates to the field of model deployment, and more particularly to a method, apparatus, computer device, and storage medium for deploying neural network models. Background Technology

[0002] Microcontroller Units (MCUs) are widely used due to their low cost, energy efficiency, and portability. With the continuous development of artificial intelligence technology, applying MCUs to AI products has become an important research direction. However, because the static memory of an MCU is typically very small, while the parameters of a neural network model increase continuously with updates, it is difficult to deploy neural network models on an MCU.

[0003] Currently, to deploy neural network models on MCUs, methods such as quantization and pruning are mainly used to compress the model size to reduce memory consumption during runtime. However, compressing the model size will reduce the precision and accuracy of the neural network model. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, and storage medium for deploying neural network models to address the aforementioned technical problems, thereby solving the problem that it is difficult to deploy neural network models on MCUs using existing technologies.

[0005] A method for deploying a neural network model, comprising:

[0006] Obtain the initial operator data for the initial neural network model;

[0007] At least one initial operator in the initial operator data is split to obtain split operator data;

[0008] The splitting operators in the splitting operator data are aggregated to generate several candidate neural network models;

[0009] Based on the memory capacity of the device to be deployed, a neural structure search process is performed on several candidate neural network models to obtain the target neural network model.

[0010] The target neural network model is deployed on the device to be deployed.

[0011] A neural network model deployment device, comprising:

[0012] The initial operator data module is used to obtain the initial operator data for the initial neural network model;

[0013] A splitting processing module is used to split at least one initial operator in the initial operator data to obtain splitting operator data;

[0014] An aggregation processing module is used to aggregate the splitting operators in the splitting operator data to generate several candidate neural network models.

[0015] The target neural network model module is used to perform neural structure search processing on several candidate neural network models according to the memory amount of the device to be deployed, so as to obtain the target neural network model.

[0016] The model deployment module is used to deploy the target neural network model on the device to be deployed.

[0017] A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-described neural network model deployment method when executing the computer-readable instructions.

[0018] One or more readable storage media storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the neural network model deployment method described above.

[0019] The aforementioned neural network model deployment method, apparatus, computer equipment, and storage medium, in steps S10-S50, involve: acquiring initial operator data of the initial neural network model; splitting at least one initial operator in the initial operator data to obtain split operator data; aggregating the split operators in the split operator data to generate several candidate neural network models; performing neural structure search processing on the several candidate neural network models according to the memory capacity of the device to be deployed to obtain a target neural network model; and deploying the target neural network model on the device to be deployed. This invention significantly reduces the deployment difficulty of neural network models by splitting and rearranging the operators of the initial neural network model to generate multiple candidate neural network models with lower memory consumption, and selecting the candidate neural network model corresponding to the memory capacity of the device to be deployed as the target neural network model for deployment on the device. Simultaneously, it can also improve the running performance of the neural network model on the device to be deployed. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1This is a schematic diagram of an application environment for a neural network model deployment method according to an embodiment of the present invention;

[0022] Figure 2 This is a flowchart illustrating a neural network model deployment method according to an embodiment of the present invention;

[0023] Figure 3-1 This is a schematic diagram of an initial operator processing in a neural network model deployment method according to an embodiment of the present invention;

[0024] Figure 3-2 This is a schematic diagram of an operator splitting process in a neural network model deployment method according to an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of a neural network model deployment device in one embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The neural network model deployment method provided in this embodiment can be applied to, for example, Figure 1 In this application environment, the client communicates with the server. Clients include, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0029] During the operation of a neural network model, as long as the input is fixed, the memory consumption at each moment is also fixed. The moment with the highest memory consumption is the memory bottleneck of the entire neural network, i.e., the peak memory usage. This application reduces the peak memory usage during the operation of the neural network model, making it meet the memory limitations of the MCU, and then deploys the neural network model on the MCU.

[0030] In one embodiment, such as Figure 2 As shown, a method for deploying a neural network model is provided, which can be applied to... Figure 1 Taking the server-side as an example, the explanation includes the following steps:

[0031] S10. Obtain the initial operator data for the initial neural network model.

[0032] Understandably, a neural network model includes several operators, each corresponding to a different output tensor. During the operation of the neural network model, calculations are performed according to a preset order of operators. When calculating the second operator, the memory consumed includes the memory used to store the output tensor of the first operator and the memory pre-allocated for the output tensor of the second operator. When calculating the third operator, since the output tensor of the first operator is no longer needed, the memory occupied by the first operator is released. That is, peak memory usage refers to the maximum memory consumption when calculating two adjacent operators. The initial neural network model refers to the neural network model before splitting. The initial operator data refers to the operator data before splitting. This initial operator data contains several initial operators. Among them, initial operators refer to the operations in the initial neural network model, such as convolutional modules, convolutional layers, pooling, fully connected layers, etc.

[0033] S20. At least one initial operator in the initial operator data is split to obtain split operator data.

[0034] Understandably, splitting refers to the process of splitting the initial operator to obtain split operators. The split operator data contains several split operators. Here, a split operator is an operator obtained by splitting the initial operator. Preferably, when splitting the initial operator data, at least one initial operator can be randomly selected as a splitting point to obtain different split operator data. Different split operator data can be aggregated to obtain different candidate neural network models. After splitting the initial operator, the output tensor of the split operator is reduced, i.e., memory consumption is reduced.

[0035] exist Figure 3-1 In the diagram, Op1, Op2, Op3, and Op4 represent initial operators, and the numbers below each initial operator indicate the memory size required for its output tensor. During the calculation from Op1 to Op2, memory is needed to store the output tensor of Op1 and also to pre-allocate memory for the output tensor of Op2. Therefore, the memory consumption during the calculation from Op1 to Op2 is 4 + 8 = 12. Similarly, the memory consumption during the calculation from Op2 to Op3 is 8 + 6 = 14, and the memory consumption during the calculation from Op3 to Op4 is 6 + 2 = 8. Clearly, in... Figure 3-1 In the calculation process from Op2 to Op3, the peak memory usage is 14. Figure 3-2In the diagram, Op1*, Op2*, Op3*, Op4*, Op1**, Op2**, Op3**, and Op4** represent splitting operators. The numbers below the splitting operator indicate the memory size required for its output tensor. The output tensor of Op1 is split in half, meaning the output tensor of Op1* is half the size of the output tensor of Op1. During computation, half of the output tensor of Op1 is first sent to subsequent operators for calculation, while the other half of the initial output tensor of the operator Op1 is temporarily stored in memory. In the process from Op1* to Op2*, since the tensor sent to Op2* is only half the size of the tensor sent to Op2, the output tensor of Op2* is also half the size of the output tensor of Op2. Therefore, the output tensor of Op2* is 4. Thus, the memory consumption from Op1* to Op2* is 2 + 4 + 2 = 8, and the memory consumption of the remaining Op2*, Op3*, and Op4* follows the same logic. After calculating half of the tensor of Op1 in Op4*, the result is saved, and the calculation of the other half of the output tensor of Op1 continues from Op2**. At this point, the calculation result of Op4* needs to be saved in memory, so the memory consumption during the process from Op4* to Op2** is 2 + 4 + 1 = 7. After Op2** is completed, the memory occupied by the other half of the output tensor of Op1 can be released, and the subsequent memory consumption follows the same pattern. Clearly, in Figure 3-2 In the middle, the peak memory usage was 9, compared to Figure 3-1 Peak memory usage has been significantly reduced.

[0036] S30. Aggregate the splitting operators in the splitting operator data to generate several candidate neural network models.

[0037] Understandingly, aggregation processing refers to the process of rearranging and combining the splitting operators in the splitting operator data to generate different candidate neural networks. Specifically, it involves obtaining the initial operators from the splitting operator data that have not undergone splitting processing; repeatedly sorting and combining the calling order of the splitting operators and the initial operators that have not undergone splitting processing to generate multiple candidate operator data; and then generating candidate neural network models based on the candidate operator data. Here, the candidate neural network model refers to the neural network model generated based on the candidate operator data.

[0038] S40. Based on the memory capacity of the device to be deployed, perform neural structure search processing on several candidate neural network models to obtain the target neural network model.

[0039] Understandably, the device to be deployed refers to a device with a small amount of memory, such as an MCU, used to deploy neural network models. Different devices have different amounts of memory. Based on the memory capacity of the device, different neural network models are selected for deployment, ensuring a better match between the memory consumption of the selected neural network model and the memory capacity of the device, thus reducing the deployment difficulty. Simultaneously, this improves the performance of the neural network model on the device. Specifically, after obtaining several neural network models, a neural architecture search tool is used to perform a neural architecture search on these models to obtain candidate neural network models corresponding to the memory capacity of the device, and these candidate neural network models are then used as the target neural network model.

[0040] S50. Deploy the target neural network model on the device to be deployed.

[0041] Understandably, the reduced memory consumption of the target neural network model compared to the initial neural network model makes it more suitable for deployment devices with limited memory.

[0042] In steps S10-S50, initial operator data of the initial neural network model is obtained; at least one initial operator in the initial operator data is split to obtain split operator data; the split operators in the split operator data are aggregated to generate several candidate neural network models; based on the memory capacity of the device to be deployed, neural structure search processing is performed on the several candidate neural network models to obtain a target neural network model; and the target neural network model is deployed on the device to be deployed. This invention significantly reduces the deployment difficulty of neural network models by splitting and rearranging the operators of the initial neural network model to generate multiple candidate neural network models with low memory consumption, and selecting the candidate neural network model corresponding to the memory capacity of the device to be deployed as the target neural network model for deployment on the device. Simultaneously, it can also improve the running performance of the neural network model on the device to be deployed.

[0043] Optionally, in step S20, i.e., splitting at least one initial operator in the initial operator data to obtain split operator data, the following steps are included:

[0044] S201. Select any one or more of the initial operators from the initial operator data;

[0045] S202. Based on the number of splits and the length of the split branches, perform the splitting process on one or more of the initial operators to obtain the splitting operator data.

[0046] Understandably, when splitting the initial operator data, any one or more initial operators from the initial operator data can be randomly selected for splitting. Different selection methods correspond to generating different split operator data. Specifically, after selecting the initial operator, the selected initial operator is split according to the number of splits and the length of the split branch. The number of splits refers to the number of parts into which the initial operator is split. For example, if the number of splits is 2, then the selected initial operator is split into 2 parts. The length of the split branch refers to the length of the split. For example, if the initial operators are Op1, Op2, Op3, and Op4, and Op1 is the split point with a split branch length of 2, then only Op1 and Op2 are split. Preferably, different initial operators correspond to different numbers of splits and different split branch lengths.

[0047] In steps S201 and S202, one or more initial operators are selected from the initial operator data; the splitting process is performed on one or more initial operators according to the number of splits and the length of the split branches to obtain the splitting operator data. This invention, by selecting one or more initial operators and performing splitting processes according to the number of splits and the length of the split branches, can obtain a large amount of splitting operator data, thereby generating a large number of candidate neural network models with low memory consumption.

[0048] Optionally, in step S30, i.e., the aggregation processing of the splitting operators in the splitting operator data to generate several candidate neural network models, includes:

[0049] S301. Obtain the initial operators in the splitting operator data that have not undergone splitting processing;

[0050] S302. The calling order of the splitting operator and the initial operator that has not been split is sorted and aggregated to generate several candidate operator data.

[0051] S303. Generate the candidate neural network model based on the candidate operator data.

[0052] Understandably, sorting and aggregation processing refers to the process of arranging and combining the calling order of splitting operators and initial operators that have not undergone splitting processing to obtain a number of candidate operator data. Here, the calling order refers to the order in which all calling operations can enable the neural network model to function correctly. The candidate operator data is generated by sorting and combining the splitting operators and initial operators that have not undergone splitting processing according to the calling order. Different calling orders and combination methods correspond to different candidate operator data. After obtaining a number of candidate operator data, a candidate neural network model is generated for each candidate operator data.

[0053] In steps S301-S303, different candidate operator data are generated according to different calling orders and combination methods of operators, which can obtain a large number of candidate neural network models to choose from, thereby improving the adaptability between the target neural network model and the device to be deployed, and thus reducing the deployment difficulty of the neural network model.

[0054] Optionally, in step S302, the order of calling the splitting operator and the initial operator without splitting is sorted and aggregated to generate several candidate operator data, including:

[0055] S3021. Obtain the splitting position and aggregation position of the splitting operator;

[0056] S3022. Based on the splitting position and the aggregation position, sort and aggregate the calling order of the splitting operator and the initial operator that has not been split to obtain the candidate operator data.

[0057] Understandably, the splitting position refers to the splitting point of the splitting operator. The aggregation position refers to the aggregation point when aggregating the splitting operator and the initial operator that has not undergone splitting. After obtaining the splitting and aggregation positions, the order in which the splitting operator and the initial operator that has not undergone splitting are called is sorted and aggregated to obtain the candidate operator data.

[0058] Optionally, in step S40, i.e., the step of performing neural architecture search processing on a number of candidate neural network models based on the memory amount of the device to be deployed, to obtain a target neural network model, includes:

[0059] S401. Obtain the first splitting parameters of several candidate neural network models;

[0060] S402. Construct a search space based on several of the first splitting parameters;

[0061] S403. Using a genetic algorithm, search the search space for a candidate neural network model that corresponds to the amount of memory of the device to be deployed, and use the candidate neural network model as the target neural network model.

[0062] Understandably, the first splitting parameters refer to the splitting parameters of the candidate neural network, including the number of splits, split points, split branch lengths, number of convolutional layers, and number of fully connected layers. A search space is constructed based on these first splitting parameters. The search space includes the first splitting parameter data for several candidate neural network models. Using a genetic algorithm, a candidate neural network model corresponding to the memory capacity of the device to be deployed can be searched within the search space.

[0063] In steps S401-S403, first splitting parameters of several candidate neural network models are obtained; a search space is constructed based on the first splitting parameters; a genetic algorithm is used to search for candidate neural network models in the search space that correspond to the memory size of the device to be deployed, and these candidate neural network models are selected as the target neural network model. This invention, by constructing a search space and using a genetic algorithm to obtain candidate neural network models corresponding to the memory size of the device to be deployed, makes the obtained target neural network model more suitable for the device to be deployed.

[0064] Optionally, in step S403, namely, using a genetic algorithm to search for a candidate neural network model in the search space corresponding to the amount of memory of the device to be deployed, and using the candidate neural network model as the target neural network model, the following steps are included:

[0065] S4031. Search the search space for a first candidate neural network model that corresponds to the amount of memory of the device to be deployed.

[0066] S4032. Obtain the second splitting parameters of the first candidate neural network model;

[0067] S4033. Based on the second splitting parameter, update the hyperparameters of the search space using a genetic algorithm until the hyperparameters tend to converge.

[0068] S4034. Search for a second candidate neural network model in the converged search space that corresponds to the amount of memory of the device to be deployed, and use the second candidate neural network model as the target neural network model.

[0069] Understandably, a genetic algorithm can be used to iteratively update the hyperparameters of the search space until they converge, at which point the updates stop. The hyperparameters of the search space include data from all the first splitting parameters. In the converged search space, a second candidate neural network model corresponding to the memory size of the device to be deployed is searched, and this second candidate neural network model is used as the target neural network model. This ensures that the memory consumption of the obtained target neural network model is more closely matched to the memory size of the device to be deployed.

[0070] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0071] In one embodiment, a neural network model deployment apparatus is provided, which corresponds one-to-one with the neural network model deployment method in the above embodiments. For example... Figure 4As shown, the neural network model deployment device includes an initial operator data module 10, a splitting processing module 20, an aggregation processing module 30, a target neural network model module 40, and a model deployment module 50. Detailed descriptions of each functional module are as follows:

[0072] The initial operator data module 10 is used to obtain the initial operator data of the initial neural network model;

[0073] The splitting processing module 20 is used to split at least one initial operator in the initial operator data to obtain splitting operator data;

[0074] The aggregation processing module 30 is used to aggregate the splitting operators in the splitting operator data to generate several candidate neural network models.

[0075] The target neural network model module 40 is used to perform neural structure search processing on several candidate neural network models according to the memory amount of the device to be deployed, so as to obtain the target neural network model.

[0076] The model deployment module 50 is used to deploy the target neural network model on the device to be deployed.

[0077] Optionally, the splitting processing module 20 includes:

[0078] An initial operator unit is used to select any one or more of the initial operators from the initial operator data;

[0079] The splitting operator data unit is used to perform the splitting process on one or more of the initial operators according to the number of splits and the length of the splitting branches to obtain the splitting operator data.

[0080] Optionally, the aggregation processing module 30 includes:

[0081] The operator data acquisition unit is used to acquire the initial operators that have not undergone splitting processing from the splitting operator data;

[0082] The operator data processing unit is used to sort and aggregate the calling order of the splitting operator and the initial operator that has not been split, and generate several candidate operator data.

[0083] The candidate neural network module unit is used to generate the candidate neural network model based on the candidate operator data.

[0084] Optionally, the operator data processing unit includes:

[0085] A location information unit is used to obtain the splitting position and aggregation position of the splitting operator;

[0086] The candidate operator data unit is used to sort and aggregate the calling order of the splitting operator and the initial operator that has not been split based on the splitting position and the aggregation position, so as to obtain the candidate operator data.

[0087] Optionally, the target neural network model module 40 includes:

[0088] The first splitting parameter unit is used to obtain the first splitting parameters of several candidate neural network models;

[0089] Search space unit, used to construct a search space based on several of the first splitting parameters;

[0090] The target neural network model unit is used to use a genetic algorithm to search for a candidate neural network model in the search space that corresponds to the amount of memory of the device to be deployed, and to use the candidate neural network model as the target neural network model.

[0091] Optionally, the target neural network model unit includes:

[0092] The first candidate neural network model unit is used to search the search space for a first candidate neural network model corresponding to the amount of memory of the device to be deployed.

[0093] The second splitting parameter unit is used to obtain the second splitting parameters of the first candidate neural network model;

[0094] The search space update unit is used to update the hyperparameters of the search space based on the second splitting parameter using a genetic algorithm until the hyperparameters tend to converge.

[0095] The target neural network model determination unit is used to search for a second candidate neural network model in the converged search space that corresponds to the amount of memory of the device to be deployed, and to use the second candidate neural network model as the target neural network model.

[0096] Specific limitations regarding the neural network model deployment device can be found in the limitations on the neural network model deployment method described above, and will not be repeated here. Each module in the aforementioned neural network model deployment device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0097] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a readable storage medium and internal memory. The readable storage medium stores an operating system, computer-readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database stores data related to the neural network model deployment method. The network interface communicates with external terminals via a network connection. When the computer-readable instructions are executed by the processor, a neural network model deployment method is implemented. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0098] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor performs the following steps when executing the computer-readable instructions:

[0099] Obtain the initial operator data for the initial neural network model;

[0100] At least one initial operator in the initial operator data is split to obtain split operator data;

[0101] The splitting operators in the splitting operator data are aggregated to generate several candidate neural network models;

[0102] Based on the memory capacity of the device to be deployed, a neural structure search process is performed on several candidate neural network models to obtain the target neural network model.

[0103] The target neural network model is deployed on the device to be deployed.

[0104] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The readable storage media stores computer-readable instructions, which, when executed by one or more processors, perform the following steps:

[0105] Obtain the initial operator data for the initial neural network model;

[0106] At least one initial operator in the initial operator data is split to obtain split operator data;

[0107] The splitting operators in the splitting operator data are aggregated to generate several candidate neural network models;

[0108] Based on the memory capacity of the device to be deployed, a neural structure search process is performed on several candidate neural network models to obtain the target neural network model.

[0109] The target neural network model is deployed on the device to be deployed.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0112] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for deploying a neural network model, characterized in that, include: Obtain the initial operator data for the initial neural network model; where initial operators refer to the operations in the initial neural network model; peak memory usage refers to the maximum memory consumption when computing two adjacent operators; At least one initial operator in the initial operator data is split to obtain split operator data; The splitting operators in the splitting operator data are aggregated to generate several candidate neural network models; Based on the memory capacity of the device to be deployed, a neural structure search process is performed on several candidate neural network models to obtain the target neural network model. Deploy the target neural network model on the device to be deployed; The step of performing neural architecture search processing on several candidate neural network models based on the memory capacity of the device to be deployed, to obtain the target neural network model, includes: Obtain the first splitting parameters of several candidate neural network models; wherein, the first splitting parameters refer to the splitting parameters of the candidate neural network, including the number of splits, splitting points, splitting branch lengths, number of convolutional layers, and number of fully connected layers of the candidate neural network; Construct a search space based on several of the first splitting parameters; Using a genetic algorithm, a candidate neural network model corresponding to the amount of memory of the device to be deployed is searched in the search space, and the candidate neural network model is used as the target neural network model. The step of using a genetic algorithm to search for candidate neural network models in the search space corresponding to the amount of memory of the device to be deployed, and then using these candidate neural network models as the target neural network model, includes: Search the search space for a first candidate neural network model that corresponds to the amount of memory in the device to be deployed. Obtain the second splitting parameters of the first candidate neural network model; Based on the second splitting parameter, the hyperparameters of the search space are updated using a genetic algorithm until the hyperparameters tend to converge; wherein, the hyperparameters of the search space include the data of all the first splitting parameters; In the converged search space, a second candidate neural network model corresponding to the amount of memory of the device to be deployed is searched, and the second candidate neural network model is used as the target neural network model.

2. The neural network model deployment method as described in claim 1, characterized in that, The step of splitting at least one initial operator in the initial operator data to obtain split operator data includes: Select any one or more of the initial operators from the initial operator data; Based on the number of splits and the length of the split branches, the splitting process is performed on one or more of the initial operators to obtain the splitting operator data.

3. The neural network model deployment method as described in claim 1, characterized in that, The aggregation process of the splitting operators in the splitting operator data to generate several candidate neural network models includes: Obtain the initial operators that have not undergone splitting processing from the splitting operator data; The order of calling the splitting operator and the initial operator that has not undergone splitting is sorted and aggregated to generate several candidate operator data. The candidate neural network model is generated based on the candidate operator data.

4. The neural network model deployment method as described in claim 3, characterized in that, The order of calling the splitting operator and the initial operator that has not undergone splitting is sorted and aggregated to generate several candidate operator data, including: Obtain the splitting position and aggregation position of the splitting operator; Based on the splitting position and the aggregation position, the calling order of the splitting operator and the initial operator that has not undergone splitting is sorted and aggregated to obtain the candidate operator data.

5. A neural network model deployment device, characterized in that, include: The initial operator data module is used to obtain the initial operator data for the initial neural network model; where initial operators refer to the operations in the initial neural network model; peak memory usage refers to the maximum memory consumption when calculating two adjacent operators; A splitting processing module is used to split at least one initial operator in the initial operator data to obtain splitting operator data; An aggregation processing module is used to aggregate the splitting operators in the splitting operator data to generate several candidate neural network models. The target neural network model module is used to perform neural structure search processing on several candidate neural network models according to the memory amount of the device to be deployed, so as to obtain the target neural network model. A model deployment module is used to deploy the target neural network model on the device to be deployed. The target neural network model module includes: The first splitting parameter unit is used to obtain the first splitting parameters of several candidate neural network models; wherein, the first splitting parameter refers to the splitting parameters of the candidate neural network, including the number of splits, splitting points, splitting branch lengths, number of convolutional layers, and number of fully connected layers of the candidate neural network. Search space unit, used to construct a search space based on several of the first splitting parameters; The target neural network model unit is used to use a genetic algorithm to search for a candidate neural network model in the search space that corresponds to the amount of memory of the device to be deployed, and to use the candidate neural network model as the target neural network model. The target neural network model unit includes: The first candidate neural network model unit is used to search the search space for a first candidate neural network model corresponding to the amount of memory of the device to be deployed. The second splitting parameter unit is used to obtain the second splitting parameters of the first candidate neural network model; The search space update unit is used to update the hyperparameters of the search space based on the second splitting parameter using a genetic algorithm until the hyperparameters tend to converge; wherein, the hyperparameters of the search space include the data of all the first splitting parameters; The target neural network model determination unit is used to search for a second candidate neural network model in the converged search space that corresponds to the amount of memory of the device to be deployed, and to use the second candidate neural network model as the target neural network model.

6. The neural network model deployment apparatus as described in claim 5, characterized in that, The splitting processing module includes: An initial operator unit is used to select any one or more of the initial operators from the initial operator data; The splitting operator data unit is used to perform the splitting process on one or more of the initial operators according to the number of splits and the length of the splitting branches to obtain the splitting operator data.

7. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the computer-readable instructions, it implements the neural network model deployment method as described in any one of claims 1 to 4.

8. One or more readable storage media storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, the one or more processors cause the neural network model deployment method as described in any one of claims 1 to 4 to be performed.

Images