Search method and device for determining neural network architecture for processor

Abstract

Provided is a search method and device for determining a neural network architecture for a processor, the search method for determining a neural network architecture for a processor comprising: obtaining an initial first neural network architecture; starting from the initial first neural network architecture, iteratively searching for the next first neural network architecture based on a processor-related loss function until the search is stopped when a first search termination condition is met; determining the searched optimal first neural network architecture as the neural network architecture to be used on the processor, wherein the processor-related loss function includes a prediction error part and a processor operation cost part in a model training process. The prediction error part and the processor operation cost part in a model training process are used together to restrict the search convergence of the neural network architecture, thereby ensuring that the processor has good operating performance when the searched neural network architecture is running.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural networks, and more specifically, relates to a search method and device for determining a neural network architecture for a processor. Background Art

[0002] In the prior art, designing neural network architecture mainly includes two methods: manual design and neural network architecture search.

[0003] Artificially designing a neural network architecture means that designers design new operators and network structures to make the neural network architecture meet the accuracy or performance requirements. This requires designers to have professional knowledge of the field (such as the image field) to which the neural network architecture will be applied, and designers need to spend a lot of time to design and verify the performance of the network. Designers try to improve the accuracy of the network, but this often results in a large number of parameters and slow execution speed. When the overall amount of calculation is reduced, the network accuracy will be affected, and the reduction in the amount of floating-point calculations does not necessarily mean that the network speed can be improved.

[0004] The existing neural network architecture search process mainly includes: defining a search space in advance, then trying to connect different network architectures using the operations in the search space, and selecting the final neural network architecture by evaluating each connection structure. Although some studies will take into account the delay caused by some operations in the search process (for example, the relationship between floating-point operations and execution delay in the neural network architecture), after being deployed to the processor, it cannot be guaranteed that the processor will run with good performance. Summary of the invention

[0005] An exemplary embodiment of the present invention aims to provide a search method and apparatus for determining a neural network architecture for a processor.

[0006] In one general aspect, a search method for determining a neural network architecture for a processor is provided, comprising: obtaining an initial first neural network architecture; starting from the initial first neural network architecture, iteratively searching for a next first neural network architecture based on a processor-related loss function until the search is stopped when a first search termination condition is met; determining the searched optimal first neural network architecture as the neural network architecture to be used on the processor, wherein the processor-related loss function includes a prediction error portion and a processor operation cost portion in a model training process.

[0007] According to the search method for determining the neural network architecture for the processor of the present invention, the prediction error part and the processor operation cost part in the model training process are used to jointly limit the search convergence of the neural network architecture, thereby ensuring that the searched neural network architecture has good operating performance when the processor is running.

[0008] In one embodiment of the present invention, the processor operation cost portion includes a time consumption hyperparameter and / or a power consumption hyperparameter for accessing DRAM during the processor execution model training process.

[0009] According to the search method for determining the neural network architecture for the processor of the present invention, the processor is more sensitive to DRAM access than the amount of calculation of the processor executing the model training process, so the processor operation cost part is determined by the time consumption hyperparameter and / or power consumption hyperparameter of accessing DRAM. The processor operation cost part includes the time consumption hyperparameter of accessing DRAM, which can ensure that the searched neural network architecture has a higher operating speed when the processor is running; the processor operation cost part includes the power consumption hyperparameter of accessing DRAM, which can ensure that the searched neural network architecture can greatly reduce the power consumption of the processor when the processor is running; the processor operation cost part includes both the time consumption hyperparameter and the power consumption hyperparameter of accessing DRAM, which can ensure that the searched neural network architecture has a higher operating speed when the processor is running and can greatly reduce the power consumption of the processor.

[0010] In one embodiment of the present invention, the step of obtaining an initial first neural network architecture includes: obtaining an initial first neural network architecture including at least one stage, wherein each stage is stacked with at least one network block, and based on the construction rules of the network blocks, at least one composite operation is filled in the network block of each stage, and the composite operation is formed by connecting at least one basic operation.

[0011] According to the search method for determining a neural network architecture for a processor of the present invention, compound operations are screened out from multiple candidate operation combinations based on a search for a second neural network architecture, and these compound operations are applied to the search process for a first neural network architecture, which can significantly improve network accuracy and reduce network size.

[0012] In one embodiment of the present invention, the composite operation is determined by the following operations: obtaining an initial second neural network architecture including at least one network block, wherein, based on the construction rules of the network blocks, each network block of the initial second neural network architecture is filled with at least one candidate combination operation from a set of candidate combination operations, and each candidate combination operation in the set includes at least one basic operation from a plurality of preset basic operations; starting from the initial second neural network architecture, iteratively searching for the next second neural network architecture until the search is stopped when a second search termination condition is met; determining the candidate combination operations in the searched one or more preferred second neural network architectures as the composite operation.

[0013] In one embodiment of the present invention, the search method further includes: determining the construction rules of the network block according to artificial settings; and / or determining the construction rules of the network block based on the network block structure preferred for the processor.

[0014] In one embodiment of the present invention, the step of determining the construction rules of the network block based on the network block structure preferred for the processor includes: on the test platform of the processor, transforming the initial neural network according to at least one transformation method to obtain multiple candidate neural networks, wherein the initial neural network includes at least one network block structure; obtaining the operating status of each candidate neural network in the test platform; and determining the construction rules of the network block based on the network block structure in one or more preferred candidate neural networks of the operating status.

[0015] According to the search method for determining a neural network architecture for a processor of the present invention, the processor hardware preference is taken into consideration when determining the construction rules of the network blocks, the construction rules of the network blocks are determined based on the network block structure preferred for the processor, and then the search space is determined based on the construction rules of the network blocks, ultimately ensuring that the searched neural network architecture is more closely matched with the performance of the processor, so that the neural network architecture has better execution performance when running on the processor.

[0016] In addition, searching for the second neural network architecture based on the construction rules of the network block determined by the network block structure preferred for the processor can ensure that the composite operation obtained by the search is more compatible with the performance of the processor, and further enable the neural network architecture to have better execution performance when running on the processor. In one embodiment of the present invention, the step of obtaining the operating status of each candidate neural network in the test platform includes: obtaining the time consumed by each candidate neural network in processing the same set of reference data on the test platform, and the operating status includes the consumed time.

[0017] In one embodiment of the present invention, the initial neural network is transformed according to at least one transformation method, including at least one of the following steps:

[0018] Expand the initial neural network horizontally;

[0019] Expand the initial neural network vertically;

[0020] Split the individual operations in the initial neural network into parallel operations;

[0021] Change the size of the feature map of the initial neural network.

[0022] In one embodiment of the present invention, the construction rule of the network block includes at least one of the following:

[0023] The priority relationship between vertical expansion and horizontal expansion;

[0024] The number of times a single operation is split into parallel operations;

[0025] Number of channels;

[0026] The size of the feature map.

[0027] In another general aspect, a search device for determining a neural network architecture for a processor is provided, the search device comprising an initial architecture acquisition module, a target architecture acquisition module, and a target architecture confirmation module.

[0028] The initial architecture acquisition module is configured to acquire an initial first neural network architecture; the target architecture acquisition module is configured to start from the initial first neural network architecture and iteratively search for the next first neural network architecture based on the processor-related loss function until the search is stopped when the first search termination condition is met; the target architecture confirmation module is configured to determine the searched optimal first neural network architecture as the neural network architecture to be used on the processor; wherein the processor-related loss function includes the prediction error part and the processor operation cost part in the model training process.

[0029] In one embodiment of the present invention, the processor operation cost portion includes a time consumption hyperparameter and / or a power consumption hyperparameter for accessing DRAM during the processor execution model training process.

[0030] In one embodiment of the present invention, the initial architecture acquisition module is configured to: acquire an initial first neural network architecture including at least one stage, wherein each stage is stacked with at least one network block, and based on the construction rules of the network blocks, at least one composite operation is filled in the network block of each stage, and the composite operation is formed by connecting at least one basic operation.

[0031] In one embodiment of the present invention, the search device also includes a compound operation generation module, which is configured to: obtain an initial second neural network architecture including at least one network block, wherein, based on the construction rules of the network blocks, each network block of the initial second neural network architecture is filled with at least one candidate combination operation in a set of candidate combination operations, and each candidate combination operation in the set includes at least one basic operation among multiple preset basic operations; starting from the initial second neural network architecture, iteratively search for the next second neural network architecture until the search is stopped when the second search termination condition is met; determine the candidate combination operations in the searched one or more preferred second neural network architectures as compound operations.

[0032] In one embodiment of the present invention, the search device also includes a construction rule determination module, which is configured to: determine the construction rules of the network block according to manual settings; and / or determine the construction rules of the network block based on the network block structure preferred by the processor.

[0033] In one embodiment of the present invention, the construction rule determination module is configured to: on a test platform of a processor, transform an initial neural network according to at least one transformation method to obtain a plurality of candidate neural networks, wherein the initial neural network includes at least one network block structure; obtain the operating status of each candidate neural network in the test platform; and determine the construction rules of the network blocks based on the network block structures in one or more preferred candidate neural networks of the operating status.

[0034] In one embodiment of the present invention, the construction rule determination module is configured to obtain the time consumed by each candidate neural network in processing the same set of reference data on the test platform, and the operating status includes the consumed time.

[0035] In one embodiment of the present invention, the construction rule determination module is configured to perform at least one of the following steps:

[0036] Expand the initial neural network horizontally;

[0037] Expand the initial neural network vertically;

[0038] Split the individual operations in the initial neural network into parallel operations;

[0039] Changing the size of the feature map of the initial neural network;

[0040] Change the number of channels of the initial neural network.

[0041] In one embodiment of the present invention, the construction rule of the network block includes at least one of the following:

[0042] The priority relationship between vertical expansion and horizontal expansion;

[0043] The number of times a single operation is split into parallel operations;

[0044] Number of channels;

[0045] The size of the feature map.

[0046] In another general aspect, a computing device is provided, comprising a processor and a memory; the memory stores a computer program, and when the computer program is executed by the processor, the above-mentioned search method for determining a neural network architecture for the processor is implemented.

[0047] In another general aspect, a computer-readable storage medium storing instructions is provided, which, when executed by at least one computing device, causes the at least one computing device to perform the above-mentioned search method for determining a neural network architecture for a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other objects, features and advantages of exemplary embodiments of the present invention will become more apparent through the following detailed description taken in conjunction with the accompanying drawings which exemplarily illustrate embodiments, in which:

[0049] Figure 1 A flow chart of a search method for determining a neural network architecture for a processor according to an exemplary embodiment of the present invention is shown;

[0050] Figure 2 A flow chart showing determination of a composite operation from a plurality of basic operations according to an exemplary embodiment of the present invention is shown;

[0051] Figure 3 A flowchart of determining a construction rule of a network block based on a network block structure preferred for a processor according to an exemplary embodiment of the present invention is shown;

[0052] Figure 4 A flowchart of a search method for determining a neural network architecture for a processor when the processor is an NPU is shown;

[0053] Figure 5 A simplified diagram of the process of determining a construction rule of a network block based on a network block structure preferred by a processor is shown;

[0054] Figure 6 A simplified diagram of a process of determining a composite operation from a plurality of basic operations according to an exemplary embodiment of the present invention is shown;

[0055] Figure 7 A simplified diagram showing a process of a search method for determining a neural network architecture for a processor according to an exemplary embodiment of the present invention;

[0056] Figure 8 A schematic diagram showing the relationship of composite operations in a network block of a first neural network architecture of an exemplary embodiment of the present invention;

[0057] Fig. 9 A block diagram of a search device for determining a neural network architecture for a processor according to an exemplary embodiment of the present invention is shown;

[0058] Fig.10 A block diagram of a computing device according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0059] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the embodiments of the present invention as defined by the claims and their equivalents. Various specific details are included to assist in understanding, but these details are considered to be exemplary only. Therefore, one of ordinary skill in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. In addition, descriptions of well-known functions and structures are omitted for clarity and brevity.

[0060] In order to train a target model for performing estimation on a specific problem based on training samples, it is necessary to pre-design the neural network architecture of the target model. The neural network architecture can be obtained through search, but the neural network architecture obtained by the search method based on the existing neural network architecture cannot guarantee that the processor will run with good performance after being deployed on the processor.

[0061] An exemplary embodiment of the present invention provides a search method for determining a neural network architecture for a processor, which utilizes the prediction error part and the processor operation cost part in the model training process to jointly limit the search convergence of the neural network architecture, thereby ensuring that the searched neural network architecture has good operating performance when the processor is running.

[0062] Based on the training samples, the neural network architecture obtained by the exemplary embodiment of the present invention can be trained into a target model for performing estimation for a specific problem, and the target model can be used to process problems such as speech recognition, image recognition, or text recognition. For example, when the neural network architecture is used to process image data, each frame of the image data can be classified based on the features of each frame of the image.

[0063] Figure 1 A flow chart of a method for searching a neural network architecture for a processor according to an exemplary embodiment of the present invention is shown.

[0064] Reference Figure 1 , in step S110, an initial first neural network architecture is obtained.

[0065] The neural network architecture includes the connection relationship between each node in the network. According to an embodiment of the present invention, the architecture of the neural network can be expressed by, for example, a directed acyclic graph (DAG) or a structure such as a network block. Accordingly, all variable factors of the neural network architecture constitute the entire search space of the network architecture. Starting from the initial neural network architecture as the starting point of the search (for example, a manually designed network architecture can be used as the starting point), an optimized neural network architecture can be iteratively searched in the search space.

[0066] In step S120, starting from the initial first neural network architecture, the next first neural network architecture is iteratively searched based on the processor-related loss function until the search is stopped when the first search termination condition is met.

[0067] Among them, the processor-related loss function includes the prediction error part and the processor operation cost part in the model training process.

[0068] Optionally, the processor has deep neural network computing capabilities, for example, it can be a dedicated processor or accelerator with acceleration function for neural networks.

[0069] As an example, the processor includes at least an embedded neural network processor (Neural-network Processing Unit, NPU), a central processing unit, a graphics processor, etc.

[0070] Here, according to an embodiment of the present invention, in order to be able to train a target model for performing estimation on a specific problem based on training samples, for example, a target model for speech recognition, image recognition or text recognition, the method of an embodiment of the present invention can be used to automatically search for the neural network architecture of the target model.

[0071] Accordingly, a training set and a validation set can be constructed in proportion based on at least a portion of the training samples to train and evaluate each candidate first neural network architecture in the search space. According to an embodiment of the present invention, a processor-related loss function that reflects the processor operation cost is used in the evaluation process, which includes not only a prediction error part, but also an additional processor operation cost part.

[0072] In step S130, the searched optimal first neural network architecture is determined as the neural network architecture to be used on the processor.

[0073] As an example, the optimal first neural network architecture may be the first neural network architecture whose loss function corresponds to an optimal value.

[0074] According to the search method for determining the neural network architecture for the processor of the present invention, the prediction error part and the processor operation cost part in the model training process are used to jointly limit the search convergence of the neural network architecture, thereby ensuring that the searched neural network architecture has good operating performance when the processor is running.

[0075] Optionally, the processor operation cost part includes a time consumption hyperparameter and / or a power consumption hyperparameter for accessing DRAM during the processor execution model training process. It should be noted here that DRAM is the abbreviation of Dynamic Random Access Memory, which means dynamic random access memory.

[0076] The inventors of this application have discovered that the processor is more sensitive to DRAM access than the amount of computation performed by the processor during the model training process, and therefore the processor computational cost portion will be determined by the time consumption hyperparameter and / or power consumption hyperparameter of accessing DRAM.

[0077] It is understandable that the processor will access DRAM during the model training process, resulting in time consumption hyperparameters and / or power consumption. The exemplary embodiment of the present invention uses time consumption hyperparameters and / or power consumption hyperparameters as part of the processor operation cost.

[0078] Optionally, the time consumption hyperparameter and / or the power consumption hyperparameter may be obtained by at least one of the following methods:

[0079] Method (a1): The time consumption hyperparameter and / or power consumption hyperparameter included in the processor operation cost part is obtained through testing.

[0080] Method (a2): The time consumption hyperparameter and / or power consumption hyperparameter included in the processor computing cost are estimated by using the processor specification parameters.

[0081] The following uses the processor operation cost part, including the time consumption hyperparameter of accessing DRAM during the processor execution model training process, as an example to illustrate the expression of the processor-related loss function.

[0082] The expression of the processor-related loss function is as follows: LOSS = L CE (w,a)+λL DRAM (a).

[0083] LOSS is the processor-dependent loss function, L CE (w,a) is the prediction error part in the model training process, L DRAM (a) is the hyperparameter for the time consumption of accessing DRAM during the processor execution model training process. λ is used to balance the impact of DRAM access on search results. The value of λ can indicate the relative importance of the execution speed and accuracy of the neural network architecture. The larger the value of λ, the more important the execution speed of the neural network architecture is than the accuracy.

[0084] The processor operation cost includes the time consumption hyperparameter for accessing DRAM, which can ensure that the processor has a higher running speed when the searched neural network architecture is running.

[0085] The processor operation cost includes the power consumption hyperparameters of accessing DRAM, which can ensure that the searched neural network architecture can greatly reduce the power consumption of the processor when the processor is running.

[0086] The processor operation cost part also includes the time consumption hyperparameter and power consumption hyperparameter for accessing DRAM, which can ensure that when the searched neural network architecture is running on the processor, the processor has a high operating speed and can greatly reduce the power consumption of the processor.

[0087] In the above manner, the search can be iteratively performed in the search space of the first neural network architecture until the first search termination condition is met.

[0088] Optionally, the first search termination condition may be determined according to actual design requirements. For example, satisfying the first search termination condition may mean that the number of search iterations has reached a preset number; or satisfying the first search termination condition may mean that the search duration has reached a preset duration; or, the value of the processor-related loss function corresponding to the searched first neural network architecture meets a preset numerical condition.

[0089] In one embodiment, the initial first neural network architecture may include at least one stage, each stage includes at least one network block, and the network block may be regarded as a component unit of the first neural network architecture. Each network block is filled with at least one composite operation, and the composite operation is formed by connecting at least one basic operation.

[0090] The number and type of compound operations filled in each network block in the initial first neural network architecture can be the same or different. When a network block is filled with more than two compound operations, the types of any two compound operations in the network block can be the same or different.

[0091] Optionally, the basic operation includes at least one of convolution, deconvolution, activation function and pooling. Of course, the basic operation can also include any operation that can be used in a neural network, which will not be listed here. When a composite operation includes more than two basic operations, the types of any two basic operations in the composite operation can be the same or different.

[0092] For example, the exemplary embodiment of the present invention provides 5 compound operations, the first compound operation includes a 1×1 convolution operation, the second compound operation includes 2 1×1 convolution operations, the third compound operation includes a 1×1 convolution operation and a 3×3 convolution operation, the fourth compound operation includes a 3×3 convolution operation and 2 pooling operations, and the fifth compound operation includes 3 pooling operations. The exemplary embodiment of the present invention does not limit the composition form of the compound operation, and the compound operation can also have more composition forms, which are not listed here one by one.

[0093] Before step S110, at least one composite operation may be determined, and then steps S110 to S130 are performed in sequence, wherein each composite operation may be formed by connecting at least one basic operation.

[0094] In one embodiment of the present invention, at least one composite operation may be determined from a set of candidate composite operations, wherein each candidate composite operation includes at least one basic operation from among a plurality of preset basic operations. The following content of this specification will specifically introduce the process of determining a composite operation.

[0095] After determining at least one composite operation, step S110 may include: obtaining an initial first neural network architecture including at least one stage, wherein each stage is stacked with at least one network block, and based on the construction rules of the network blocks, at least one composite operation is filled in the network block of each stage, and the composite operation is formed by connecting at least one basic operation.

[0096] It should be noted that before step S110, the construction rules of the network block can be determined according to manual settings, or the construction rules of the network block can be determined based on the network block structure preferred for the processor. The subsequent content of this specification will specifically introduce the steps of determining the construction rules of the network block based on the network block structure preferred for the processor.

[0097] Optionally, the number of stages included in the initial first neural network architecture may be determined according to actual design requirements. For example, the initial first neural network architecture may include only one stage; or, the initial first neural network architecture may include two or more stages.

[0098] When the initial first neural network architecture includes more than two stages, the stages can be connected, and the connection method between the stages can be determined according to actual design requirements.

[0099] As an example of the connection between stages, multiple stages can be distributed in sequence along a first direction, and each stage is connected to one or two adjacent stages along the first direction. The first direction here can be consistent with the transmission direction of the network node operation result. Figure 7 The stage 1 shown in FIG. 1 is connected to the stage after it, and the last stage (such as Figure 7 The stage X shown in the figure is connected to the previous stage, and for the remaining stages (such as Figure 7 The stage 2) shown is connected to the two adjacent stages before and after it respectively.

[0100] by Figure 7 For example, the initial first neural network includes stage 1 to stage X, and stages 1 to stage X are sequentially distributed along the first direction, stage 1 is connected to stage 2, stage 2 is connected to stage 3, and so on. As another example of the connection between stages, multiple stages can be connected in the first direction and the second direction (here, as Figure 7As shown, the second direction is different from the first direction), and each stage is connected to the stages adjacent to the first direction and / or the second direction. Of course, the multiple stages in the initial first neural network architecture can also have other distribution and connection methods, which are not listed here one by one.

[0101] It should be noted here that the first direction and the second direction in the present invention do not refer to directions in the physical sense. The concepts of the first direction and the second direction are introduced to facilitate the distinction of all different distributions and connection methods of multiple stages in the initial first neural network architecture.

[0102] Optionally, when the initial first neural network architecture further includes a reduction layer, each reduction layer includes at least one network block, and the stages are connected via the reduction layer. Figure 7 For example, stage 1 to stage X are sequentially distributed along the first direction, stage 1 is connected to stage 2 via reduction layer 1, and stage 2 is connected to stage 3 via reduction layer 2.

[0103] Optionally, the number of network blocks included in each stage and each reduction layer can be preset according to actual design needs or learned in an iterative process. Taking a certain stage as an example, the stage can include only one network block; or the stage can include two or more network blocks.

[0104] The number and type of network blocks in each stage (or reduction layer) can be the same or different. When a stage (or reduction layer) is filled with more than two network blocks, the types of any two network blocks in the stage (or reduction layer) can be the same or different.

[0105] In one embodiment, when executing step S120, at least one of the number of stages, the connection method between stages, the number of network blocks in each stage, the stacking method of network blocks in each stage, and the filling method of composite operations in each network block can be determined as a variable factor through search.

[0106] In one embodiment, when executing step S120, the number of stages remains unchanged and the connection method between the stages remains unchanged, and the number of network blocks in each stage, the stacking method of the network blocks in each stage, and the filling method of the composite operations in each network block can be determined as variable factors through search.

[0107] The process of step S120 includes: based on the processor-related loss function, using NAS to perform a filling search for each network block and the composite operation therein in the initial first neural network architecture. NAS is the abbreviation of Neural Architecture Search, i.e., neural architecture search.

[0108] It can be understood that the process of searching for the next first neural network architecture is actually a change to the current first neural network architecture, that is, changing at least one of the number of stages of the current first neural network architecture, the connection method between stages, the number of network blocks in each stage, the stacking method of network blocks in each stage, and the filling method of compound operations in each network block, and then training and verifying the replaced first neural network architecture.

[0109] When training and verifying each first neural network architecture in the search space, the training set can be used to perform model training on the candidate first neural network architecture. After completing at least one training, the trained model can be verified using the verification set.

[0110] In the process of training the model, it is necessary to continuously calculate the value of the corresponding processor-related loss function. Optionally, the value of the processor-related loss function can be used to evaluate the first neural network architecture, and whether the first search termination condition is met can also be determined based on the value of the processor-related loss function. For example, when the value of the processor-related loss function corresponding to the searched first neural network architecture meets the preset numerical condition, it is determined that the first search termination condition is met.

[0111] As mentioned above, the processor-related loss function includes the prediction error part and the processor operation cost part in the model training process. The prediction error part can be obtained when the trained model is verified using the verification set, and the processor operation cost part may include the time consumption hyperparameter generated by the processor accessing DRAM during the model training process. The following introduces a method for obtaining the time consumption hyperparameter.

[0112] The time consumption hyperparameter for accessing DRAM during the processor execution model training process includes the first sum of the time consumption for accessing DRAM by all network blocks in the first neural network architecture and the sum of the weights of all compound operations in the first neural network architecture. The expression of the time consumption hyperparameter for accessing DRAM during the processor execution model training process is as follows:

[0113]

[0114] In the above expression, L DRAM It is a hyperparameter for the time consumed by the processor to access DRAM during model training. The sum of the time consumed by all network blocks in the first neural network architecture to access DRAM, k is the sum of the weights of all composite operations in the first neural network architecture. 0 is a constant term, k 1k is the coefficient of the sum of the time consumption of all network blocks accessing DRAM in the first neural network architecture, 2 is the coefficient of the sum of the weights of all composite operations in the first neural network architecture.

[0115] The time consumption of a network block accessing DRAM in the first neural network architecture includes the second sum of the time consumption of all compound operations in the network block accessing DRAM. Figure 8 As shown, a network block includes a composite operation mixop 1 To the composite operation mixop N , compound operation mixop 1 The weight is p 1 , compound operation mixop 2 The weight is p 2 , and so on, the composite operation mixop N The weight is p N . Figure 8 The network blocks y-1, y, and y+1 in are all network blocks in the first neural network architecture. Figure 8 In the example, input means input and output means output.

[0116] The expression for the time consumption of a network block accessing DRAM in the first neural network architecture is:

[0117] L_DRAM y =p 1 ×D(mixop 1 )+p 2 ×D(mixop 2 )+……p N ×D(mixop N ).

[0118] Among them, L_DRAM y is the time consumption of network block y accessing DRAM, D(mixop N ) is the composite operation mixop N The time consumption of accessing DRAM, p N Composite operation mixop N The weight of . N is a positive integer, and N represents the number of composite operations of a network block.

[0119] The time consumption of a composite operation in a network block accessing DRAM includes the sum of the time consumption of all basic operations in the composite operation accessing DRAM. Figure 8 As shown, a composite operation mixop in the network block m Including basic operations op 1 To basic operation op M, a composite operation mixop in the network block m The expression for the time consumption of accessing DRAM is:

[0120] D(mixop m )=D(op 1 )+D(op 1 )+……D(op M ).

[0121] D(mixop m ) is the composite operation mixop m The time consumption of accessing DRAM, D(op M ) is the basic operation op M The time consumption of accessing DRAM. The time consumption of each basic operation to access DRAM depends on the basic operation and size, the number of channels and the size of the feature map. Where M is a positive integer, and M represents the number of basic operations in a compound operation.

[0122] The process of determining the composite operation is specifically introduced below. It should be noted here that when determining the composite operation, the construction rules of the network block can be determined first. That is to say, in one embodiment of the present invention, the construction rules of the network block are first determined, then at least one composite operation is determined, and finally steps S110 to S130 are performed in sequence. Among them, the construction rules of the network block can be determined according to artificial settings, or they can be determined based on the network block structure preferred for the processor. The subsequent content of this specification will specifically introduce the steps of determining the construction rules of the network block based on the network block structure preferred for the processor.

[0123] Figure 2 A flow chart of determining a composite operation from a plurality of basic operations according to an exemplary embodiment of the present invention is shown.

[0124] Reference Figure 2 In step S210, an initial second neural network architecture including at least one network block is obtained. Based on the construction rule of the network block, at least one candidate combination operation in a set of candidate combination operations is filled in each network block of the initial second neural network architecture, and each candidate combination operation in the set includes at least one basic operation among a plurality of preset basic operations.

[0125] The initial second neural network includes one or more network blocks, each of which is filled with at least one candidate combination operation. Different types of candidate combination operations can be obtained by filling different types and / or numbers of candidate combination operations in the network blocks. The initial second neural network architecture can include only a single network block, and the initial second neural network architecture can also include more than two network blocks of the same or different types.

[0126] Optionally, a candidate combined operation may include only one basic operation, or may include two or more basic operations. A plurality of basic operations may be randomly combined, and at least one combined basic operation may be connected to form a candidate combined operation.

[0127] Optionally, based on one or more groups of random seeds, multiple basic operations are connected into at least one candidate combined operation. In this way, it can be effectively ensured that the search space is appropriately increased so as to effectively find the preferred combined operation.

[0128] In step S220, starting from the initial second neural network architecture, the next second neural network architecture is iteratively searched until the search is stopped when the second search termination condition is met.

[0129] When executing step S220, at least one of the number of network blocks in the second neural network architecture, the stacking method of the network blocks, and the filling method of the candidate operations in each network block can be determined as variable factors through search.

[0130] Optionally, when executing step S220, NAS may be used to perform a fill search for candidate combination operations in each network block in the second neural network architecture. Here, as an example, the network block itself may be fixed, for example, the second neural network architecture is fixedly set to include a single network block; or, the composition of the network block may be one of the learned hyperparameters, that is, the network block and the operations therein are all obtained through learning.

[0131] It can be understood that the process of searching for the next second neural network architecture is actually to change the network blocks and / or candidate combination operations of the current second neural network architecture, replace the network blocks in the current second neural network architecture and the types and / or quantities of candidate combination operations filled therein, and train and verify the replaced second neural network architecture.

[0132] As an example, when training and verifying each second neural network architecture in the search space, the training set can be used to perform model training on the candidate second neural network architecture. After completing at least one training, the trained model can be verified using the verification set.

[0133] Optionally, the second search termination condition can be determined according to actual design requirements. For example, satisfying the second search termination condition may mean that the number of searches has reached a preset number of times; or satisfying the second search termination condition may mean that the search duration has reached a preset duration; or, the prediction error of the searched second neural network architecture meets a preset error value condition.

[0134] It should be noted here that the preset error value conditions corresponding to the first neural network architecture and the second neural network architecture may be the same or different.

[0135] In step S230, candidate combination operations in the searched one or more preferred second neural network architectures are determined as composite operations.

[0136] As an example, one or more second neural network architectures whose prediction errors meet preset error value conditions are taken as preferred second neural network architectures.

[0137] Based on step S210 and step S220, among multiple candidate operation combinations, compound operations are screened out based on the search of the second neural network architecture, and these compound operations are applied to the search process of the first neural network architecture, which can significantly improve the network accuracy and reduce the network size.

[0138] Figure 3 A flow chart of determining a construction rule of a network block based on a network block structure preferred for a processor according to an exemplary embodiment of the present invention is shown.

[0139] Reference Figure 3 In step S310, on a test platform of a processor, the initial neural network is transformed according to at least one transformation method to obtain a plurality of candidate neural networks.

[0140] The initial neural network includes at least one network block structure. Optionally, the initial neural network may include only one network block structure, or may include two or more network block structures. Each network block structure in the initial neural network is transformed in the same manner.

[0141] As an example, transforming the initial neural network according to at least one transformation method includes at least one of the following steps:

[0142] Expand the initial neural network horizontally;

[0143] Expand the initial neural network vertically;

[0144] Split the individual operations in the initial neural network into parallel operations;

[0145] Changing the size of the feature map of the initial neural network;

[0146] Change the number of channels of the initial neural network.

[0147] In step S320, the running status of each candidate neural network in the test platform is obtained.

[0148] As an example, the step of obtaining the operating status of each candidate neural network in the test platform includes: obtaining the time consumed by each candidate neural network in processing the same set of reference data on the test platform, and the operating status includes the consumed time.

[0149] As an example, time consumption may refer to the time taken by the candidate neural network to process a set of reference data. Time consumption may also refer to the amount of reference data processed by the candidate neural network per unit time, that is, the speed at which the candidate neural network processes the reference data.

[0150] Optionally, the reference data may be image data, text data, voice data, etc. Taking the case where the reference data is image data as an example, each candidate neural network recognizes the same set of image data. The time consumed by the candidate neural network to complete the recognition of a set of image data is taken as the time consumption, or the amount of image data recognized by the candidate neural network in a unit time is taken as the time consumption, for example, the number of image frames recognized by the candidate neural network in 1 second is taken as the time consumption.

[0151] In step S330, based on the network block structure in one or more preferred candidate neural networks of the running status, a construction rule of the network block is determined.

[0152] Optionally, one or more preferred candidate neural networks for operating conditions are determined based on the time taken by each candidate neural network to process a set of reference data, and a construction rule for network blocks is determined based on the network block structure in the one or more preferred candidate neural networks for operating conditions.

[0153] Optionally, one or more preferred candidate neural networks for operating conditions are determined based on the amount of reference data processed by each candidate neural network per unit time, and network block construction rules are determined based on the network block structure in the one or more preferred candidate neural networks for operating conditions.

[0154] Optionally, determining a construction rule of a network block may include at least one of the following contents:

[0155] Prioritize horizontal and vertical expansion;

[0156] The number of times a single operation is split into parallel operations;

[0157] Number of channels;

[0158] The size of the feature map.

[0159] The present invention takes into account the preferences of processor hardware, determines the construction rules of the network blocks based on the network block structure preferred for the processor, and then determines the search space based on the construction rules of the network blocks, ultimately ensuring that the searched neural network architecture is more closely matched with the performance of the processor, so that the neural network architecture has better execution performance when running on the processor.

[0160] In addition, searching for the second neural network architecture based on the construction rules of the network blocks determined by the network block structure preferred for the processor can ensure that the searched composite operations are more closely matched with the performance of the processor, further enabling the neural network architecture to have better execution performance when running on the processor.

[0161] Figure 4 A flowchart of a search method for determining a neural network architecture for a processor when the processor is an NPU is shown. For the details involved, please refer to the relevant description above and will not be repeated here.

[0162] Reference Figure 4 In step S401, on a test platform of a processor, the initial neural network is transformed according to at least one transformation method to obtain a plurality of candidate neural networks.

[0163] The initial neural network includes multiple network block structures, such as Figure 5 As shown, the multiple network block structures include network block n-1, network block n and network block n+1, etc., and each network block structure in the initial neural network is transformed in the same way.

[0164] As an example, the initial neural network is transformed in at least one transformation mode:

[0165] Expand the initial neural network horizontally;

[0166] Expand the initial neural network vertically;

[0167] Split the individual operations in the initial neural network into parallel operations;

[0168] Changing the size of the feature map of the initial neural network;

[0169] Change the number of channels of the initial neural network.

[0170] Specifically, in an exemplary embodiment of the present invention, the initial neural network is vertically expanded according to different expansion values ​​to obtain multiple first candidate neural networks. The initial neural network is horizontally expanded according to different expansion values ​​to obtain multiple second candidate neural networks. The single operation in the initial neural network is split in parallel according to different quantities to obtain multiple third candidate neural networks. The size of the feature map of the initial neural network is changed to different values ​​to obtain multiple fourth candidate neural networks. The number of channels of the initial neural network is changed to different values ​​to obtain multiple fifth candidate neural networks.

[0171] It should be noted here that the expansion value of horizontal expansion refers to the number of expanded nodes, and the expansion value of vertical expansion refers to the number of expanded layers.

[0172] In step S402, the running status of each candidate neural network in the test platform is obtained.

[0173] As an example, the step of obtaining the operating status of each candidate neural network in the test platform includes: obtaining the time consumed by each candidate neural network in processing the same set of reference data on the test platform, and the operating status includes the consumed time.

[0174] As an example, time consumption may refer to the time taken by the candidate neural network to process a set of reference data. Time consumption may also refer to the amount of reference data processed by the candidate neural network per unit time, that is, the speed at which the candidate neural network processes the reference data.

[0175] Optionally, the reference data may be image data, text data, voice data, etc. Taking the case where the reference data is image data as an example, each candidate neural network recognizes the same set of image data.

[0176] In an exemplary embodiment of the present invention, a set of image data is used as a set of reference data, and the number of image frames recognized by the candidate neural network within 1 second is used as the time consumption. The set of image data is recognized using the multiple candidate networks obtained in step S401, and the number of image frames recognized by each candidate network within 1 second is determined.

[0177] Referring to the figures, in Tables 1 to 4, the FPS in each table is the abbreviation of Frames Per Second, which means the number of image frames recognized by the candidate network within 1 second.

[0178] Curve (1) in Table 1 shows the number of image frames recognized by each first candidate network within 1 second when multiple first candidate neural networks process a group of images.

[0179] Curve (2) in Table 1 shows the number of image frames recognized by each second candidate network within 1 second when multiple second candidate neural networks process a group of images.

[0180] The curve in Table 2 shows the number of image frames recognized by each third candidate network within 1 second when multiple third candidate neural networks process a group of images.

[0181] The curve in Table 3 shows the number of image frames recognized by each fourth candidate network within 1 second when multiple fourth candidate neural networks process a group of images.

[0182] The curve in Table 4 shows the number of image frames recognized by each fifth candidate network within 1 second when multiple fifth candidate neural networks process a group of images.

[0183] In step S403, based on the network block structure in one or more preferred candidate neural networks of the running status, a construction rule of the network block is determined.

[0184] In Table 1, the decreasing rate of curve (2) is less than that of curve (1), which means that compared with the candidate neural network obtained by horizontal expansion of the initial neural network, the network block structure expanded vertically is the preferred network block structure for the processor. Therefore, "vertical expansion takes precedence over horizontal expansion" can be used as a network block construction rule.

[0185] In Table 2, when the single operation in the initial neural network is split into a number not exceeding 4 in parallel, the third candidate neural network obtained recognizes more image frames within 1 second; when the single operation in the initial neural network is split into a number greater than 4 in parallel, the third candidate neural network obtained recognizes fewer image frames within 1 second. In other words, the network block structure in which the number of single operations split into no more than 4 in parallel is the preferred network block structure for the processor, and therefore, "the number of single operations split into no more than 4 in parallel" can be used as a network block construction rule.

[0186] The curve in Table 3 is in a step shape. At the step of the curve, when the feature map changes, the number of image frames recognized by the fourth candidate neural network within 1 second remains basically unchanged, that is, the rate at which the fourth candidate neural network processes image data is relatively stable. The size of the feature map corresponding to each step in Table 3 is generally a multiple of 8 (8X), that is, the network block structure with a feature map size that is a multiple of 8 is the preferred network block structure for the processor, so "the size of the feature map is a multiple of 8" can be used as a construction rule for a network block.

[0187] The curve in Table 4 is in a step shape. At the step of the curve, when the number of channels changes, the number of image frames recognized by the fifth candidate neural network within 1 second remains basically unchanged, that is, the rate at which the fifth candidate neural network processes image data is relatively stable. The number of channels corresponding to each step in Table 4 is a multiple of 16 (16X), that is, the network block structure with a multiple of 16 channels is the preferred network block structure for the processor. Therefore, "a multiple of 16 channels" can be used as a construction rule for a network block.

[0188] Optionally, the construction rule of the network block includes at least one of the following:

[0189] Vertical expansion takes precedence over horizontal expansion;

[0190] The number of parallel splits of a single operation does not exceed 4;

[0191] The number of channels is a multiple of 16;

[0192] The size of the feature map is a multiple of 8.

[0193] It can be understood that the network block structure preferred for a processor is related to the type and parameters of the processor. For processors of different types or parameters, the network block structure may be different, and the construction rules of the network block may also be different.

[0194] In step S404, an initial second neural network architecture is obtained.

[0195] As an example, the initial second neural network architecture is constructed by a single network block; or, the initial second neural network architecture is constructed by stacking at least one network block.

[0196] The construction rules of the network block of the exemplary embodiment of the present invention include all the construction rules determined in step S403, that is, vertical expansion takes precedence over horizontal expansion, the number of parallel splits of a single operation does not exceed 4, the number of channels is a multiple of 16, and the size of the feature map is a multiple of 8.

[0197] Based on the construction rule of the network block determined in step S403, at least one candidate combination operation in the set of candidate combination operations is filled in each network block in different ways.

[0198] Each candidate combined operation in the set includes at least one basic operation among a plurality of preset basic operations. As an example, the basic operations include at least Conv_1x1, Conv_3x3, Conv_5x5, Conv_3x1_1x3, max_pool_3x3, and sep_conv_3x3, etc. Based on one or more groups of random seeds, the above-mentioned multiple basic operations are connected into at least one candidate combined operation.

[0199] In step S405, starting from the initial second neural network architecture, the next second neural network architecture is iteratively searched until the search is stopped when the second search termination condition is met.

[0200] When executing step S405, at least one of the number of network blocks in the second neural network architecture, the stacking method of the network blocks, and the filling method of the candidate operations in each network block can be determined as a variable factor through search.

[0201] The process of step S405 can be regarded as using NAS to search for filling of candidate combination operations in each network block in the initial second neural network architecture. It can be understood that the process of searching for the next second neural network architecture is actually a change to the initial second neural network architecture, that is, changing the type and / or number of candidate combination operations filled in at least one network block in the initial second neural network architecture, and training and verifying the initial second neural network architecture that has completed the change of candidate combination operations to obtain the next second neural network architecture, and determining the prediction error of the next second neural network architecture.

[0202] As an example, when training and verifying the initial second neural network architecture that has completed the candidate combination operation change, a part of a set of image data can be used as a training set and the other part as a verification set. The training set is used to train the model of the initial second neural network architecture that has completed the candidate combination operation change. After completing at least one training, the trained initial first neural network architecture is verified using the verification set.

[0203] Optionally, when a candidate combination operation is changed on the initial second neural network architecture, the specific manner of changing the current candidate combination operation can be determined based on the prediction error of the previous second neural network architecture.

[0204] Optionally, the second search termination condition can be determined according to actual design requirements. For example, satisfying the second search termination condition may mean that the number of searches has reached a preset number of times; or satisfying the second search termination condition may mean that the search duration has reached a preset duration; or, the prediction error of the searched second neural network architecture meets a preset error value condition.

[0205] It should be noted here that the preset error value conditions corresponding to the first neural network architecture and the second neural network architecture may be the same or different.

[0206] In step S406, the candidate combination operations in the searched one or more preferred second neural network architectures are determined as compound operations.

[0207] As an example, one or more second neural network architectures whose prediction errors meet the preset error value conditions are used as the preferred second neural network architecture. Figure 6 As shown, step S406 obtains a set of composite operations.

[0208] In step S407, an initial first neural network architecture is obtained.

[0209] In one embodiment, the initial first neural network architecture may include at least one stage, each stage includes at least one network block, and the network block can be regarded as a component unit of the first neural network architecture. Each network block is filled with at least one composite operation, and the composite operation is formed by connecting at least one basic operation.

[0210] As an example, multiple stages are sequentially distributed along the first direction, and each stage is connected to two adjacent stages along the first direction. Figure 7 For example, stage 1 to stage X are sequentially distributed along the first direction, stage 1 is connected to stage 2, stage 2 is connected to stage 3, and so on.

[0211] Optionally, when the initial first neural network architecture further includes a reduction layer, each reduction layer includes at least one network block, and the stages are connected via the reduction layer. Figure 7 For example, stage 1 to stage X are sequentially distributed along the first direction, stage 1 is connected to stage 2 via reduction layer 1, and stage 2 is connected to stage 3 via reduction layer 2.

[0212] Optionally, the number of network blocks included in each stage and each reduction layer can be preset according to actual design needs or learned in an iterative process. Taking a certain stage as an example, the stage can include only one network block; or the stage can include two or more network blocks.

[0213] like Figure 7 As shown, stage 1 includes n1 network blocks, stage 2 includes n2 network blocks, and so on. Stage 1 includes nx network blocks. Each reduction layer includes 1 network block. A neural network architecture has a total of (n1+n2+…nx+n-1) network blocks.

[0214] The number and type of network blocks in each stage (or reduction layer) can be the same or different. When a stage (or reduction layer) is filled with more than two network blocks, the types of any two network blocks in the stage (or reduction layer) can be the same or different.

[0215] As an example, the step of obtaining an initial first neural network architecture includes: obtaining an initial first neural network architecture including at least one stage, wherein each stage is stacked with at least one network block, and based on the construction rules of the network blocks, at least one composite operation is filled in the network block of each stage, and the composite operation is formed by connecting at least one basic operation.

[0216] Among them, the construction rules of the network block include all the construction rules determined in step S403, that is, vertical expansion takes precedence over horizontal expansion, the number of parallel splits of a single operation does not exceed 4, the number of channels is a multiple of 16, and the size of the feature map is a multiple of 8.

[0217] In step S408, starting from the initial first neural network architecture, the next first neural network architecture is iteratively searched based on the processor-related loss function until the search is stopped when the first search termination condition is met.

[0218] Among them, the processor-related loss function includes the prediction error part and the processor operation cost part in the model training process.

[0219] The process of step S408 includes: based on the processor-related loss function, using NAS to perform a filling search for each network block in the initial first neural network architecture and the composite operation therein. It can be understood that the process of searching for the next first neural network architecture is actually a change to the current first neural network architecture, that is, changing the number of stages of the current first neural network architecture, the connection method between stages, the number of network blocks in each stage, the stacking method of network blocks in each stage, and the filling method of composite operations in each network block. At least one of the following, and then training and verifying the replaced first neural network architecture.

[0220] When training and verifying each first neural network architecture in the search space, the training set can be used to perform model training on the candidate first neural network architecture. After completing at least one training, the trained model can be verified using the verification set.

[0221] Optionally, the first search termination condition may be determined according to actual design requirements. For example, satisfying the first search termination condition may mean that the number of searches has reached a preset number of times; or satisfying the first search termination condition may mean that the search duration has reached a preset duration; or, the value of the processor-related loss function corresponding to the searched first neural network architecture meets a preset numerical condition.

[0222] Optionally, during the process of training the model, it is necessary to continuously calculate the value of the corresponding processor-related loss function. The processor-related loss function includes a prediction error part and a processor operation cost part during the model training process, wherein the prediction error part can be obtained when the trained model is verified using a verification set, and the processor operation cost part can include a time consumption hyperparameter generated by the processor accessing DRAM during the model training process. A method for obtaining the time consumption hyperparameter is referred to the previous content and will not be repeated here.

[0223] In step S409, the searched optimal first neural network architecture is determined as the neural network architecture to be used on the processor.

[0224] As an example, the optimal first neural network architecture may be the first neural network architecture whose loss function corresponds to an optimal value.

[0225] Fig. 9 A block diagram of a search device for determining a neural network architecture for a processor according to an exemplary embodiment of the present invention is shown. The functional units of the search device for determining a neural network architecture for a processor can be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present invention. It can be understood by those skilled in the art that Fig. 9The functional units described may be combined or divided into sub-units to implement the principles of the above invention. Therefore, the description herein may support any possible combination, division, or further limitation of the functional units described herein.

[0226] The following is a brief description of the functional units that a search device for determining a neural network architecture for a processor may have and the operations that each functional unit may perform. For the details involved, please refer to the relevant description above and will not be repeated here.

[0227] Reference Fig. 9 , a search device for determining a neural network architecture for a processor according to an exemplary embodiment of the present invention includes an initial architecture acquisition module 510 , a target architecture acquisition module 520 and a target architecture confirmation module 530 .

[0228] The initial architecture acquisition module 510 is configured to acquire an initial first neural network architecture; the target architecture acquisition module 520 is configured to start from the initial first neural network architecture and iteratively search for the next first neural network architecture based on the processor-related loss function until the search is stopped when the first search termination condition is met; the target architecture confirmation module 530 is configured to determine the searched optimal first neural network architecture as the neural network architecture to be used on the processor; wherein the processor-related loss function includes the prediction error part and the processor operation cost part in the model training process.

[0229] In one embodiment of the present invention, the processor operation cost portion includes a time consumption hyperparameter and / or a power consumption hyperparameter for accessing DRAM during the processor execution model training process.

[0230] In one embodiment of the present invention, the initial architecture acquisition module 510 is configured to: acquire an initial first neural network architecture including at least one stage, wherein each stage is stacked with at least one network block, and based on the construction rules of the network blocks, at least one composite operation is filled in the network block of each stage, and the composite operation is formed by connecting at least one basic operation.

[0231] In one embodiment of the present invention, the search device also includes a compound operation generation module 540, which is configured to: obtain an initial second neural network architecture including at least one network block, wherein, based on the construction rules of the network blocks, each network block of the initial second neural network architecture is filled with at least one candidate combination operation in a set of candidate combination operations, and each candidate combination operation in the set includes at least one basic operation among multiple preset basic operations; starting from the initial second neural network architecture, iteratively search for the next second neural network architecture until the search is stopped when the second search termination condition is met; determine the candidate combination operations in the searched one or more preferred second neural network architectures as compound operations.

[0232] In one embodiment of the present invention, the search device also includes a construction rule determination module 550, which is configured to: determine the construction rules of the network block according to manual settings; and / or determine the construction rules of the network block based on the network block structure preferred by the processor.

[0233] In one embodiment of the present invention, the construction rule determination module 550 is configured to: on a test platform of a processor, transform an initial neural network according to at least one transformation method to obtain a plurality of candidate neural networks, wherein the initial neural network includes at least one network block structure; obtain the operating status of each candidate neural network in the test platform; and determine the construction rules of the network blocks based on the network block structures in one or more preferred candidate neural networks of the operating status.

[0234] In one embodiment of the present invention, the construction rule determination module 550 is configured to obtain the time consumed by each candidate neural network in processing the same set of reference data on the test platform, and the operating status includes the consumed time.

[0235] In one embodiment of the present invention, the construction rule determination module 550 is configured to perform at least one of the following steps:

[0236] Expand the initial neural network horizontally;

[0237] Expand the initial neural network vertically;

[0238] Split the individual operations in the initial neural network into parallel operations;

[0239] Changing the size of the feature map of the initial neural network;

[0240] Change the number of channels of the initial neural network.

[0241] In one embodiment of the present invention, the construction rule of the network block includes at least one of the following:

[0242] The priority relationship between vertical expansion and horizontal expansion;

[0243] The number of times a single operation is split into parallel operations;

[0244] Number of channels;

[0245] The size of the feature map.

[0246] Fig.10 A block diagram of a computing device according to an exemplary embodiment of the present invention is shown.

[0247] Reference Fig.10 According to an exemplary embodiment of the present invention, a computing device is also provided, and the computing device includes a processor 610 and a memory 620. The memory 620 stores a computer program, and when the computer program is executed by the processor 610, the above-mentioned search method for determining a neural network architecture for the processor is implemented.

[0248] The processor 610 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 610 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0249] The memory 620 can be a ROM (Read-Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read-Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0250] According to an exemplary embodiment of the present invention, a computer-readable storage medium storing instructions is also provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the above-mentioned search method for determining a neural network architecture for a processor. The computer-readable recording medium is any data storage device that can store data read by a computer system. Examples of computer-readable recording media include: read-only memory, random access memory, read-only optical disk, magnetic tape, floppy disk, optical data storage device, and carrier wave (such as data transmission through the Internet via a wired or wireless transmission path).

[0251] The above describes the exemplary embodiments of the present application. It should be understood that the above description is only exemplary and not exhaustive, and the present application is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present application, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present application should be based on the scope of the claims.

Claims

1. A search method for determining a neural network architecture for a processor, comprising: Obtaining an initial first neural network architecture; Starting from the initial first neural network architecture, iteratively searching for a next first neural network architecture based on a processor-dependent loss function until the search is stopped when a first search termination condition is satisfied; determining the searched optimal first neural network architecture as the neural network architecture to be used on the processor, Wherein, the processor-related loss function includes a prediction error part and a processor operation cost part in the model training process; The method of starting from the initial first neural network architecture and iteratively searching for the next first neural network architecture based on the processor-related loss function includes: Evaluate each candidate first neural network architecture in the search space using the processor-dependent loss function; Wherein, the processor operation cost part includes a time consumption hyperparameter and / or a power consumption hyperparameter of accessing DRAM during the processor execution model training process; The initial first neural network architecture includes at least one composite operation, and the composite operation is determined by the following operations: Acquire an initial second neural network architecture including at least one network block, wherein, based on a construction rule of the network block, each network block of the initial second neural network architecture is filled with at least one candidate combination operation in a set of candidate combination operations, and each candidate combination operation in the set includes at least one basic operation among a plurality of preset basic operations; Starting from the initial second neural network architecture, iteratively searching for the next second neural network architecture until the search is stopped when a second search termination condition is met; The candidate combination operations in the searched one or more preferred second neural network architectures are determined as composite operations.

2. The search method according to claim 1, wherein: The step of obtaining an initial first neural network architecture includes: obtaining an initial first neural network architecture including at least one stage, wherein each stage is stacked with at least one network block, and based on a construction rule of the network block, the network block of each stage is filled with the at least one composite operation.

3. The search method according to claim 1 or 2, further comprising: Determine the construction rules of network blocks based on human settings; And / or, determining a construction rule of a network block based on a network block structure preferred for the processor.

4. The search method according to claim 3, wherein: The step of determining a construction rule of a network block based on a network block structure preferred for the processor comprises: On a test platform of the processor, transforming an initial neural network according to at least one transformation mode to obtain a plurality of candidate neural networks, wherein the initial neural network includes at least one network block structure; Obtaining the running status of each of the candidate neural networks in the test platform; Determine a construction rule of a network block based on one or more preferred network block structures in the candidate neural network according to the operating conditions; The step of transforming the initial neural network in at least one transformation mode comprises at least one of the following steps: Expanding the initial neural network horizontally; Vertically expanding the initial neural network; Splitting the individual operations in the initial neural network in parallel; Changing the size of the feature map of the initial neural network; The number of channels of the initial neural network is changed.

5. The search method according to claim 4, wherein: The step of obtaining the operating status of each candidate neural network in the test platform includes: The time consumed by each of the candidate neural networks in processing the same set of reference data on the test platform is obtained, and the operating status includes the consumed time.

6. The search method according to claim 4, wherein: The construction rules of the network block are set from at least one of the following aspects: The priority relationship between vertical expansion and horizontal expansion; The number of times a single operation is split into parallel operations; Number of channels; The size of the feature map.

7. A search device for determining a neural network architecture for a processor, comprising: An initial architecture acquisition module, configured to acquire an initial first neural network architecture; A target architecture acquisition module is configured to iteratively search for a next first neural network architecture based on a processor-related loss function starting from an initial first neural network architecture until the search is stopped when a first search termination condition is met; a target architecture confirmation module, configured to determine the searched optimal first neural network architecture as the neural network architecture to be used on the processor; Wherein, the processor-related loss function includes a prediction error part and a processor operation cost part in the model training process; Wherein, the target architecture acquisition module is configured as follows: Evaluate each candidate first neural network architecture in the search space using the processor-dependent loss function; Wherein, the processor operation cost part includes a time consumption hyperparameter and / or a power consumption hyperparameter of accessing DRAM during the processor execution model training process; The initial first neural network architecture includes at least one composite operation, and the device further includes a composite operation generation module, wherein the composite operation generation module is configured to: Acquire an initial second neural network architecture including at least one network block, wherein, based on a construction rule of the network block, each network block of the initial second neural network architecture is filled with at least one candidate combination operation in a set of candidate combination operations, and each candidate combination operation in the set includes at least one basic operation among a plurality of preset basic operations; Starting from the initial second neural network architecture, iteratively searching for the next second neural network architecture until the search is stopped when a second search termination condition is met; The candidate combination operations in the searched one or more preferred second neural network architectures are determined as composite operations.

8. The search device according to claim 7, wherein: The initial architecture acquisition module is configured as follows: An initial first neural network architecture including at least one stage is obtained, wherein each stage is stacked with at least one network block, and based on a construction rule of the network block, the network block of each stage is filled with the at least one composite operation.

9. The search device according to claim 7 or 8, wherein: It also includes a construction rule determination module, wherein the construction rule determination module is configured to: The construction rules of the network block are determined according to artificial settings; and / or the construction rules of the network block are determined based on the network block structure preferred for the processor.

10. The search device according to claim 9, wherein: The construction rule determination module is configured to: On a test platform of the processor, transforming an initial neural network according to at least one transformation mode to obtain a plurality of candidate neural networks, wherein the initial neural network includes at least one network block structure; Obtaining the running status of each of the candidate neural networks in the test platform; Determine a construction rule of a network block based on one or more preferred network block structures in the candidate neural network according to the operating conditions; The construction rule determination module is configured to perform at least one of the following steps: Expanding the initial neural network horizontally; Vertically expanding the initial neural network; Splitting the individual operations in the initial neural network in parallel; Changing the size of the feature map of the initial neural network; The number of channels of the initial neural network is changed.

11. The search device according to claim 10, wherein: The construction rule determination module is configured to: The time consumed by each of the candidate neural networks in processing the same set of reference data on the test platform is obtained, and the operating status includes the consumed time.

12. The search device according to claim 10, wherein: The construction rules of the network block are set from at least one of the following aspects: The priority relationship between vertical expansion and horizontal expansion; The number of times a single operation is split into parallel operations; Number of channels; The size of the feature map.

13. A computing device, characterized in that: The computing device comprises: processor; A memory storing a computer program, which, when executed by a processor, implements a search method for determining a neural network architecture for a processor as described in any one of claims 1 to 6.

14. A computer-readable storage medium storing instructions, wherein: When the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the search method for determining a neural network architecture for a processor as claimed in any one of claims 1 to 6.

Images