Neural network searching method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a neural network search method, device, electronic equipment and storage medium. The method comprises: constructing a super network based on a first search space, wherein the first search space comprises a depth separable convolution module, the depth separable convolution module comprises a plurality of first convolution branches with different convolution kernel sizes, and the plurality of first convolution branches are respectively cascaded with a first batch normalization layer, and the output of each first batch normalization layer is connected to a next operation layer; iteratively training the super network based on a training sample until the super network meets a convergence condition; and enabling the search space to extract features with different receptive field sizes from the same tensor, searching a target subnetwork that meets a deployment condition from the trained super network, thereby solving the problem that the search space of the related art can only extract features with a fixed receptive field size, and improving the performance of the model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a neural network search method, apparatus, electronic device, and storage medium. Background Technology

[0002] Neural Architecture Search (NAS) is a neural network search method based on weight sharing. This method first constructs a supernet using a specific search space, then trains the supernet. Once the supernet is fully trained, it searches for target subnets that meet deployment criteria, and finally deploys the obtained target subnets to the production environment. However, related techniques can only extract features with a fixed receptive field size, limiting the model's performance. Summary of the Invention

[0003] In view of this, the present disclosure provides a neural network search method, apparatus, electronic device and storage medium, so that the search space can extract features of the same tensor with different receptive field sizes.

[0004] According to one aspect of this disclosure, a neural network search method is provided, comprising: constructing a supernet based on a first search space, wherein the first search space includes a depthwise separable convolutional module, the depthwise separable convolutional module includes multiple first convolutional branches with different kernel sizes, and the multiple first convolutional branches are respectively cascaded with a first batch of normalized layers, and the outputs of each of the first batch of normalized layers are merged and connected to the next operating layer; iteratively training the supernet based on training samples until the supernet meets the convergence condition; and searching for a target subnet that meets the deployment conditions from the trained supernet.

[0005] In one optional implementation, constructing a supernet based on a first search space includes: merging the plurality of first convolutional branches with the first batch of cascaded normalized layers to obtain a plurality of second convolutional branches, and merging the plurality of second convolutional branches to obtain a third convolutional branch.

[0006] In one optional implementation, merging the plurality of second convolutional branches to obtain a third convolutional branch includes: padding the convolutional kernel matrices of each of the second convolutional branches to ensure that the convolutional kernel sizes of the plurality of second convolutional branches are consistent; and adding the convolutional kernel matrices of the plurality of second convolutional branches after the boundary padding to obtain the third convolutional branch.

[0007] In one alternative implementation, the plurality of first convolutional branches include at least two of the following kernel sizes: 1, 3, 5, and 7.

[0008] In one optional implementation, the first search space further includes: a first feature extraction unit and a second feature extraction unit, wherein the first feature extraction unit, the depthwise separable convolution module, and the second feature extraction unit are connected in sequence; wherein the first feature extraction unit includes a first convolutional layer, a second batch normalization layer, and a first activation layer connected in sequence; and the second feature extraction unit includes a second activation layer and an attention module connected in sequence.

[0009] In one optional implementation, the first search space further includes: a skip connection layer carrying a batch normalization operation, wherein the skip connection layer is disposed in parallel with the depthwise separable convolution module between the first feature extraction unit and the second feature extraction unit.

[0010] In one optional implementation, constructing a supernet based on a first search space includes: merging the skip connection layer with multiple second convolutional branches to obtain a third convolutional branch, wherein the multiple second convolutional branches are obtained by merging the multiple first convolutional branches with the first batch of cascaded normalized layers respectively.

[0011] In one optional implementation, the first search space further includes: a third feature extraction unit, the input of which is connected to the output of the second feature extraction unit, the third feature extraction unit including a second convolutional layer and a third batch normalization layer connected in sequence; constructing a supernet based on the first search space includes: adding the output feature map of the third feature extraction unit to the input feature map of the first feature extraction unit to obtain the output feature map of the first search space.

[0012] In one optional implementation, searching for a target subnet that meets the deployment conditions from a fully trained supernet includes: selecting a subnet whose bit width and convolutional kernel size meet the deployment conditions as the target subnet.

[0013] According to another aspect of this disclosure, a neural network search apparatus is provided, comprising: a construction module for constructing a supernet based on a first search space, wherein the first search space includes a depthwise separable convolutional module, the depthwise separable convolutional module includes multiple first convolutional branches with different kernel sizes, and the multiple first convolutional branches are respectively cascaded with a first batch of normalized layers, the outputs of each of the first batch of normalized layers are merged and connected to the next operating layer; a training module for iteratively training the supernet based on training samples until the supernet satisfies a convergence condition; and a search module for searching for a target subnet that satisfies the deployment conditions from the trained supernet.

[0014] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the neural network search method described in any of the preceding aspects.

[0015] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the neural network search method described in any of the preceding aspects.

[0016] One or more technical solutions provided in this disclosure construct a supernet based on a first search space. The first search space includes a depthwise separable convolutional module, which comprises multiple first convolutional branches with different kernel sizes. These multiple first convolutional branches are cascaded with a first batch of normalized layers, and the outputs of each first batch of normalized layers are merged and connected to the next operational layer. The supernet is iteratively trained based on training samples until it meets the convergence condition. This enables the search space to extract features of the same tensor with different receptive field sizes, searching for target subnets that meet the deployment conditions from the fully trained supernet. This solves the problem that related technologies can only extract features with fixed receptive field sizes, thus improving model performance. Attached Figure Description

[0017] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0018] Figure 1 This diagram illustrates the structure of the search space in related technologies.

[0019] Figure 2 A flowchart of a neural network search method according to an exemplary embodiment of the present disclosure is shown;

[0020] Figure 3 A schematic diagram of the structure of the first search space according to an exemplary embodiment of the present disclosure is shown. Figure 1 ;

[0021] Figure 4 A schematic diagram of the structure of the first search space according to an exemplary embodiment of the present disclosure is shown. Figure 2 ;

[0022] Figure 5 A schematic diagram of the structure of the first search space according to an exemplary embodiment of the present disclosure is shown. Figure 3 ;

[0023] Figure 6 A schematic diagram showing the merging of a first convolutional branch with a cascaded batch normalized layer according to an exemplary embodiment of the present disclosure is shown;

[0024] Figure 7 A schematic diagram of the second convolutional branch merging according to an exemplary embodiment of the present disclosure is shown;

[0025] Figure 8 A schematic diagram of the structure of a neural network search apparatus according to an exemplary embodiment of the present disclosure is shown;

[0026] Figure 9 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0030] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0032] See Figure 1The diagram shows the structure of the search space in the relevant technology. This search space is implemented based on MBConv (MobileNetConv), and includes: a first feature extraction unit, a depth-wise convolutional layer, a batch normalization layer (BN), an activation layer, an attention module (Squeeze & Excite), and a second feature extraction unit connected in sequence. The first feature extraction unit includes a convolutional layer, a batch normalization layer, and an activation layer connected in sequence; the second feature extraction unit includes a convolutional layer and a batch normalization layer connected in sequence. In this search space, `in_channels` represents the number of channels in the input tensor of the current structure, `out_channels` represents the number of channels in the output tensor of the current structure, `a` represents the dilation coefficient of the input tensor channels, and `k` represents the kernel size of the depth-wise convolution. Typically, `a` is set to [3, 6], and `k` is set to [3, 5, 7]. The search space achieves subnet search in the dimensions of model bit width and kernel size by controlling the two parameters `a` and `k`.

[0033] Research has revealed that within the aforementioned search space, the convolutional kernel corresponding to each channel-wise convolutional layer can only be one of [3, 5, 7]. This means that a fixed kernel size can only extract features with a fixed receptive field size, thus limiting model performance. Therefore, in one embodiment of this disclosure, a neural network search method is provided that improves the search space used to construct supernets by increasing the branching structure of the search space, thereby enhancing subnet performance.

[0034] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0035] To facilitate understanding of this embodiment, a detailed description of the neural network search method disclosed in this disclosure is provided first. The execution entity of the neural network search method provided in this disclosure is generally a computer device with a certain computing capability. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this neural network search method can be implemented by a processor calling computer-readable instructions stored in memory.

[0036] The neural network search method provided in the embodiments of this disclosure will be described below. See also Figure 2 The diagram shown is a flowchart of a neural network search method provided in an embodiment of this disclosure. The process includes the following steps:

[0037] Step S101: Construct a supernet based on the first search space, wherein the first search space includes a depthwise separable convolution module, the depthwise separable convolution module includes multiple first convolution branches with different kernel sizes, and the multiple first convolution branches are respectively cascaded with the first batch of normalized layers, and the outputs of each first batch of normalized layers are merged and connected to the next operation layer.

[0038] A supernet refers to a super network built upon a first search space. Each layer of a supernet has multiple selectable subnets, all of which share parameters during construction. The first search space can be implemented using MBConv, an inverted linear bottleneck structure with depthwise separable convolutions. The first convolutional branch refers to channel-wise convolution. In this step, the MBConv technique is improved by adding multiple first convolutional branches with different kernel sizes to the depthwise separable convolution module to support feature extraction from multiple receptive fields.

[0039] Optionally, the first search space also includes other operational layers; see [link to relevant documentation]. Figure 3 The diagram shown is a structural illustration of a first search space provided in an embodiment of this disclosure. Figure 1 The first search space includes: a first feature extraction unit, a depthwise separable convolutional module, a second feature extraction unit, and a third feature extraction unit connected in sequence. The first feature extraction unit includes a first convolutional layer, a second batch normalization layer, and a first activation layer connected in sequence; the second feature extraction unit includes a second activation layer and an attention module connected in sequence; the third feature extraction unit includes a second convolutional layer and a third batch normalization layer connected in sequence; the depthwise separable convolutional module includes multiple first convolutional branches with different kernel sizes, and each of the multiple first convolutional branches is cascaded with a first batch normalization layer, with the outputs of each first batch normalization layer merged and connected to the second feature extraction unit. Further, the first search space also includes a skip connection layer carrying batch normalization operations, which is arranged side-by-side with the depthwise separable convolutional module between the first and second feature extraction units. This arrangement creates residual connections within the first search space, effectively improving the performance of deep network structures. Optionally, the multiple first convolutional branches include at least two of the following kernel sizes: 1, 3, 5, and 7. Figure 3 In the first search space, there are multiple first convolutional branches with kernel sizes of 1, 3, 5, and 7. Optionally, the kernel sizes of the multiple first convolutional branches in the first search space can also be 1 and 3 or 1, 3, and 5.

[0040] The following describes the image data processing process of the first search space. In the first feature extraction unit, the initial feature map is first processed by a first convolutional layer with a size and stride of 1 and a channel number of a multiplied by in_channels. Then, a second batch of normalization layers is used to normalize the feature map. After that, the activation function in the first activation layer is used to activate the feature map output by the second batch of normalization layers. After a series of operation layers in the first feature extraction unit, the obtained feature map is input to the depthwise separable convolution module. The depthwise separable convolution module includes multiple first convolutional branches with different kernel sizes. Each first convolutional branch is implemented using channel-wise convolution, and the number of channels is the same as the number of channels in the previous convolutional kernel. After channel-wise convolution, the output feature map is processed by the cascaded first batch of normalization layers, and the normalized feature map is added to the feature map output by the skip connection layer. The summed feature map is input to the second feature extraction unit and processed by the second activation layer and the attention module. Finally, the output feature map is input into the third feature extraction unit. A convolutional kernel with a stride of 1, a size of 1, and out channels is used to integrate the feature maps output by the previous operation layers. The integrated feature map is then batch-normalized. Furthermore, to prevent gradient vanishing, a residual block is added, which adds the output feature map of the third feature extraction unit to the initial feature map input to the first feature extraction unit to obtain the output feature map of the first search space. Both the first and second activation layers can use the swish activation function.

[0041] Step S102: Iteratively train the supernet based on the training samples until the supernet meets the convergence condition.

[0042] When the supernet is iteratively trained to meet a preset convergence condition, a sampled subnet is used for performance evaluation, eliminating the need to retrain the subnet. For example, during supernet training, a single path can be selected using a uniform path sampling method until all subnets in the supernet converge. Since subnets in the supernet share weights, the training time for the supernet is essentially the same as training a single subnet individually. Within this training framework, the accuracy of subnets within the supernet can be improved by enhancing sampling fairness or increasing the number of subnets sampled at each step, bringing them closer to the true accuracy and thus improving the consistency of the supernet.

[0043] Step S103: Search for target subnets that meet the deployment conditions from the fully trained supernet.

[0044] The target subnet refers to the neural network model to be deployed on a pre-defined device. The deployment conditions include network accuracy and computational load conditions set based on the hardware resources of the pre-defined device and the business scenario. Optionally, after the supernet is fully trained, a suitable bit width and kernel size are determined based on the deployment conditions to ensure that the bit width and kernel size of the subnet meet the deployment conditions. Then, the target subnet is determined based on these two dimensions: the suitable bit width and kernel size.

[0045] Based on the above steps S101 to S103, the first search space is improved by adding first convolution branches with different kernel sizes to the first search space, so that the first search space can extract features with different receptive field sizes for the same tensor. This increases the branch structure of the first search space, solves the problem that the search space can only extract features with a fixed receptive field size in related technologies, and thus improves the performance of the model.

[0046] After using the improved first search space described above for supernet construction, supernet training, and subnet search, subnets containing branches with convolutional kernels of different sizes are obtained. Such a model structure suffers from long inference time and a large number of model parameters. To address this issue, in one embodiment, during supernet construction, ... Figure 3 Based on this, multiple first convolutional branches are merged with the first batch of cascaded normalized layers to obtain multiple second convolutional branches (see...). Figure 4 The diagram shown is a structural illustration of a first search space provided in an embodiment of this disclosure. Figure 2 ), and merge multiple second convolution branches to obtain the third convolution branch (see Figure 5 The diagram shown is a structural illustration of a first search space provided in an embodiment of this disclosure. Figure 3 ).exist Figure 4 In this context, the BN layer (1×1 special depth-wise conv), which is parallel to the channel-wise convolution, can be regarded as a special depth-wise convolution with a kernel size of 1.

[0047] The process involves merging multiple second convolutional branches to obtain a third convolutional branch. This includes: padding the convolutional kernel matrices of each second convolutional branch to ensure consistent kernel sizes; and summing the padded kernel matrices of the second convolutional branches to obtain the third convolutional branch. Further, the skip connection layer is merged with the multiple second convolutional branches to obtain the third convolutional branch. The multiple second convolutional branches are obtained by merging them with the first batch of cascaded normalized layers, each via a first convolutional branch. This configuration, during the model inference phase, merges branches with different kernel sizes in the first search space, resulting in a first search space containing only the identity mapping branch and the main branch. This achieves improved inference speed and reduced model parameters without changing model accuracy.

[0048] To more clearly illustrate the construction process of this public supernet, see [link to relevant documentation]. Figure 6 The diagram illustrates the merging of a first convolutional branch and a cascaded batch normalized layer according to an embodiment of this disclosure. All first convolutional branches and the first batch of cascaded normalized layers are merged to obtain multiple second convolutional branches. For example, for a convolutional kernel with a size of 2 and a channel count of 2, its weight parameter is ω, and its bias is b; during inference, the calculation formula for the first batch of normalized layers is as follows:

[0049]

[0050] Where α and β are learnable scaling and offset values, mean and val are the moving averaged feature mean and feature variance processed by the first batch of normalization layers, and ε is the minimum value. Since these parameters remain constant during inference, the first batch of normalization layers can be considered as having a size of 1 and weights of 1. Offset is The two operators above can be combined into a single convolution operation due to the additivity of convolution.

[0051] See Figure 7 The diagram illustrates a second convolution branch merging method provided in this embodiment. The convolution kernel matrix of the second convolution branch is padded. For example, the channel-wise convolution kernel weights with kernel sizes of 1, 3, and 5 are padded with 3, 2, and 1 zero elements respectively, resulting in convolution kernel weights with a kernel size of 7. Finally, all convolution kernel weights with a kernel size of 7 are added together to merge the convolution kernels of different sizes, i.e., to merge multiple second convolution branches, resulting in a first search space with a kernel size of 7.

[0052] Figure 7The diagram illustrates the merging of multiple second convolutional branches with kernel sizes of 1, 3, 5, and 7 in the first search space. In some embodiments, when the kernel sizes of the multiple second convolutional branches in the first search space are 1 and 3, the channel-wise convolutional kernel weights with a kernel size of 1 can be padded with one zero element around them to obtain convolutional kernel weights with a kernel size of 3. Then, the convolutional kernel weights with a kernel size of 3 are added together to merge the convolutional kernels of different sizes, resulting in a first search space with a kernel size of 3. In some embodiments, when the kernel sizes of the multiple second convolutional branches in the first search space are 1, 3, and 5, the channel-wise convolutional kernel weights with kernel sizes of 1 and 3 can be padded with 2 and 1 zero elements around them to obtain convolutional kernel weights with a kernel size of 5. Then, the convolutional kernel weights with a kernel size of 5 are added together to merge the convolutional kernels of different sizes, resulting in a first search space with a kernel size of 5.

[0053] In conjunction with the neural network search method of the above embodiments, this embodiment also provides a neural network search device. Figure 8 This is a schematic diagram of the neural network search device in this embodiment, as shown below. Figure 8 As shown, the device includes:

[0054] The supernet is constructed based on a first search space, which includes a depthwise separable convolutional module. The depthwise separable convolutional module includes multiple first convolutional branches with different kernel sizes, and the multiple first convolutional branches are cascaded with a first batch of normalized layers. The outputs of each first batch of normalized layers are merged and connected to the next operating layer. The training module is used to iteratively train the supernet based on training samples until the supernet meets the convergence condition. The search module is used to search for a target subnet that meets the deployment conditions from the trained supernet.

[0055] The processing flow of each module in the device and the interaction flow between modules can be described in the relevant descriptions in the above method embodiments, and will not be elaborated here. It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0056] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform a method according to an embodiment of this disclosure.

[0057] See Figure 9 The diagram shown is a structural block diagram of an electronic device provided as an embodiment of this disclosure. It is an example of a hardware device that can be applied to various aspects of this disclosure, such as serving as a server or client. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the disclosure described and / or claimed herein.

[0058] like Figure 9 As shown, the electronic device 200 includes a computing unit 201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 202 or a computer program loaded from a storage unit 208 into a random access memory (RAM) 203. The RAM 203 may also store various programs and data required for the operation of the device 200. The computing unit 201, ROM 202, and RAM 203 are interconnected via a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0059] Multiple components in electronic device 200 are connected to I / O interface 205, including: input unit 206, output unit 207, storage unit 208, and communication unit 209. Input unit 206 can be any type of device capable of inputting information to electronic device 200. Input unit 206 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 207 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 208 may include, but is not limited to, disks and optical discs. Communication unit 209 allows electronic device 200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0060] The computing unit 201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 201 performs the various methods and processes described above. For example, in some embodiments, the neural network search method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 208. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 200 via ROM 202 and / or communication unit 209. In some embodiments, the computing unit 201 can be configured to perform the neural network search method by any other suitable means (e.g., by means of firmware).

[0061] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to embodiments of this disclosure.

[0062] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0063] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0064] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0065] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0066] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0067] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0068] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A neural network search method, characterized in that, include: A supernet is constructed based on a first search space, wherein the first search space includes a first feature extraction unit, a second feature extraction unit, a depthwise separable convolutional module, and a third feature extraction unit. The depthwise separable convolutional module includes multiple first convolutional branches with different kernel sizes, and the multiple first convolutional branches are respectively cascaded with a first batch of normalization layers. The outputs of each first batch of normalization layers are merged and connected to the next operation layer. The first feature extraction unit, the depthwise separable convolutional module, and the second feature extraction unit are connected in sequence. The input end of the third feature extraction unit is connected to the output end of the second feature extraction unit. The third feature extraction unit includes a second convolutional layer and a third batch of normalization layers connected in sequence. The supernet is iteratively trained based on the training samples until the supernet meets the convergence condition. Search for target subnets that meet the deployment conditions from the fully trained supernet. The deployment conditions include network accuracy conditions and computational load conditions set based on the hardware resources of the preset device and the business scenario. The preset device is a computer device. The process of processing image data using the first search space includes: In the first feature extraction unit, the initial feature map is processed through the first convolutional layer for feature extraction, the feature map is normalized using the second batch normalization layer, and the feature map output by the second batch normalization layer is activated using the activation function in the first activation layer. The feature map obtained after processing by the first feature extraction unit is input into the depthwise separable convolution module for channel-wise convolution operation to output the feature map. The depthwise separable convolution module includes multiple first convolution branches with different convolution kernel sizes, and each first convolution branch is implemented by channel-wise convolution. The feature map output after being processed by the depthwise separable convolution module is passed to the first batch of cascaded normalization layers for processing. The normalized feature map and the feature map output by the skip connection layer are added together to obtain the summed feature map. The summed feature map is input into the second feature extraction unit, processed by the second activation layer and attention module, and the output feature map is input into the third feature extraction unit. The feature map output by the operation layer is integrated using a convolution kernel, and the integrated feature map is batch normalized. The output feature map of the third feature extraction unit is added to the input feature map of the first feature extraction unit to obtain the output feature map of the first search space.

2. The neural network search method as described in claim 1, characterized in that, Constructing a supernet based on the first search space, including: The multiple first convolutional branches are merged with the first batch of cascaded normalized layers to obtain multiple second convolutional branches, and the multiple second convolutional branches are merged to obtain a third convolutional branch.

3. The neural network search method as described in claim 2, characterized in that, The multiple second convolutional branches are merged to obtain a third convolutional branch, including: The kernel matrix of each of the second convolutional branches is padded to ensure that the kernel size of the multiple second convolutional branches is consistent. The third convolutional branch is obtained by adding the convolution kernel matrices of the multiple second convolutional branches after boundary padding.

4. The neural network search method according to any one of claims 1 to 3, characterized in that, The plurality of first convolutional branches include at least two of the following kernel sizes: 1, 3, 5, and 7.

5. The neural network search method as described in any one of claims 1 to 3, characterized in that, The first feature extraction unit includes a first convolutional layer, a second batch normalization layer, and a first activation layer connected in sequence; The second feature extraction unit includes a second activation layer and an attention module connected in sequence.

6. The neural network search method as described in claim 5, characterized in that, The first search space further includes: a skip connection layer carrying batch normalization operation, wherein the skip connection layer and the depthwise separable convolution module are arranged side by side between the first feature extraction unit and the second feature extraction unit.

7. The neural network search method as described in claim 6, characterized in that, Constructing a supernet based on the first search space, including: The skip connection layer is merged with multiple second convolutional branches to obtain a third convolutional branch, wherein the multiple second convolutional branches are obtained by merging the multiple first convolutional branches with the first batch of cascaded normalization layers respectively.

8. The neural network search method as described in claim 1, characterized in that, Search for target subnets that meet the deployment conditions from the fully trained supernet, including: The subnet whose bit width and convolution kernel size meet the deployment conditions is taken as the target subnet.

9. A neural network search device, characterized in that, include: A construction module is used to construct a supernet based on a first search space, wherein the first search space includes a first feature extraction unit, a second feature extraction unit, a depthwise separable convolutional module, and a third feature extraction unit. The depthwise separable convolutional module includes multiple first convolutional branches with different kernel sizes, and the multiple first convolutional branches are respectively cascaded with a first batch of normalization layers. The outputs of each first batch of normalization layers are merged and connected to the next operation layer. The first feature extraction unit, the depthwise separable convolutional module, and the second feature extraction unit are connected in sequence. The input end of the third feature extraction unit is connected to the output end of the second feature extraction unit. The third feature extraction unit includes a second convolutional layer and a third batch of normalization layers connected in sequence. The training module is used to iteratively train the supernet based on training samples until the supernet meets the convergence condition. The search module is used to search for target subnets that meet the deployment conditions from the fully trained supernet. The deployment conditions include network accuracy conditions and computational load conditions set based on the hardware resources of the preset device and the business scenario. The preset device is a computer device. The image data processing using the first search space includes: in the first feature extraction unit, the initial feature map is processed through a first convolutional layer for feature extraction; the feature map is normalized using a second batch normalization layer; and the feature map output from the second batch normalization layer is activated using the activation function in the first activation layer. The feature map obtained after processing by the first feature extraction unit is then input into the depthwise separable convolution module for channel-wise convolution to output a feature map. The depthwise separable convolution module includes multiple first convolution branches with different kernel sizes, each implemented using channel-wise convolution. The image data processed by the depthwise separable convolution module is then processed. The feature map output by the module is processed by the first batch of cascaded normalization layers. The normalized feature map is then added to the feature map output by the skip connection layer to obtain a summed feature map. The summed feature map is then input to the second feature extraction unit, where it is processed by the second activation layer and the attention module. The output feature map is then input to the third feature extraction unit, where convolutional kernels are used to integrate the feature maps output by the operation layer, and the integrated feature map is batch normalized. Finally, the output feature map of the third feature extraction unit is added to the input feature map of the first feature extraction unit to obtain the output feature map of the first search space.

10. An electronic device, characterized in that, include: processor; And a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 8.

Images