Network structure searching method and device, terminal equipment and storage medium

Abstract

Embodiments of the present application provide a network structure search method and device, terminal equipment and storage medium. The network structure search method comprises: initializing a network architecture, the network architecture comprising N networks, a first network comprising M candidate sub-networks, the number of nodes of any two sub-networks in the M candidate sub-networks being different from each other, the first network being any one of the N networks, N being an integer greater than or equal to 1, and M being an integer greater than or equal to 2; searching for a sub-network from the first network in a hierarchical search manner, and constructing a target network structure based on the searched N sub-networks. The embodiments of the present application can improve the performance of the searched target network structure.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a network structure search method, apparatus, terminal device, and storage medium. Background Technology

[0002] Currently, the camera functions of mobile phones and other terminal devices are becoming increasingly sophisticated, with image quality approaching that of DSLRs. This is mainly due to the crucial role played by the built-in image signal processing (ISP) module. The ISP module is typically implemented through several approaches: 1. Traditional multi-stage processing methods to progressively complete image de-mosaicing, noise reduction, and color equalization; 2. Replacing the various stages of traditional processing methods with one or more manually designed deep networks. Currently, manually designed network models have fixed and simplistic structures, resulting in relatively poor network performance. Summary of the Invention

[0003] This application provides a network structure search method, apparatus, terminal device, and storage medium, which can be used to design network structures with better performance.

[0004] The first aspect of this application provides a network structure search method, including:

[0005] Initialize the network architecture, which includes N networks. The first network includes M candidate sub-networks. The number of nodes in any two sub-networks among the M candidate sub-networks are different. The first network is any one of the N networks, where N is an integer greater than or equal to 1 and M is an integer greater than or equal to 2.

[0006] A hierarchical search is used to search for a subnetwork from the first network, and the target network structure is constructed based on the searched N subnetworks.

[0007] A second aspect of this application provides a network structure search device, comprising:

[0008] An initialization unit is used to initialize the network architecture, which includes N networks, the first network including M candidate sub-networks, and the number of nodes in any two sub-networks among the M candidate sub-networks are different. The first network is any one of the N networks, where N is an integer greater than or equal to 1 and M is an integer greater than or equal to 2.

[0009] The hierarchical search unit is used to search for a subnetwork from the first network using a hierarchical search method, and to construct the target network structure based on the searched N subnetworks.

[0010] A third aspect of this application provides a terminal device including a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions to execute the step instructions as described in the first aspect of this application.

[0011] A fourth aspect of this application provides a computer-readable storage medium storing a computer program for electronic data interchange, the computer program including program instructions that, when executed by a processor, cause the processor to perform the step instructions as described in the first aspect of this application.

[0012] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a computer program, the computer program including program instructions, and the program instructions, when executed by a processor, cause the processor to perform the step instructions as described in the first aspect of this application.

[0013] In this embodiment, a network architecture is initialized, comprising N networks. The first network includes M candidate sub-networks, where any two sub-networks have a different number of nodes. The first network is any one of the N networks, where N is an integer greater than or equal to 1 and M is an integer greater than or equal to 2. A hierarchical search is used to search for a sub-network from the first network, and a target network structure is constructed based on the searched N sub-networks. This network structure search method, after initializing the network structure, constructs the target network structure from the N searched sub-networks using a hierarchical search, which improves the performance of the searched target network structure, enables automatic search of the target network structure, and enhances the intelligence of the target network structure search. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating a network structure search method provided in an embodiment of this application;

[0016] Figure 2 This is a flowchart illustrating another network structure search method provided in an embodiment of this application;

[0017] Figure 3This is a schematic diagram of a network architecture designed based on the search Unet network, provided in an embodiment of this application;

[0018] Figure 4 This is a schematic diagram illustrating the evolution of a candidate subnetwork according to an embodiment of this application;

[0019] Figure 5a This application provides a structure for searching a downsampling module on a second dataset.

[0020] Figure 5b This application provides a structure for searching an upsampling module on a second dataset.

[0021] Figure 6 This is a schematic diagram of the structure of a network structure search device provided in an embodiment of this application;

[0022] Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0025] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0026] The terminal devices involved in the embodiments of this application may include various devices with data processing functions.

[0027] Please see Figure 1 , Figure 1 This is a flowchart illustrating a network structure search method provided in an embodiment of this application. Figure 1 As shown, the network structure search method may include the following steps.

[0028] 101. The terminal device initializes the network architecture, which includes N networks. The first network includes M candidate sub-networks, and the number of nodes in any two sub-networks among the M candidate sub-networks is different.

[0029] The first network is any one of the N networks, where N is an integer greater than or equal to 1 and M is an integer greater than or equal to 2.

[0030] In this embodiment of the application, initializing the network architecture may include the number of networks included in the initializing network architecture, the number of nodes that each network may contain, and the number and type of operators connecting the nodes.

[0031] Optionally, in step 101, the terminal device initializes the network architecture, which may include the following steps:

[0032] The terminal device sets the initial architecture parameters of the network architecture according to a uniform distribution strategy.

[0033] In this embodiment, the uniform distribution strategy can be uniform initialization. For example, for a certain weight parameter, it can be set to the same value in each subnet.

[0034] The initial architecture parameters are the initial values ​​of the network architecture's parameters. Architecture parameters can include the network's own weight parameters, the connection weight parameters between internal nodes of sub-networks, and the weight parameters between each sub-network. Using a uniformly distributed strategy to set the initial architecture parameters facilitates the rapid subsequent search for operators exhibiting the aforementioned trends (each operator corresponds to the same initial architecture parameters, starting from the same point; during training, determining which operator increases or decreases only requires comparing the magnitudes of the architecture parameters), thereby improving the speed of network structure search.

[0035] 102. The terminal device uses a hierarchical search method to search for a sub-network from the first network, and constructs the target network structure based on the searched N sub-networks.

[0036] In this embodiment, the network architecture includes N networks, and the first network includes M candidate sub-networks. The number of nodes in any two sub-networks among the M candidate sub-networks is different. The terminal device can select one sub-network from the M candidate sub-networks contained in each of the N networks. For example, the network architecture includes two networks (Network 1 and Network 2), and each network can include three candidate sub-networks. Network 1 includes candidate sub-network 1-1 (3 nodes), candidate sub-network 1-2 (4 nodes), and candidate sub-network 1-3 (5 nodes), and Network 2 includes candidate sub-network 2-1 (3 nodes), candidate sub-network 2-2 (4 nodes), and candidate sub-network 2-3 (5 nodes). The hierarchical search approach can be divided into two layers. The first layer searches for a subnetwork from network 1 (e.g., candidate subnetwork 1-1) and a subnetwork from network 2 (e.g., candidate subnetwork 2-3). The second layer searches for the operator types and number of operators among the three nodes in candidate subnetwork 1-1, and for the operator types and number of operators among the five nodes in candidate subnetwork 2-3. This embodiment uses a hierarchical search approach (two-layer search) to find one subnetwork from each of the N networks, enabling automatic searching of the number of nodes and operator types in the network structure, thus obtaining a better-performing network structure.

[0037] The operator type is determined by the specific application task and the function to be achieved. For example, in image signal processing (ISP) tasks, the operator type can include convolution operators, pooling operators, and so on.

[0038] Search for one subnetwork from each of the N networks, and construct the target network structure based on the N searched subnetworks. These N target subnetworks can be stacked according to a network architecture to obtain the target network structure.

[0039] Optionally, in step 102, the terminal device searches for a sub-network from the first network using a hierarchical search method, which may specifically include the following steps:

[0040] (11) The terminal device searches for a first subnetwork from the M candidate subnetworks. The first subnetwork includes P nodes and Q operators connecting the P nodes; P is an integer greater than or equal to 2, and Q is an integer greater than or equal to 2.

[0041] (12) The terminal device searches for R operators from the Q operators of the first sub-network to obtain the compressed first sub-network. The compressed first sub-network includes P nodes and R operators connecting the P nodes, where R is an integer less than Q.

[0042] The hierarchical search method in this application embodiment can be divided into two layers: the first layer search is the number of search nodes; the second layer search is the search operator.

[0043] Before the search, every two nodes in the candidate subnetwork can be connected by a hybrid operator (an operator obtained by mixing multiple operators), which is formed by fusing multiple operators (e.g., multiple operators of different types). After the second-layer search process is completed, one or more useful operators are selected from the hybrid operator; the number of operators selected here is less than the number of operators in the hybrid operator before the search. The number of operators in the compressed first subnetwork is less than the number of operators in the first subnetwork before the search.

[0044] Optionally, in step (11), the terminal device searches for a first sub-network from the M candidate sub-networks, including:

[0045] (111) The terminal device acquires the second dataset for training;

[0046] (112) The terminal device trains the network architecture using the second dataset and optimizes the architecture parameters; the architecture parameters include the weight parameters of the network architecture itself, the connection weight parameters between the internal nodes of the M candidate sub-networks, and the allocation weight parameters of the M candidate sub-networks;

[0047] (113) The terminal device selects the candidate subnetwork with the largest allocation weight parameter from the M candidate subnetworks as the first subnetwork.

[0048] The weight parameters of the network architecture itself are optimized using stochastic gradient descent (SGD), and the connection weight parameters between the internal nodes of the M candidate sub-networks and the weight parameters between the M candidate sub-networks are optimized using alternating optimization (Adam), which can improve the optimization effect of the weight parameters.

[0049] In this embodiment, the second dataset can be selected based on the requirements of the target network structure. For example, if a network structure for image signal processing needs to be selected, an image dataset can be used. For instance, the second dataset could be the Zurich image signal processing (ISP) dataset.

[0050] The weight parameters w of the network architecture itself can include parameters inherent to the operators, such as weights and biases in operators (e.g., convolution). These are related to the candidate subnetworks because the candidate subnetworks contain operators.

[0051] The connection weight parameter α between the internal nodes of the M candidate subnetworks refers to the connection weight parameter between the internal nodes of each of the M candidate subnetworks.

[0052] The weight parameter β assigned to the M candidate subnetworks refers to the weight parameter assigned to the M candidate subnetworks in the same search phase.

[0053] Generally, the larger the weight parameters of a candidate subnetwork, the greater its contribution to the overall network architecture. A softmax operation can be performed on the weight parameters of each of the M candidate subnetworks, mapping them to the range [0,1]. A larger value indicates a larger weight parameter.

[0054] In this embodiment, the candidate subnetwork with the largest assigned weight parameter can be selected from M candidate subnetworks as the first subnetwork, and the N candidate subnetworks that contribute the most to the entire network architecture can be selected from N networks, thereby improving the performance of the entire network architecture.

[0055] The training in the embodiments of this application can be supervised training.

[0056] Optionally, (112) optimizing the terminal device architecture parameters may include the following steps:

[0057] The terminal device optimizes the architecture parameters through an objective function; the objective function is obtained by weighted summation of the L1 loss function and network performance indicators; the network performance indicators include at least one of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

[0058] In this embodiment, a higher PSNR value indicates less image distortion. PSNR is an objective image quality evaluation metric based on the error between corresponding pixels, i.e., it is an error-sensitive image quality evaluation. SSIM is a metric that measures the similarity between two images, with structural similarity ranging from -1 to 1. When two images are identical, the SSIM value is equal to 1.

[0059] This application embodiment can optimize architecture parameters through an objective function. Specifically, the architecture parameters can be optimized by minimizing the objective function. In the process of optimizing the architecture parameters through the objective function, either stochastic gradient descent (SGD) or alternating optimization (Adam) can be used.

[0060] Optionally, in step (12), the terminal device searches for R operators from the Q operators of the first sub-network, which may specifically include:

[0061] During the training process of the network architecture using the second dataset, the terminal device selects R operators whose architecture parameters are on an upward trend from the Q operators of the first sub-network.

[0062] In this embodiment, the architecture parameters corresponding to the operator may include connection weight parameters α between the internal nodes of the M candidate sub-networks. An operator whose architecture parameters are on an upward trend refers to an operator whose architecture parameter weights increase during the training of the network architecture using the second dataset. Generally, the larger the weight of the architecture parameter corresponding to an operator, the greater the contribution of that operator to the model; conversely, the smaller the weight of the architecture parameter corresponding to an operator, the smaller its contribution to the model. A softmax operation can be performed on the architecture parameters (e.g., weight parameters α) corresponding to each of the Q operators in the first sub-network, mapping them to the range [0,1]. A larger value indicates a larger architecture parameter corresponding to the operator.

[0063] Among the multiple operators between two nodes, the number of operators showing an upward trend can be one or more. You can select all operators with an upward trend, or you can select the operator with the largest upward trend. In this embodiment, you can select R operators whose corresponding architectural parameters are showing an upward trend from the Q operators of the first sub-network, and you can select the R operators that contribute the most to the first sub-network from the Q operators of the first sub-network, thereby improving the performance of the first sub-network.

[0064] The network structure search method of this application, after initializing the network structure, constructs the target network structure by searching N sub-networks in a hierarchical search manner. This can improve the performance of the searched target network structure, realize automatic search of the target network structure, and improve the intelligence of the target network structure search.

[0065] Please see Figure 2 , Figure 2 This is a flowchart illustrating another network structure search method provided in an embodiment of this application. For example... Figure 2 As shown, the network structure search method may include the following steps.

[0066] 201. The terminal device initializes the network architecture, which includes N networks. The first network includes M candidate sub-networks, and the number of nodes in any two of the M candidate sub-networks is different.

[0067] The first network is any one of the N networks, where N is an integer greater than or equal to 1 and M is an integer greater than or equal to 2.

[0068] 202. The terminal device uses a hierarchical search method to search for a sub-network from the first network, and constructs the target network structure based on the searched N sub-networks.

[0069] For details on the implementation of steps 201 and 202, please refer to [link / reference needed]. Figure 1 Steps 101 to 102 shown are not repeated here.

[0070] 203. The terminal device acquires the first dataset for training, and trains the target network structure using the first dataset to obtain the trained target network structure.

[0071] In this embodiment, after the target network structure is found through steps 201 and 202, step 203 trains the target network structure using the first dataset. Steps 201 and 202 train the network architecture using the second dataset to find a high-performance target network structure. Step 203 trains the target network structure using the first dataset to obtain a well-trained target network structure. The well-trained target network structure can be used for network inference and validation.

[0072] The first and second datasets can use the same data or different datasets. The training in this embodiment is supervised training.

[0073] In this embodiment, the target network structure is trained using a first dataset to obtain a trained target network structure. The trained target network structure can be used for network inference and verification, and can be used to implement specific application tasks (e.g., ISP tasks) and functions.

[0074] The following example, using a Unet network as the target network structure, illustrates the process of network structure search.

[0075] Please see Figure 3 , Figure 3 This is a schematic diagram of a network architecture designed based on a search Unet network, as provided in an embodiment of this application. Figure 3As shown, the network structure includes N=8 networks, each containing M=3 candidate sub-networks. This network architecture can include 4 downsampling networks (networks 31, 32, 33, and 34), 4 upsampling networks (networks 35, 36, 37, and 38), a head operator, a tail operator, and a connection operator. The head operator includes a convolution with stride = 1 and an activation operator (leaky ReLU). The tail operator includes a transpose convolution with stride = 2, an activation operator, and an upsampling operator (pixel shuffle operation). The connection operator includes a convolution with stride = 2 and an activation operator. The input image size of this network architecture is H*W, and the output image size is 2H*2W. The head and tail operators can achieve a 2x decrease and a 2x increase in feature map size.

[0076] Network 31 includes three candidate subnetworks: candidate subnetwork 1-1 (3 nodes), candidate subnetwork 1-2 (4 nodes), and candidate subnetwork 1-3 (5 nodes). Network 32 includes three candidate subnetworks: candidate subnetwork 2-1 (3 nodes), candidate subnetwork 2-2 (4 nodes), and candidate subnetwork 2-3 (5 nodes). Network 33 includes three candidate subnetworks: candidate subnetwork 3-1 (3 nodes), candidate subnetwork 3-2 (4 nodes), and candidate subnetwork 3-3 (5 nodes). Network 34 includes three candidate subnetworks: candidate subnetwork 4-1 (3 nodes), candidate subnetwork 4-2 (4 nodes), and candidate subnetwork 4-3 (5 nodes). Network 35 includes three candidate subnetworks: candidate subnetwork 5-1 (3 nodes), candidate subnetwork 5-2 (4 nodes), and candidate subnetwork 5-3 (5 nodes). Network 36 includes three candidate subnetworks: candidate subnetwork 6-1 (3 nodes), candidate subnetwork 6-2 (4 nodes), and candidate subnetwork 6-3 (5 nodes). Network 37 includes three candidate subnetworks: candidate subnetwork 7-1 (3 nodes), candidate subnetwork 7-2 (4 nodes), and candidate subnetwork 7-3 (5 nodes). Network 38 includes three candidate subnetworks: candidate subnetwork 8-1 (3 nodes), candidate subnetwork 8-2 (4 nodes), and candidate subnetwork 8-3 (5 nodes).

[0077] Figure 3In the diagram, networks 31, 32, 33, and 34 are downsampling networks, while networks 35, 36, 37, and 38 are upsampling networks.

[0078] During the network architecture search process, a candidate subnetwork can be searched from network 31, a candidate subnetwork from network 32, a candidate subnetwork from network 33, and a candidate subnetwork from network 34. The four candidate subnetworks searched from networks 31, 32, 33, and 34 are used as downsampling modules for the U-shaped network (Unet).

[0079] A candidate subnetwork can be searched from network 35, network 36, network 37, and network 38. These four candidate subnetworks are used as upsampling modules for the Unet network. The downsampling module replaces the convolution / activation / pooling (Conv / ReLU / Pooling) structure in the encoding of the original Unet network, and the upsampling module replaces the convolution / activation / pooling (Conv / ReLU / Pooling) structure in the decoding of the original Unet network, thus constructing the ISP image denoising backbone network (i.e., the target network structure obtained through the search: the Unet network).

[0080] Regarding the search space, in this embodiment of the application, the downsampling module and upsampling module to be searched each consist of a variable number of network nodes (e.g., Figure 3 The network consists of candidate subnetworks (including three types: 3 nodes, 4 nodes, and 5 nodes). The random combination of these nodes and operators constitutes the search space of the network architecture. The parameters of the nodes and the weights of the operators can be learned by using an alternating optimization method.

[0081] Objective function definition: In terms of the search objective function, the optimization objective function with energy consumption constraints is defined as argmin L_1+γ·L_SSIM, which measures the loss of the network architecture by the weighted sum of the 1 norm and SSIM between the network output and the given true value.

[0082] During the search process, the objective metrics for evaluating the network architecture performance are Peak Signal-Noise Ratio (PSNR) and Structural Similarity (SSIM).

[0083] The weight parameters of the network architecture itself are optimized using stochastic gradient descent (SGD), while the connection weight parameters between internal nodes of candidate subnetworks and the weight parameters between candidate subnetworks within each network are optimized using alternating optimization (Adam).

[0084] The evolution of each candidate subnetwork can be found in [reference]. Figure 4 . Figure 4 This is a schematic diagram illustrating the evolution of a candidate subnetwork according to an embodiment of this application. For example... Figure 4 As shown, the candidate subnetwork has 4 nodes, and the connection lines between each pair of nodes represent "operators". In Stage 1, before training the candidate subnetwork, any two nodes in the candidate subnetwork can be connected by 3 operators, and the architecture parameters of the candidate subnetwork can be set by uniform distribution. In Stage 2, the candidate subnetwork is trained using the second dataset. During training, operators in the candidate subnetwork that are on an upward trend (connection lines marked AS) are searched. In Stage 3, operators in the candidate subnetwork that are on an upward trend (connection lines marked AS) are retained, and operators in the candidate subnetwork that are on a downward trend (connection lines marked DS) are removed.

[0085] In this embodiment, all operators exhibiting an upward trend are selected as the final network operators, while all operators without an upward trend are discarded. After selecting these operators with an upward trend, this embodiment uses the averaging of their output feature maps to complete operator-level fusion. Please refer to... Figure 5a and Figure 5b , Figure 5a This application provides a structure for searching a downsampling module on a second dataset. Figure 5b This application provides a structure for searching an upsampling module on a second dataset. For example... Figure 5a As shown in the figure, D[XXX] represents the downsampling operator XXX. Figure 5b As shown, U[XXX] identifies the fusion of the upsampling operator XXX. Figure 5a and Figure 5b In the diagram, ①, ②, and ③ represent three nodes, and inputs 0 and 1 represent the two inputs to the candidate subnetwork. The numbers in 'XXX' represent operator indices, and N[XXX] represents the fusion of the identity mapping operator XXX. At the output of the candidate subnetwork, either a downsampling module or an upsampling module is obtained by concatenating the nodes.

[0086] The soft-compressed network structure is reconstructed based on all retained operators. The searched target network structure is then retrained (using the first dataset) to obtain the final trained target network structure that can be used for inference. The trained target network structure can then be used for network inference and validation.

[0087] This invention proposes a soft compression method for ISP networks based on a hierarchical search space, addressing the issue of insufficient intelligence in hyperparameter settings during differential search. The embodiments of this application can simultaneously automate the search for operator types and hyperparameters (such as the number of nodes, the number of output layer channels, etc.) in the network structure through the design of a hierarchical search space. The search space provides multiple possibilities for network design, from which the optimal solution is searched. The embodiments of this application demonstrate better performance than manually designed single networks. It can achieve superior performance compared to manually designed networks, and the searched network structure exhibits good versatility and portability.

[0088] The network structure search method of this application, after initializing the network structure, constructs the target network structure by searching N sub-networks in a hierarchical search manner. This can improve the performance of the searched target network structure, realize automatic search of the target network structure, and improve the intelligence of the target network structure search.

[0089] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the terminal device includes the corresponding hardware structure and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0090] This application embodiment can divide the terminal device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0091] Please see Figure 6 , Figure 6This is a schematic diagram of a network structure search device provided in an embodiment of this application. The network structure search device 600 is applied to a terminal device and may include an initialization unit 601 and a hierarchical search unit 602, wherein:

[0092] Initialization unit 601 is used to initialize the network architecture, which includes N networks, the first network includes M candidate sub-networks, and the number of nodes in any two sub-networks among the M candidate sub-networks are different. The first network is any one of the N networks, where N is an integer greater than or equal to 1 and M is an integer greater than or equal to 2.

[0093] The hierarchical search unit 602 is used to search for a subnetwork from the first network using a hierarchical search method, and to construct the target network structure based on the searched N subnetworks.

[0094] Optionally, the network structure search device 600 may also include a training unit 603.

[0095] The training unit 603 is used to obtain a first dataset for training after the hierarchical search unit 602 constructs the target network structure based on the search of N sub-networks, and trains the target network structure using the first dataset to obtain the trained target network structure.

[0096] The initialization unit 601 initializes the network architecture, including setting the initial architecture parameters of the network architecture according to a uniform distribution strategy.

[0097] Optionally, the hierarchical search unit 602 searches for a subnetwork from the first network using a hierarchical search method, including: searching for a first subnetwork from the M candidate subnetworks, the first subnetwork including P nodes and Q operators connecting the P nodes; P is an integer greater than or equal to 2, and Q is an integer greater than or equal to 2; searching for R operators from the Q operators of the first subnetwork to obtain a compressed first subnetwork, the compressed first subnetwork including P nodes and R operators connecting the P nodes, where R is an integer less than Q.

[0098] Optionally, the hierarchical search unit 602 searches for a first sub-network from the M candidate sub-networks, including: obtaining a second dataset for training, training the network architecture using the second dataset, and optimizing the architecture parameters; the architecture parameters include the weight parameters of the network architecture itself, the connection weight parameters between the internal nodes of the M candidate sub-networks, and the allocation weight parameters of the M candidate sub-networks;

[0099] Select the candidate subnetwork with the largest allocation weight parameter from the M candidate subnetworks as the first subnetwork;

[0100] The weight parameters of the network architecture itself are optimized using stochastic gradient descent, and the connection weight parameters between internal nodes of the M candidate sub-networks and the weight parameters between the M candidate sub-networks are optimized using an alternating optimization method.

[0101] Optionally, the hierarchical search unit 602 optimizes the architecture parameters, including:

[0102] The architecture parameters are optimized through an objective function; the objective function is obtained by weighted summation of the L1 loss function and network performance metrics; the network performance metrics include at least one of peak signal-to-noise ratio and structural similarity.

[0103] Optionally, the hierarchical search unit 602 searches for R operators from the Q operators of the first sub-network, including: during the training of the network architecture using the second dataset, selecting R operators from the Q operators of the first sub-network whose corresponding architecture parameters are on an upward trend.

[0104] In this embodiment, the initialization unit 601, the hierarchical search unit 602, and the training unit 603 can be processors in a terminal device.

[0105] In this embodiment of the application, after initializing the network structure, the target network structure is constructed by searching N sub-networks through hierarchical search. This can improve the performance of the searched target network structure, enable automatic search of the target network structure, and enhance the intelligence of the target network structure search.

[0106] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application, such as... Figure 7 As shown, the terminal device 700 includes a processor 701 and a memory 702, which can be interconnected via a communication bus 703. The communication bus 703 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 703 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 7 The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Memory 702 stores computer programs, which include program instructions. Processor 701 is configured to call these program instructions, which include instructions for execution. Figures 1-2Some or all of the steps in the method shown.

[0107] The processor 701 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme.

[0108] The memory 702 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0109] In addition, the terminal device 700 may also include general components such as communication interfaces and antennas, which will not be described in detail here.

[0110] In this embodiment of the application, after initializing the network structure, the target network structure is constructed by searching N sub-networks through hierarchical search. This can improve the performance of the searched target network structure, enable automatic search of the target network structure, and enhance the intelligence of the target network structure search.

[0111] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the network structure search methods described in the above method embodiments.

[0112] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0113] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0116] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0117] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0118] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0119] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A network structure search method, characterized in that, include: The initial architecture parameters of the network architecture are set according to a uniform distribution strategy. The network architecture includes N networks, the first network includes M candidate sub-networks, and the number of nodes in any two sub-networks in the M candidate sub-networks are different. The first network is any one of the N networks, where N is an integer greater than or equal to 1 and M is an integer greater than or equal to 2. A subnetwork is searched from the first network using a hierarchical search method. A target network structure is constructed based on the searched N subnetworks, where the N subnetworks are respectively derived from the N networks. The target network structure is used for image signal processing. The step of searching for a subnetwork from the first network using a hierarchical search method includes: A first subnetwork is searched from the M candidate subnetworks. The first subnetwork includes P nodes and Q operators connecting the P nodes; P is an integer greater than or equal to 2, and Q is an integer greater than or equal to 2. Search for R operators from the Q operators of the first sub-network to obtain the compressed first sub-network. The compressed first sub-network includes P nodes and R operators connecting the P nodes, where R is an integer less than Q. The step of searching for the first sub-network from the M candidate sub-networks includes: A second dataset is obtained for training. The network architecture is trained using the second dataset to optimize the architecture parameters. The architecture parameters include the weight parameters of the network architecture itself, the connection weight parameters between the internal nodes of the M candidate sub-networks, and the assigned weight parameters of the M candidate sub-networks. The second dataset is an image dataset. The candidate subnetwork with the largest assigned weight parameter is selected from the M candidate subnetworks as the first subnetwork.

2. The method according to claim 1, characterized in that, After constructing the target network structure using N sub-networks based on search, the method further includes: Obtain a first dataset for training, and train the target network structure using the first dataset to obtain a trained target network structure.

3. The method according to claim 1, characterized in that, The weight parameters of the network architecture itself are optimized using stochastic gradient descent, and the connection weight parameters between the internal nodes of the M candidate sub-networks and the weight parameters between the M candidate sub-networks are optimized using an alternating optimization method.

4. The method according to claim 3, characterized in that, The optimized architecture parameters include: The architecture parameters are optimized through an objective function; the objective function is obtained by weighted summation of the L1 loss function and network performance metrics; the network performance metrics include at least one of peak signal-to-noise ratio and structural similarity.

5. The method according to claim 3, characterized in that, The step of searching for R operators from the Q operators of the first sub-network includes: During the training of the network architecture using the second dataset, R operators whose architecture parameters are on an upward trend are selected from the Q operators of the first sub-network.

6. A network structure search device, characterized in that, include: An initialization unit is used to set the initial architecture parameters of the network architecture according to a uniform distribution strategy. The network architecture includes N networks, the first network includes M candidate sub-networks, and the number of nodes in any two sub-networks among the M candidate sub-networks are different. The first network is any one of the N networks, where N is an integer greater than or equal to 1 and M is an integer greater than or equal to 2. A hierarchical search unit is used to search for a sub-network from the first network using a hierarchical search method, and to construct a target network structure based on the searched N sub-networks, wherein the N sub-networks are respectively from the N networks, and the target network structure is used for image signal processing; The hierarchical search unit searches for a subnetwork from the first network using a hierarchical search method, including: searching for a first subnetwork from the M candidate subnetworks, the first subnetwork including P nodes and Q operators connecting the P nodes; P is an integer greater than or equal to 2, and Q is an integer greater than or equal to 2; searching for R operators from the Q operators of the first subnetwork to obtain a compressed first subnetwork, the compressed first subnetwork including P nodes and R operators connecting the P nodes, where R is an integer less than Q; The hierarchical search unit searches for a first subnetwork from the M candidate subnetworks, including: obtaining a second dataset for training; training the network architecture using the second dataset; and optimizing the architecture parameters; the architecture parameters include the network architecture's own weight parameters, the connection weight parameters between the internal nodes of the M candidate subnetworks, and the assigned weight parameters of the M candidate subnetworks; the second dataset is an image dataset; and selecting the candidate subnetwork with the largest assigned weight parameter from the M candidate subnetworks as the first subnetwork.

7. A terminal device, characterized in that, The device includes a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 5.

Images