Model search methods and related products

Abstract

The embodiment of the present application discloses a model search method and related products, the method comprising: determining an upsampling cell body and a downsampling cell body corresponding to a preset model, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers; determining a first mixed operation and a second mixed operation corresponding to each of the feature layers in the upsampling cell body and the downsampling cell body; when searching and training the preset model, searching for multiple preselected operations included in the first mixed operation to obtain a first target operation; searching for multiple preselected operations included in the second mixed operation to obtain a second target operation; and determining a target model based on the first target operation and the second target operation. The use of the embodiment of the present application is conducive to reducing video memory usage, thereby reducing the amount of calculation, and is conducive to searching larger and more complex neural networks.

Description

Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to a model search method and related products. Background Art

[0002] With advances in deep learning technology, neural network algorithms are increasingly being used for image denoising. While the emergence of neural network search algorithms has reduced the complexity of manually designed neural networks, it has also introduced new challenges. Currently, most neural network algorithms are computationally intensive during the search process and consume significant amounts of device memory. Summary of the Invention

[0003] The embodiments of the present application provide a model search method and related products, which are beneficial to reducing video memory usage, thereby reducing the amount of calculation and facilitating the search of larger and more complex neural networks.

[0004] In a first aspect, an embodiment of the present application provides a model search method, the method comprising:

[0005] Determine an upsampling cell body and a downsampling cell body corresponding to a preset model, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers;

[0006] Determine a first mixing operation and a second mixing operation corresponding to each of the feature layers in the up-sampling cell body and the down-sampling cell body respectively;

[0007] When searching and training the preset model, searching for a plurality of preselected operations included in the first mixed operation to obtain a first target operation;

[0008] Searching a plurality of preselected operations included in the second mixed operation to obtain a second target operation;

[0009] A target model is determined according to the first target operation and the second target operation.

[0010] In a second aspect, an embodiment of the present application provides a model search device, which is applied to an electronic device. The device includes: a determination unit and a search unit, wherein:

[0011] The determining unit is configured to determine an up-sampling cell body and a down-sampling cell body corresponding to a preset model, wherein the up-sampling cell body and / or the down-sampling cell body include multiple feature layers;

[0012] The determining unit is further configured to determine a first mixing operation and a second mixing operation corresponding to each of the feature layers in the up-sampling cell body and the down-sampling cell body respectively;

[0013] The search unit is configured to search for a plurality of preset operations included in the first mixed operation to obtain a first target operation when performing search training on the preset model;

[0014] The search unit is further configured to search for a plurality of preset operations included in the second mixed operation to obtain a second target operation;

[0015] The determining unit is further configured to determine a target model according to the first target operation and the second target operation.

[0016] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program comprises instructions for executing the steps of any method of the first aspect of the embodiment of the present application.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the above-mentioned computer-readable storage medium stores a computer program for electronic data exchange, wherein the above-mentioned computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0018] In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0019] It can be seen that in the embodiment of the present application, the upsampling cell body and the downsampling cell body corresponding to the preset model can be determined, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers; the first mixed operation and the second mixed operation corresponding to each of the feature layers in the upsampling cell body and the downsampling cell body are determined; when searching and training the preset model, the multiple pre-selected operations included in the first mixed operation are searched to obtain a first target operation; the multiple pre-selected operations included in the second mixed operation are searched to obtain a second target operation; and the target model is determined based on the first target operation and the second target operation. In this way, one of the target operations can be selected to participate in the entire training in each mixed operation, which is beneficial to reducing video memory usage, thereby reducing the amount of calculation, and is beneficial to searching for larger and more complex neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1A This is a schematic diagram of the structure of a model search system provided in an embodiment of the present application;

[0022] Figure 1B This is a schematic diagram of the structure of an upsampling cell body and / or a downsampling cell body provided in an embodiment of the present application;

[0023] Figure 1C This is a flow chart of a CConv module provided in an embodiment of the present application;

[0024] Figure 2 This is a flow chart of a model search method provided in an embodiment of the present application;

[0025] Figure 3 This is a flow chart of a model search method provided in an embodiment of the present application;

[0026] Figure 4 This is a flow chart of a model search method provided in an embodiment of the present application;

[0027] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0028] Figure 6 This is a block diagram of the functional units of a model search device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0031] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0032] The electronic device may be a portable electronic device that also includes other functions such as a personal digital assistant and / or a music player, such as a mobile phone, a tablet computer, a wearable electronic device with wireless communication functions (such as a smart watch, smart glasses), a vehicle-mounted device, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with an iOS system, an Android system, a Microsoft system, or other operating systems. The portable electronic device may also be other portable electronic devices, such as a laptop computer. It should also be understood that in some other embodiments, the electronic device may not be a portable electronic device, but a desktop computer.

[0033] In the first part, the example application scenarios disclosed in the embodiments of this application are introduced as follows.

[0034] Figure 1A A structural diagram of a model search system applicable to the present application is shown, which may include: a data input module, one or more upsampling cells, one or more downsampling cells, a splicing operation module, a CConv module and a data output module.

[0035] The schematic diagram of the structure of the above-mentioned upsampling cell body and / or downsampling cell body is as follows: Figure 1B As shown, each upsampling cell body and / or downsampling cell body may include: an input node, an intermediate node and an output node.

[0036] Among them, such as Figure 1BThe figure shows a schematic diagram of the structure of an upsampling cell and / or a downsampling cell. The upsampling cell and / or downsampling cell comprises a multi-layer feature layer structure, wherein a first feature layer and a second feature layer are provided between the input node and the intermediate node, and a third feature layer is provided between the intermediate node and the output node. The first feature layer corresponds to a first operation, which is performed on the input data of the input node to obtain intermediate nodes S1 and S2; the second feature layer corresponds to a second operation, which is performed on intermediate node S1 and / or intermediate node S2 and / or the input data to obtain intermediate node S3; finally, the third feature layer corresponds to a third operation, which is performed on intermediate nodes S1, S2, and S3 to obtain the output data after processing of the upsampling cell and / or downsampling cell.

[0037] It should be noted that the first operation and / or the second operation and / or the third operation corresponding to the above-mentioned upsampling cell body and the downsampling cell body are different.

[0038] Among them, the input data of the above-mentioned input upsampling cell body and / or downsampling cell body and / or data input module can be four-dimensional tensor data x[B, H, W, Ci], wherein the above-mentioned B represents the quantity, Ci represents the image channel, H represents the height, and W represents the width.

[0039] In a possible example, for the upsampling cell body, the above-mentioned first operation can perform dimensionality increase processing on the input data, and can transform the input data of dimension [B, H, W, Ci] into intermediate nodes S1 and S2 of [B, H*2, W*2, Co]. The first operation may include at least one of the following: unpooling (kernel is 2x2), interpolation operations (bilinear interpolation, nearest neighbor interpolation), deconvolution (strides is 2x2), etc., which are not limited here; furthermore, the above-mentioned second operation can perform dimensionality reduction processing on the data input to the second feature layer, and transform the input data of dimension [B, H, W, Ci] into intermediate node S3 of [B, H, W, Co]. The second operation may include at least one of the following: direct connection (Identity), convolution (strides is 1x1), depth-wise separable convolution (strides is 1x1), etc., which are not limited here; the above-mentioned third operation may include direct connection operation, etc., which are not limited here.

[0040] In a possible example, for the downsampled cell body, the above first operation can transform the input data of dimension [B, H, W, Ci] into intermediate nodes S1 and S2 of dimension [B, H / 2, W / 2, Co], that is, the input data of the downsampled cell body can be subjected to dimensionality reduction processing. The above first operation may include at least one of the following: pooling (kernel is 2x2), interpolation operations (bilinear interpolation, nearest neighbor interpolation), convolution (strides is 2x2), depth-separable convolution (strides is 2x2), etc., which are not limited here. ; The above-mentioned second operation can transform the input data of dimension [B, H, W, Ci] into the intermediate node S3 of dimension [B, H, W, Co]. The second operation includes at least one of the following: direct connection (Identity), convolution (strides is 1x1), depth-wise separable convolution (strides is 1x1), etc., which are not limited here; the above-mentioned third operation may include at least one of the following: direct connection (Identity), convolution (strides is 1x1), depth-wise separable convolution (strides is 1x1), etc., which are not limited here.

[0041] Among them, such as Figure 1C As shown in FIG, it is a schematic diagram of the process applied to the CConv module. The CConv module consists of two operations, namely, feature fusion operation (Cancat) and convolution operation (Conv).

[0042] Among them, for Figure 1A The system structure shown, when using this structure for search training, for the upsampling cell body and / or the downsampling cell body, each feature layer (any one of the first feature layer, the second feature layer and the third feature layer) may include multiple mixing operations, wherein each feature layer in the upsampling cell body may correspond to a first mixing operation (which may include multiple pre-selected operations), and each feature layer in the downsampling cell body may correspond to a second mixing operation (which may include multiple pre-selected operations).

[0043] In the present application, the upsampling cell body and downsampling cell body corresponding to the model architecture can be determined, and the input data can be transmitted to the downsampling cell body and the upsampling cell body in sequence. When searching and training the model, the first mixing operation and the second mixing operation corresponding to each feature layer in the upsampling cell body and the downsampling cell body are searched, and each search training of the model is completed, and the first target mixing operation corresponding to the upsampling cell body and the second target mixing operation corresponding to the downsampling cell body can be obtained. According to the first target mixing operation and the second target mixing operation, a target model of a new network structure is obtained. In this way, in each mixing operation, one of the target operations is selected to participate in the entire training, which is beneficial to reduce the memory usage, thereby reducing the amount of calculation, and is conducive to searching larger and more complex neural networks.

[0044] In this application, the above-mentioned multiple may refer to two or more than two, which will not be repeated later.

[0045] In the second part, the protection scope of the claims disclosed in the embodiments of the present application is introduced as follows.

[0046] See also Figure 2 , Figure 2 This is a flow chart of a model search method provided in an embodiment of the present application, which is applied to electronic devices. As shown in the figure, the model search method includes the following operations.

[0047] S201. Determine an upsampling cell body and a downsampling cell body corresponding to a preset model, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers.

[0048] Among them, the above preset model can be Figure 1A The structural diagram shown or the Figure 1A The variant structure of the model architecture shown in the figure, for example, the preset model may include one upsampling cell body and one downsampling cell body, or may include three upsampling cell bodies and three downsampling cell bodies, which is not limited here. The structural diagram of the upsampling cell body and / or downsampling cell body is shown in FIG. Figure 1B As shown, no further details are given here.

[0049] S202: Determine a first mixing operation and a second mixing operation corresponding to each of the feature layers in the up-sampling cell body and the down-sampling cell body.

[0050] The first mixing operation may be applied to multiple feature layers (including a first feature layer, a second feature layer, and a third feature layer) of the upsampled cell body, and each of the multiple feature layers may correspond to multiple pre-selected operations.

[0051] The second mixing operation may be applied to multiple feature layers (including a first feature layer, a second feature layer, and a third feature layer) of the downsampled cell body, and each of the multiple feature layers may correspond to multiple pre-selected operations.

[0052] S203. When searching and training the preset model, search for multiple pre-selected operations included in the first mixed operation to obtain a first target operation.

[0053] In a specific implementation, each time the preset model is searched and trained, a plurality of pre-selected operations of each feature layer of the up-sampled cell body are searched, and a first target operation is selected therefrom.

[0054] S204: Search multiple pre-selected operations included in the second mixed operation to obtain a second target operation.

[0055] In a specific implementation, each time the preset model is searched and trained, a plurality of pre-selected operations of each feature layer of the down-sampled cell body are searched, and a second target operation is selected from them.

[0056] S205: Determine a target model according to the first target operation and the second target operation.

[0057] In a specific implementation, after the search training is completed, the up-sampled cell body and the down-sampled cell body obtained after the search training, as well as Figure 1A The data input module, splicing operation module, CConv module and data output module shown constitute the target model.

[0058] After the search training is completed, the first target operation corresponding to each feature layer in the upsampled cell body and the second target operation corresponding to each feature layer in the downsampled cell body determined after the search training are searched. The target model composed of the first target operation and the second target operation can be used to complete the denoising process of the image.

[0059] It can be seen that the model search method described in the embodiment of the present application can determine the upsampling cell body and downsampling cell body corresponding to the preset model, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers; determine the first mixed operation and the second mixed operation corresponding to each of the feature layers in the upsampling cell body and the downsampling cell body; when searching and training the preset model, search the multiple pre-selected operations included in the first mixed operation to obtain the first target operation; search the multiple pre-selected operations included in the second mixed operation to obtain the second target operation; determine the target model based on the first target operation and the second target operation. In this way, in each mixed operation, one of the target operations can be selected to participate in the entire training, which is beneficial to reducing video memory usage, thereby reducing the amount of calculation, and is beneficial to searching larger and more complex neural networks.

[0060] In a possible example, the upsampling cell body and / or the downsampling cell body includes: an input node, an intermediate node, and an output node.

[0061] In one possible example, the multiple feature layers include a first feature layer, a second feature layer, and a third feature layer; the method further includes the following steps: determining that the feature layer from the input node to the intermediate node is the first feature layer, the feature layer from the input node to the intermediate node is the second feature layer, and the feature layer from the intermediate node to the output node is the third feature layer, wherein the first feature layer is used to complete any first pre-selected operation and / or the first operation among the multiple first pre-selected operations, the second feature layer is used to complete any second pre-selected operation and / or the second operation among the multiple second pre-selected operations, and the third feature layer is used to complete any third pre-selected operation and / or the third operation among the multiple third pre-selected operations.

[0062] Among them, such as Figure 1B As shown, the upsampling cell may include three feature layers, namely the first feature layer, the second feature layer, and the third feature layer. The first type may correspond to the first feature layer, the second type may correspond to the second feature layer, and the third operation may correspond to the third feature layer.

[0063] Among them, the first feature layer can be used to complete the first pre-selected operation or the first operation. The first preset operation and / or the first operation may include at least one of the following: anti-pooling (kernel is 2x2), interpolation operation (bilinear interpolation, nearest neighbor interpolation), deconvolution (strides is 2x2), etc., which are not limited here; the second feature layer can be used to complete the second preset operation or the second operation. The second preset operation and / or the second operation may include at least one of the following: direct connection (Identity), convolution (strides is 1x1), depth-separable convolution (strides is 1x1), etc., which are not limited here. The above-mentioned third feature layer can be used to complete the third preset operation or the third operation. The above-mentioned first preset operation may include the first operation, the second preset operation may include the second operation, and the third preset operation includes the third operation.

[0064] It can be seen that in this example, the upsampling cell body is composed of multiple feature layers, each feature layer can complete a corresponding operation. In the specific search training process, it is not necessary for all operations in the feature layer to participate, which is conducive to saving calculation amount.

[0065] In one possible example, the first target operation includes a first operation corresponding to a first type, a second operation corresponding to a second type, and a third operation corresponding to a third type; the multiple pre-selected operations included in the first mixed operation are searched to obtain the first target operation, and the above method includes the following steps: determining the multiple first pre-selected operations corresponding to the first type, the multiple second pre-selected operations corresponding to the second type, and the multiple third pre-selected operations corresponding to the third type among the multiple pre-selected operations included in the first mixed operation; when searching and training the preset model, selecting a first pre-selected operation from the multiple first pre-selected operations corresponding to the first type, and determining the selected first pre-selected operation as the first operation; selecting a second pre-selected operation from the multiple second pre-selected operations corresponding to the first type, and determining the selected first pre-selected operation as the second operation; selecting a third pre-selected operation from the multiple third pre-selected operations corresponding to the first type, and determining the selected first pre-selected operation as the third operation.

[0066] Among them, the function of each feature layer is different, and the corresponding operation types are different. For example, the first feature layer may include a first operation, and the first type may refer to a dimensionality increase operation on the height and width of the four-dimensional input data [B, H, W, Ci]; the second feature layer may include a second operation, and the second type may refer to a change in the image channel of the four-dimensional input data [B, H, W, Ci].

[0067] Among them, the operations corresponding to each type may correspond to different types. For example, the first operation may include at least one of the following: unpooling (kernel is 2x2), interpolation operations (bilinear interpolation, nearest neighbor interpolation), deconvolution (strides is 2x2), etc., which are not limited here; the second operation may include at least one of the following: direct connection (Identity), convolution (strides is 1x1), depth-separable convolution (strides is 1x1), etc., which are not limited here.

[0068] In a specific implementation, each time a preset model is searched and trained, in the process of selecting the pre-selected operation corresponding to each feature layer, a sampling method is adopted to select one of the pre-selected operations, and the preset model is searched and trained. After determining that the preset model has converged, the first operation, the second operation, and the third operation are further determined, and the first target operation is obtained by combining the above-mentioned first operation, second operation, and third operation.

[0069] It can be seen that in this example, the corresponding multiple pre-selected operations (including multiple first pre-selected operations, multiple second pre-selected operations, and multiple third pre-selected operations) can be determined according to the type corresponding to each feature layer, and then a first pre-selected operation, a second pre-selected operation, and a third pre-selected operation can be selected from the multiple first pre-selected operations, the multiple second pre-selected operations, and the multiple third pre-selected operations, and the search training can be completed. In this way, in the mixed operation calculation process, one of the multiple pre-selected operations corresponding to each feature layer can be selected, and combined with the pre-selected operations selected from other feature layers to obtain the first target operation corresponding to the up-sampled cell body, so as to complete the model search training for the input data. In the search process, it is not necessary for every operation to participate, which is conducive to reducing the amount of calculation.

[0070] In a possible example, the above method may also include the following steps: each time the preset model is search-trained, determining the input data for the search training; passing the input data through the input node, the intermediate node and the output node in sequence, and completing any first pre-selected operation and / or the first operation among the multiple first pre-selected operations, any second pre-selected operation and / or the second operation among the multiple second pre-selected operations, and any third pre-selected operation and / or the third operation among the multiple third pre-selected operations in sequence to obtain output data.

[0071] In this application, only one upsampling cell body is used for illustration. This example can be applied to Figure 1B In the schematic diagram of the upsampling cell structure shown in FIG, for example, Figure 3 As shown, it is a flow chart corresponding to the model search method corresponding to the upsampling cell body. The input data of the upsampling cell body can be input through the input node, and the first operation, the second operation and the third operation of the input data can be completed in sequence through the intermediate nodes (S1, S2 and S3) and the output node.

[0072] It can be seen that in this example, the screening of the mixed operation can be completed by upsampling the cell body and / or downsampling the cell body during each search training, and each operation can independently contribute to the output of the mixed operation during the search process.

[0073] In one possible example, selecting a first pre-selected operation from a plurality of first pre-selected operations corresponding to the first type and determining the selected first pre-selected operation as the first operation includes: each time the preset model is searched and trained, arbitrarily selecting a first pre-selected operation from the plurality of first pre-selected operations according to a preset sampling function, and acting on the first feature layer; after completing the search training of the preset model according to each first pre-selected operation, determining a plurality of weighting coefficients corresponding to the plurality of first pre-selected operations; and selecting the first pre-selected operation corresponding to the maximum weighting coefficient from the plurality of weighting coefficients as the first operation.

[0074] The above-mentioned preset sampling function can be set by the user or by the system default, which is not limited here; the preset sampling function can be expressed as:

[0075] y=Sample(α1*op1, α2*op2,···,α N *op N )(x);

[0076] In each search training, for an upsampled cell, the mixed operation corresponding to the first feature layer, the second feature layer and the third feature layer is composed of N pre-selected operations op, where α i It is a learnable architectural parameter representing the i-th operation, x represents the input data, y represents the output of the mixed operation, Sample is a preset sampling function, and the output of the mixed operation is to extract a preselected operation from it according to the probability (the size of the probability is represented by ) and act on the input data x to obtain the output data y corresponding to the upsampled cell.

[0077] Then, until the preset model converges, all search training is completed, and the probability corresponding to each preset operation is determined in each search training process. Then, the weighting coefficient corresponding to each preset operation in each feature layer can be determined, and then multiple weighting coefficients can be obtained. From the above multiple weighting coefficients, the preset operation corresponding to the maximum weighting coefficient can be selected as the final desired operation, for example, the first operation, the second operation and the third operation.

[0078] It can be seen that in this example, the pre-selected operation can be matched for each feature layer by sampling. Then, after multiple search trainings are performed on the preset model and the search training is completed, the optimal operation corresponding to each feature layer can be determined, that is, the operation corresponding to the preset operation with the largest weighted coefficient. In this way, in the hybrid operation, the operations involved in the calculation are determined according to a certain sampling, which alleviates the problem of direct connection operations dominating the network search process. It is also beneficial to obtain the target model later and to search for a better image denoising model based on the target model.

[0079] It should be noted that, in the present application, the selection method for the second target operation in the down-sampling cell body is consistent with the selection method for the first target operation in the up-sampling cell body, and will not be repeated here.

[0080] Consistent with the above, see Figure 4 , Figure 4 It is a flow chart of a model search method provided by an embodiment of the present application. As shown in the figure, the preset model includes: 4 upsampling cell bodies and 4 downsampling cell bodies, as well as multiple splicing operation modules and CConv modules. As shown in the figure, for the downsampling cell body, each input data is affected by the output data of the previous downsampling cell body in the previous process. Each upsampling cell body is affected not only by the output data of the previous upsampling cell body, but also by the output of its corresponding downsampling cell body; for example, for downsampling cell body 2, the corresponding upsampling cell body is upsampling cell body 2, the input data of downsampling cell body 2 is the output data y12 of downsampling cell body 1, and the input of upsampling cell body 2 is the data y24 obtained by splicing the output data y22 of upsampling cell body 3 and the output data y13 of downsampling cell body 2. Combined with Figure 1B As shown in the structural schematic diagram corresponding to the upsampling cell body and / or downsampling cell body, in the embodiment of the present application, sampling and screening of mixed operations can be implemented in each upsampling cell body and / or downsampling cell body, and in each model search process, each feature layer selects a pre-selected operation and combines it into a mixed operation of the upsampling cell body and / or downsampling cell body, and combines the connection method and data flow method of the preset model to complete the training of the preset model to obtain the target model, and image denoising processing can be implemented through the target model, which is beneficial to reducing the time of designing the image denoising network model and improving the image denoising effect.

[0081] See also Figure 5 , Figure 5 This is a structural diagram of an electronic device provided by an embodiment of the present application. As shown in the figure, the electronic device includes a processor, a memory, a communication interface, and one or more programs, which are applied to the electronic device, wherein the one or more programs are stored in the memory, and the one or more programs are configured to cause the processor to execute the following steps:

[0082] Determine an upsampling cell body and a downsampling cell body corresponding to a preset model, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers;

[0083] Determine a first mixing operation and a second mixing operation corresponding to each of the feature layers in the up-sampling cell body and the down-sampling cell body respectively;

[0084] When searching and training the preset model, searching for a plurality of preselected operations included in the first mixed operation to obtain a first target operation;

[0085] Searching a plurality of preselected operations included in the second mixed operation to obtain a second target operation;

[0086] A target model is determined according to the first target operation and the second target operation.

[0087] It can be seen that the electronic device described in the embodiment of the present application can determine the upsampling cell body and downsampling cell body corresponding to the preset model, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers; determine the first mixed operation and the second mixed operation corresponding to each of the feature layers in the upsampling cell body and the downsampling cell body; when searching and training the preset model, search the multiple pre-selected operations included in the first mixed operation to obtain the first target operation; search the multiple pre-selected operations included in the second mixed operation to obtain the second target operation; determine the target model based on the first target operation and the second target operation. In this way, in each mixed operation, one of the target operations can be selected to participate in the entire training, which is beneficial to reducing the memory usage, thereby reducing the amount of calculation, and is beneficial to searching for larger and more complex neural networks.

[0088] In a possible example, the upsampling cell body and / or the downsampling cell body includes: an input node, an intermediate node, and an output node.

[0089] In a possible example, the first target operation includes a first operation corresponding to a first type, a second operation corresponding to a second type, and a third operation corresponding to a third type;

[0090] In terms of searching for a plurality of pre-selected operations included in the first mixed operation to obtain a first target operation, the program includes instructions for executing the following steps:

[0091] determining, among a plurality of pre-selected operations included in the first mixed operation, a plurality of first pre-selected operations corresponding to the first type, a plurality of second pre-selected operations corresponding to the second type, and a plurality of third pre-selected operations corresponding to the third type;

[0092] When searching and training the preset model, selecting a first pre-selected operation from a plurality of first pre-selected operations corresponding to the first type, and determining the selected first pre-selected operation as the first operation;

[0093] Selecting a second pre-selected operation from a plurality of second pre-selected operations corresponding to the first type, and determining the selected first pre-selected operation as the second operation;

[0094] A third pre-selected operation is selected from a plurality of third pre-selected operations corresponding to the first type, and the selected first pre-selected operation is determined as the third operation.

[0095] In a possible example, the multiple feature layers include a first feature layer, a second feature layer, and a third feature layer; and the program further includes instructions for executing the following steps:

[0096] Determine that the feature layer from the input node to the intermediate node is the first feature layer, the feature layer from the input node to the intermediate node is the second feature layer, and the feature layer from the intermediate node to the output node is the third feature layer, wherein the first feature layer is used to complete any first pre-selected operation and / or the first operation among the multiple first pre-selected operations, the second feature layer is used to complete any second pre-selected operation and / or the second operation among the multiple second pre-selected operations, and the third feature layer is used to complete any third pre-selected operation and / or the third operation among the multiple third pre-selected operations.

[0097] In one possible example, the program further includes instructions for executing the following steps:

[0098] Each time the preset model is searched and trained, input data for the search and training is determined;

[0099] The input data passes through the input node, the intermediate node and the output node in sequence, and any first pre-selected operation among the multiple first pre-selected operations and / or the first operation, any second pre-selected operation among the multiple second pre-selected operations and / or the second operation, and any third pre-selected operation among the multiple third pre-selected operations and / or the third operation are completed in sequence to obtain output data.

[0100] In one possible example, in terms of selecting a first pre-selected operation from a plurality of first pre-selected operations corresponding to the first type and determining the selected first pre-selected operation as the first operation, the program includes instructions for performing the following steps:

[0101] Each time the preset model is searched and trained, a first pre-selected operation is arbitrarily selected from the plurality of first pre-selected operations according to a preset sampling function, and is applied to the first feature layer;

[0102] After completing the search training of the preset model according to each of the first pre-selected operations, determining a plurality of weighting coefficients corresponding to the plurality of first pre-selected operations;

[0103] A first pre-selected operation corresponding to a maximum weighting coefficient is selected from the plurality of weighting coefficients as the first operation.

[0104] The above mainly introduces the solution of the embodiment of the present application from the perspective of the execution process of the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0105] The embodiment of the present application can divide the functional units of the electronic device according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0106] In the case of dividing each functional module into corresponding functional modules, Figure 6 A schematic diagram of a model search device is shown, as Figure 6 As shown, the device is applied to electronic equipment, and the model search device 600 may include: a determination unit 601 and a search unit 602, wherein,

[0107] The determining unit 601 may be configured to support the terminal device in executing the above steps S201 , S202 , and S205 and / or other processes for the technology described herein.

[0108] The search unit 602 may be configured to support the terminal device in executing the above steps S203 - S204 , and / or other processes of the technology described herein.

[0109] It can be seen that the model search device provided in the embodiment of the present application can determine the upsampling cell body and downsampling cell body corresponding to the preset model, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers; determine the first mixed operation and the second mixed operation corresponding to each of the feature layers in the upsampling cell body and the downsampling cell body; when searching and training the preset model, search the multiple pre-selected operations included in the first mixed operation to obtain the first target operation; search the multiple pre-selected operations included in the second mixed operation to obtain the second target operation; determine the target model based on the first target operation and the second target operation. In this way, in each mixed operation, one of the target operations can be selected to participate in the entire training, which is beneficial to reducing the memory usage, thereby reducing the amount of calculation, and is beneficial to searching larger and more complex neural networks.

[0110] In one possible example, in one possible example, the first target operation includes a first operation corresponding to the first type, a second operation corresponding to the second type, and a third operation corresponding to the third type;

[0111] In terms of searching the plurality of pre-selected operations included in the first mixed operation to obtain the first target operation, the search unit 602 is specifically configured to:

[0112] determining, among a plurality of pre-selected operations included in the first mixed operation, a plurality of first pre-selected operations corresponding to the first type, a plurality of second pre-selected operations corresponding to the second type, and a plurality of third pre-selected operations corresponding to the third type;

[0113] When searching and training the preset model, selecting a first pre-selected operation from a plurality of first pre-selected operations corresponding to the first type, and determining the selected first pre-selected operation as the first operation;

[0114] Selecting a second pre-selected operation from a plurality of second pre-selected operations corresponding to the first type, and determining the selected first pre-selected operation as the second operation;

[0115] A third pre-selected operation is selected from a plurality of third pre-selected operations corresponding to the first type, and the selected first pre-selected operation is determined as the third operation.

[0116] In a possible example, the multiple feature layers include a first feature layer, a second feature layer, and a third feature layer; and the determining unit 601 is further configured to:

[0117] Determine that the feature layer from the input node to the intermediate node is the first feature layer, the feature layer from the input node to the intermediate node is the second feature layer, and the feature layer from the intermediate node to the output node is the third feature layer, wherein the first feature layer is used to complete any first pre-selected operation and / or the first operation among the multiple first pre-selected operations, the second feature layer is used to complete any second pre-selected operation and / or the second operation among the multiple second pre-selected operations, and the third feature layer is used to complete any third pre-selected operation and / or the third operation among the multiple third pre-selected operations.

[0118] In a possible example, the determining unit 601 is further configured to:

[0119] Each time the preset model is searched and trained, input data for the search and training is determined;

[0120] The input data passes through the input node, the intermediate node and the output node in sequence, and any first pre-selected operation among the multiple first pre-selected operations and / or the first operation, any second pre-selected operation among the multiple second pre-selected operations and / or the second operation, and any third pre-selected operation among the multiple third pre-selected operations and / or the third operation are completed in sequence to obtain output data.

[0121] In a possible example, in selecting a first pre-selected operation from a plurality of first pre-selected operations corresponding to the first type and determining the selected first pre-selected operation as the first operation, the search unit 602 is specifically configured to:

[0122] Each time the preset model is searched and trained, a first pre-selected operation is arbitrarily selected from the plurality of first pre-selected operations according to a preset sampling function, and is applied to the first feature layer;

[0123] After completing the search training of the preset model according to each of the first pre-selected operations, determining a plurality of weighting coefficients corresponding to the plurality of first pre-selected operations;

[0124] A first pre-selected operation corresponding to a maximum weighting coefficient is selected from the plurality of weighting coefficients as the first operation.

[0125] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0126] The electronic device provided in this embodiment is used to execute the above-mentioned model search method, and thus can achieve the same effect as the above-mentioned implementation method.

[0127] When integrated units are used, the electronic device may include a processing module, a storage module, and a communication module. The processing module may be used to control and manage the operation of the electronic device. For example, it may be used to support the electronic device in executing the steps performed by the determination unit 601 and the search unit 602. The storage module may be used to support the electronic device in executing and storing program code and data. The communication module may be used to support communication between the electronic device and other devices.

[0128] The processing module may be a processor or a controller. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, or a Wi-Fi chip.

[0129] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0130] The present application also provides a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may comprise an electronic device.

[0131] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0132] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0134] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0136] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0137] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0138] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A model search method, characterized in that: The method comprises: Determine an upsampling cell body and a downsampling cell body corresponding to a preset model, wherein the upsampling cell body and / or the downsampling cell body include multiple feature layers; Determine a first mixing operation and a second mixing operation corresponding to each of the feature layers in the up-sampling cell body and the down-sampling cell body respectively; When searching and training the preset model, multiple pre-selected operations included in the first mixed operation are searched to obtain a first target operation, where the first target operation includes a first operation corresponding to a first type, a second operation corresponding to a second type, and a third operation corresponding to a third type. Among the multiple pre-selected operations included in the first mixed operation, multiple first pre-selected operations corresponding to the first type, multiple second pre-selected operations corresponding to the second type, and multiple third pre-selected operations corresponding to the third type are determined. When searching and training the preset model, a first pre-selected operation is selected from the multiple first pre-selected operations corresponding to the first type, and the selected first pre-selected operation is determined as the first operation. Specifically, each time the preset model is searched and trained, a first pre-selected operation is arbitrarily selected from the multiple first pre-selected operations according to a preset sampling function and applied to the first feature layer. After completing the search and training of the preset model according to the first pre-selected operation, multiple weighting coefficients corresponding to the multiple first pre-selected operations are determined. The first pre-selected operation corresponding to the maximum weighting coefficient is selected from the multiple weighting coefficients as the first operation. Searching a plurality of preselected operations included in the second mixed operation to obtain a second target operation; A target model is determined according to the first target operation and the second target operation.

2. The method according to claim 1, characterized in that The upsampling cell body and / or the downsampling cell body includes: an input node, an intermediate node and an output node.

3. The method according to claim 2, characterized in that The searching of the plurality of pre-selected operations included in the first mixed operation to obtain a first target operation includes: selecting a second pre-selected operation from a plurality of second pre-selected operations corresponding to the second type, and determining the selected second pre-selected operation as the second operation; A third pre-selected operation is selected from a plurality of third pre-selected operations corresponding to the third type, and the selected third pre-selected operation is determined as the third operation.

4. The method according to claim 3, characterized in that The plurality of feature layers include the first feature layer, the second feature layer, and the third feature layer; the method further includes: Determine that the feature layer from the input node to the intermediate node is the first feature layer, the feature layer from the input node to the intermediate node is the second feature layer, and the feature layer from the intermediate node to the output node is the third feature layer, wherein the first feature layer is used to complete any first pre-selected operation and / or the first operation among the multiple first pre-selected operations, the second feature layer is used to complete any second pre-selected operation and / or the second operation among the multiple second pre-selected operations, and the third feature layer is used to complete any third pre-selected operation and / or the third operation among the multiple third pre-selected operations.

5. The method according to claim 4, characterized in that The method further comprises: Each time the preset model is searched and trained, input data for the search and training is determined; The input data passes through the input node, the intermediate node and the output node in sequence, and any first pre-selected operation among the multiple first pre-selected operations and / or the first operation, any second pre-selected operation among the multiple second pre-selected operations and / or the second operation, and any third pre-selected operation among the multiple third pre-selected operations and / or the third operation are completed in sequence to obtain output data.

6. A model search device, characterized in that: The device includes: a determination unit and a search unit, wherein: The determining unit is configured to determine an up-sampling cell body and a down-sampling cell body corresponding to a preset model, wherein the up-sampling cell body and / or the down-sampling cell body include multiple feature layers; The determining unit is further configured to determine a first mixing operation and a second mixing operation corresponding to each of the feature layers in the up-sampling cell body and the down-sampling cell body respectively; The search unit is configured to search for multiple preset operations included in the first mixed operation to obtain a first target operation when searching and training the preset model, where the first target operation includes a first operation corresponding to a first type, a second operation corresponding to a second type, and a third operation corresponding to a third type, wherein the multiple first pre-selected operations corresponding to the first type, the multiple second pre-selected operations corresponding to the second type, and the multiple third pre-selected operations corresponding to the third type are determined among the multiple pre-selected operations included in the first mixed operation; when searching and training the preset model, a first pre-selected operation is selected from the multiple first pre-selected operations corresponding to the first type, and the selected first pre-selected operation is determined as the first operation; specifically, each time the preset model is searched and trained, a first pre-selected operation is arbitrarily selected from the multiple first pre-selected operations according to a preset sampling function and applied to the first feature layer; after completing the search and training of the preset model according to the first pre-selected operation, a plurality of weighting coefficients corresponding to the multiple first pre-selected operations are determined; and the first pre-selected operation corresponding to the maximum weighting coefficient is selected from the multiple weighting coefficients as the first operation; The search unit is further configured to search for a plurality of preset operations included in the second mixed operation to obtain a second target operation; The determining unit is further configured to determine a target model according to the first target operation and the second target operation.

7. An electronic device, characterized in that: The method comprises a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 5.

9. A computer program product, wherein: The computer program product comprises a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute the method according to any one of claims 1 to 5.

Images