A network structure searching method, device and equipment and computer storage medium

Abstract

The embodiment of the application discloses a network structure searching method, device and equipment and a computer storage medium. The method comprises the following steps: constructing a search space of a preset network structure; searching and training the preset network structure according to the search space, determining a target operator by using a sharpness perception minimization method, and obtaining a target up-sampling module and a target down-sampling module according to the target operator; and determining a target network model based on the target up-sampling module and the target down-sampling module; wherein the target network model is used for image super-resolution processing on a to-be-processed image. In this way, the overfitting problem of architecture parameters in network structure searching can be solved, the network structure searching process can be simplified, the processing efficiency of image super-resolution application is improved, and the network model has strong generalization performance and migration performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural structure search, and particularly relates to a network structure search method and device, equipment and a computer storage medium. BACKGROUND

[0002] Deep learning has achieved great success in natural language processing, image recognition, speech recognition and other fields, which is closely related to the design of deep neural network structure. At present, most network structure designs are manually designed according to prior knowledge. In this case, the method of automatically generating neural networks has very important research significance, and people usually call this method of automatically generating neural network structure as network structure search (Neural Architecture Search, NAS).

[0003] In the related art, there are some network structure search methods, such as a neural network search-based image super-resolution method and a differentiable neural network automatic search method. Although these search methods can solve the neural network search problem, these search methods either have an extremely complex network structure and are difficult to implement, or the differences between different operators are not significant in the network parameter optimization process, which makes it difficult to select operators and is prone to overfitting. SUMMARY

[0004] The present application provides a network structure search method, device, equipment and computer storage medium, which can not only solve the overfitting problem of architecture parameters in network structure search, but also simplify the network structure search process, and make the network model have strong generalization performance and migratability.

[0005] The technical solution of the present application is implemented as follows:

[0006] In a first aspect, the present application provides a network structure search method, which comprises the following steps:

[0007] Constructing a search space of a preset network structure;

[0008] Searching and training the preset network structure according to the search space, determining a target operator by using a sharpness-aware minimization method, and obtaining a target up-sampling module and a target down-sampling module according to the target operator;

[0009] Determining a target network model based on the target up-sampling module and the target down-sampling module; wherein the target network model is used for image super-resolution processing of a to-be-processed image.

[0010] In a second aspect, an embodiment of the present application provides a network structure search device, which comprises a construction unit, a search unit and a determination unit; wherein,

[0011] The construction unit is configured to construct a search space of a preset network structure.

[0012] The search unit is configured to search and train the preset network structure according to the search space, determine a target operator by using a sharpness perception minimization method, and obtain a target up-sampling module and a target down-sampling module according to the target operator.

[0013] The determination unit is configured to determine a target network model based on the target up-sampling module and the target down-sampling module; wherein the target network model is used for image super-resolution processing on a to-be-processed image.

[0014] In a third aspect, an embodiment of the present application provides a chip, which comprises the network structure search device as described in the second aspect.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, which comprises a memory and a processor; wherein,

[0016] The memory is configured to store a computer program capable of running on the processor.

[0017] The processor is configured to execute the method as described in the first aspect when running the computer program.

[0018] In a fifth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program, and the computer program is executed by at least one processor to implement the method as described in the first aspect.

[0019] The network structure search method, device, equipment and computer storage medium provided in the embodiments of the present application can construct a search space of a preset network structure, search and train the preset network structure according to the search space, determine a target operator by using a sharpness perception minimization method, and obtain a target up-sampling module and a target down-sampling module according to the target operator; and then determine a target network model based on the target up-sampling module and the target down-sampling module; wherein the target network model is used for image super-resolution processing on a to-be-processed image. In this way, the preset network structure is searched and trained according to the search space, and the target up-sampling module and the target down-sampling module can be obtained after the target operator is determined, which not only can solve the overfitting problem of architecture parameters in network structure search, but also can simplify the network structure search process, thereby improving the processing efficiency of image super-resolution application, and enabling the network model to have strong generalization performance and migration performance. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 An image diagram for digital zooming using image super-resolution technology;

[0021] Figure 2 A network structure diagram based on neural network search;

[0022] Figure 3 An evolution process diagram of differentiable neural network automatic search;

[0023] Figure 4 An evolution trend diagram of architecture parameters;

[0024] Figure 5 A flow diagram of a network structure search method provided by an embodiment of the present application;

[0025] Figure 6 A loss function topography diagram of SAM optimization provided by an embodiment of the present application;

[0026] Figure 7 A network framework diagram of a target network model provided by an embodiment of the present application;

[0027] Figure 8 A specific implementation architecture diagram of a cell structure provided by an embodiment of the present application;

[0028] Figure 9 A specific transformation process diagram of a PixelShuffle operator provided by an embodiment of the present application;

[0029] Figure 10 A component structure diagram of a network structure search device provided by an embodiment of the present application;

[0030] Figure 11 A specific hardware structure diagram of an electronic device provided by an embodiment of the present application;

[0031] Figure 12 A component structure diagram of a chip provided by an embodiment of the present application;

[0032] Figure 13 A specific hardware structure diagram of a chip provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to enable persons skilled in the art to better understand the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings, which are only used for reference and do not limit the embodiments of the present application.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to be limiting of this application.

[0035] In the following description, reference is made to the accompanying drawings which form a part hereof, and in which are shown by way of illustration embodiments in which the subject matter of this application can be practiced. It is to be understood that other embodiments can be utilized and structural or logical changes can be made without departing from the scope of the present application. The following detailed description, therefore, is not to be taken in a limiting sense, as the scope of the present application is defined by the appended claims.

[0036] In recent years, with the popularization of high-definition devices, the resolution of user display devices has been generally improved to 2K or even higher. As a technology for making blurred images clear, image super resolution (SR) technology has been widely used in games, movies, cameras, medical images, and other fields. Through image super resolution technology, multimedia content can be reconstructed in high definition without consuming a large amount of resources. In the above process, only a small amount of modification is needed to achieve results comparable to manual design, greatly simplifying the work process and reducing the cost of work. On the other hand, image super resolution technology also has a wide range of applications in camera shooting processes, especially the zoom capability of the camera, which is an important indicator of shooting performance, is often valued by users. It can usually be divided into two parts: optical zoom and digital zoom. Among them, optical zoom adjusts the focal length by adjusting the lens. Due to the limitation of the size of the device, the adjustment capability is limited. In contrast, digital zoom adjusts the image through an algorithm to simulate optical zoom. The pros and cons of the algorithm largely determine the zoom factor and the quality of the result. That is, image super resolution technology can often provide a larger zoom factor and better image quality compared to traditional image interpolation algorithms, and has been widely adopted in recent years. For example, see Figure 1 , (a) shows an image taken using image super resolution technology, and the result obtained after digital zoom on the local area in the bold frame in the image is shown in (b). It can be seen that the local area is still clear.

[0037] For the image super-resolution technology, the image super-resolution specifically refers to using an intelligent method to magnify the image, so that the resolution is higher, and more clear details and textures are obtained than traditional magnification; or in the case of unchanged resolution, the compression noise is removed, and a more sharp and clean picture is obtained.

[0038] In the related art, the U-shaped network structure (U-shaped Network, UNET) is a deep learning framework, which can generally include the processes of encoding and decoding, and the size of the input and output remains unchanged. Exemplarily, Figure 2 A network structure schematic diagram based on neural architecture search (NAS) is shown. As Figure 2 As shown, the network structure is a NAS-based image super-resolution method, and the method adopts the UNET structure, and is also equipped with a channel attention block (CAB) and a superkernel-densely connected residual block (SK-DCRB). Each encoder layer has the same network structure. Specifically, it includes a convolution layer with a kernel size of 3x3, followed by an activation function and a SK-DCRB module, and the activation function can be a rectified linear unit (ReLU). Here, like the ordinary UNET structure, the output of the encoder layer is passed to the decoder layer at the same spatial level; each decoder layer obtains the output from the skip connection and the result from the previous layer. The difference from the ordinary UNET structure is that the CAB module will be applied after the cascading operation in each decoder layer. It should be noted that the filter number is represented by n, the step is represented by s, and the convolution layer with a step equal to 2 is used for downsampling, and the dashed arrow represents the subpixel layer for upsampling. In this way, the optimal convolution kernel size and channel number can be found through this neural network search method.

[0039] In addition, the related art also provides a differentiable architecture search (DARTS) technology. In which, the DARTS technology relaxes the discrete search space, so that the network search can be optimized in the form of gradient update, and finally solves the neural network search problem. Because of the efficiency of this method, it has been widely studied and used by scholars in the field. In this method, the search space is completed by stacking cells, Figure 3 An evolution process schematic diagram of the differentiable architecture search is shown. As Figure 3As shown, (a) defines a cell unit, which can be regarded as a directed acyclic graph, including 4 network nodes: 0, 1, 2, 3; the edges between the network nodes represent possible operations (for example: 3x3 convolution, max pooling, identity mapping, etc.), which are unknown at initialization (represented by?). (b) continuously relaxes the search space, and each edge is regarded as a mixture of all sub-operations. (c) jointly optimizes, updates the sub-operations, and jointly optimizes the mixture probability and network weights. Here, the thicker the edge between the network nodes represents the greater the mixture probability, the greater the corresponding weight, and the greater the possibility of being selected. (d) After optimization, the sub-operation with the maximum probability can be directly selected. Specifically, first, define each network node in the cell as a feature map, and connect the network nodes through algorithm operation. At the same time, the number of network nodes in each cell needs to be determined in this step, and the operator types in the search space are defined. Second, based on the given operators in the search space, each operator is assigned an initial value of the connection weight parameter, the connection between the network nodes is determined, and a directed acyclic graph is constructed to represent the structure of the cell. Third, update the weight parameters corresponding to each operator through the gradient update strategy based on the architecture parameters. Finally, after all parameters converge, select the operator with the maximum connection weight between each pair of network nodes as the operator in the final network. In this way, the required network structure can be searched. In this way, NAS-UNET searches the up-sampling cell and down-sampling cell in the UNET structure through the above search method of DARTS, and achieves good results in the medical image segmentation task.

[0040] However, for the NAS-based network structure, the technology adopts relatively complex channel attention mechanism structures, which are difficult to implement in hardware such as Application Specific Integrated Circuit (ASIC); moreover, the technology searches in two dimensions of convolution kernel size and channel number, and cannot determine that the operators in the preset network structure are all optimal. For the network structure of the DARTS technology, the architecture parameter update method used in the NAS-UNET method finds that the differences between the finally converged architecture parameter weights are not very significant in the processing of the Image Signal Processor (ISP), which makes it difficult to choose the operator, and this phenomenon can be seen from Figure 4 . Figure 4In this way, the weight coefficients of each operator herein converge to 0.167, resulting in difficulty in making a choice between the operators. A relatively intuitive optimization method is to simultaneously optimize the architecture parameters and the weight parameters in a single stage; however, due to the number of architecture parameters being much smaller than the number of weight parameters, the architecture parameters are prone to overfitting.

[0041] Based on this, an embodiment of the present application provides a network structure search method, which constructs a search space of a preset network structure; searches and trains the preset network structure according to the search space, determines a target operator by using a sharpness-aware minimization method, and obtains a target up-sampling module and a target down-sampling module according to the target operator; determines a target network model based on the target up-sampling module and the target down-sampling module; and the target network model is used for image super-resolution processing on a to-be-processed image. In this way, the preset network structure is searched and trained according to the search space, and the target up-sampling module and the target down-sampling module can be obtained after the target operator is determined, which not only solves the overfitting problem of the architecture parameters in the network structure search, but also simplifies the network structure search process, thereby improving the processing efficiency of the image super-resolution application, and enabling the network model to have strong generalization performance and migratability.

[0042] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0043] Reference is made to Figure 5 which shows a flowchart of a network structure search method provided by an embodiment of the present application. As shown in Figure 5 , the method can include the following steps.

[0044] S501: Construct a search space of a preset network structure.

[0045] It should be noted that the execution subject of the embodiments of the present application is a network structure search device, or an electronic device integrated with the device. The electronic device can be, for example, a smart phone, a tablet computer, a notebook computer, a palm computer, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a wearable device, a digital camera, etc., and is not limited herein.

[0046] It should be noted that the preset network structure described herein can refer to a neural network structure, and specifically can refer to a differentiable neural network automatic search UNET structure. Before constructing the search space, the search target needs to be determined first. In terms of the preset network structure, the search target of the embodiments of the present application is to search for a target up-sampling module and a target down-sampling module that are more friendly to the performance of the image super-resolution task. Here, the target up-sampling module and the target down-sampling module can replace the convolution (Conv) / activation (ReLU) / pooling (Pooling) structure in the encoder network (Encoding) and the convolution (Conv) / activation (ReLU) / up-sampling (Upsampling) structure in the decoder network (Decoding) of the ordinary UNET structure, respectively, so as to be used for constructing the backbone network of the image super-resolution processing.

[0047] In the embodiments of the present application, the preset network structure can include an initial up-sampling module and an initial down-sampling module. The initial up-sampling module and the initial down-sampling module are respectively composed of a plurality of network nodes and a plurality of operators. In this way, for the search space, in some embodiments, the search space of the preset network structure can include: determining the search space of the preset network structure according to the combination of the plurality of network nodes and the plurality of operators.

[0048] Further, in some embodiments, the operator can be one of the following: an up-sampling (Up Sampling) operator, a down-sampling (Down Sampling) operator, and an identity mapping (Identity Mapping) operator.

[0049] That is, for the initial up-sampling module and the initial down-sampling module, they can be composed of four network nodes and three operators (such as the up-sampling operator, the down-sampling operator, and the identity mapping operator), respectively. The random combination of these network nodes and operators constitutes the search space of the preset network structure.

[0050] S502: Search and train the preset network structure according to the search space, determine the target operator by using the sharpness-aware minimization method, and obtain the target up-sampling module and the target down-sampling module according to the target operator.

[0051] It should be noted that after the search space is determined, the preset network structure is searched and trained according to the search space, the target operator is determined, and specifically can include: searching and training the preset network structure according to the search space, and determining the target operator by using the sharpness-aware minimization method.

[0052] In the embodiments of the present application, the sharpness-aware minimization (SAM) method is a new second-order optimization method. The method considers that when the parameters converge to a flat region of the loss value, the network parameter optimization process is less likely to produce overfitting phenomenon, and the trained network model can achieve better accuracy and generalization performance. The SAM method has been verified on network structures such as residual network (ResNet), transformer model, and mixed multi-layer perceptron (Mixed-MLP). Since the SAM method minimizes the loss value and its sharpness at the same time, the model performance and generalization are improved.

[0053] In order to make the network model have strong generalization performance and migratability, in some embodiments, the searching and training the preset network structure according to the search space and determining the target operator by using the sharpness-aware minimization method can include:

[0054] searching and training the preset network structure according to the search space and updating the network parameters in the preset network structure by using the sharpness-aware minimization method;

[0055] when the loss values corresponding to all the network parameters in the search neighborhood range satisfy the preset condition, determining the optimized network parameters;

[0056] determining the target operator according to the optimized network parameters.

[0057] It should be noted that the embodiments of the present application can update the network parameters in the preset network structure by using the SAM method, and then determine the optimized network parameters when the loss values corresponding to all the network parameters in the search neighborhood range satisfy the minimization. Then, the target operator is determined according to the optimized network parameters.

[0058] Here, for the SAM method, from the mathematical point of view, the SAM redefinition defines the loss function of network optimization, as shown below.

[0059]

[0060] where w is the parameter to be optimized in the network, and ‖∈‖2≤ρ defines the search neighborhood range with the current parameter as the center and the radius ρ.

[0061] From the physical meaning, the SAM is not to minimize the loss value corresponding to the current parameter, but to minimize the loss values corresponding to all the parameters in a search neighborhood range of the current parameter. Exemplarily, Figure 6A loss function topography diagram of SAM optimization provided by an embodiment of the present application is shown. As shown in Figure 6 the current parameters are taken as examples of black points, and the space surrounded by the dashed line represents the search neighborhood range; in this way, for SAM optimization, all parameters in the search neighborhood range need to be minimized.

[0062] That is, the network parameters here include weight parameters and architecture parameters, and the SAM method is used to update the weight parameters and architecture parameters simultaneously until all parameters converge. Based on the SAM method for optimization, the minimum value in the search neighborhood range is searched, so that the convergence region of the loss function in the network training process is more gentle, which is conducive to improving the generalization performance of the model. For details, see Figure 6 shown.

[0063] It should be further pointed out that in the search training process, for the index measurement, the performance evaluation objective index of the network model can use the peak signal noise ratio (PSNR) and the structural similarity (SSIM) as the measurement index, but the present application is not limited in this regard. In addition, for the search objective function, the optimization objective function with energy consumption constraint can be defined as argmin L1, that is, the network loss is measured by the 1-norm between the network output and the given true value, but the present application is not limited in this regard. Here, it should be noted that the 1-norm represents the sum of the absolute values of the elements of the vector, and argmin L1 represents the variable value when the objective function L1 takes the minimum value.

[0064] Further, the present application can select a target operator according to the optimization trend and value of the architecture parameters. Therefore, in some embodiments, the determining a target operator according to the optimized network parameters can include: determining a target operator according to the optimized architecture parameters using a preset selection strategy.

[0065] It should be noted that the preset selection strategy can include a first selection strategy, a second selection strategy, and a third selection strategy, and the first selection strategy, the second selection strategy, and the third selection strategy are different. For example, taking the first connection between the network nodes as an example, the first selection strategy can refer to selecting the top-k operators in the first connection as the reserved operators, the second selection strategy can refer to selecting the operators corresponding to the architecture parameters with an upward trend as the reserved operators, and the third selection strategy can refer to selecting the operators that satisfy the first selection strategy and the second selection strategy at the same time as the reserved operators, and then discarding the remaining operators except the reserved operators. In the embodiments of the present application, since the network nodes are connected through the operator operation, the first connection between the network nodes is taken as an example for detailed description.

[0066] In a possible implementation, the determining the target operator by using the preset selection strategy can include:

[0067] selecting the top-k operators in the first connection between the network nodes as the reserved operators based on the first connection between the network nodes, and discarding the remaining operators except the reserved operators;

[0068] obtaining the target operator corresponding to the first connection according to the selected top-k operators; wherein k is an integer greater than zero.

[0069] In another possible implementation, the determining the target operator by using the preset selection strategy can include:

[0070] selecting the operators corresponding to the architecture parameters with an upward trend as the reserved operators based on the first connection between the network nodes, and discarding the remaining operators except the reserved operators;

[0071] obtaining the target operator corresponding to the first connection according to the selected reserved operators.

[0072] In yet another possible implementation, the determining the target operator by using the preset selection strategy can include:

[0073] selecting the top-k operators in the first connection between the network nodes and the operators corresponding to the architecture parameters with an upward trend as the reserved operators based on the first connection between the network nodes, and discarding the remaining operators except the reserved operators;

[0074] obtaining the target operator corresponding to the first connection according to the selected reserved operators.

[0075] That is, the embodiments of the present application provide three operator selection strategies:

[0076] The first selection strategy is to select top-k operators in the same connection as the final retained operators, splice and fuse the outputs of the operators as the final output, and discard the remaining operators that are not selected.

[0077] The second selection strategy is to observe the evolution trend of the architecture parameters in the network search process, select the operators corresponding to the architecture parameters with an upward trend, splice and fuse the outputs of the operators as the final output, and discard the remaining operators that are not selected.

[0078] The third selection strategy is to select operators that meet the first selection strategy and the second selection strategy at the same time, and splice and fuse the outputs of the operators as the final output of the current connection.

[0079] In this way, according to the above three operator selection strategies, the target operator (i.e., the retained operator) can be determined, and then the target up-sampling module and the target down-sampling module can be determined according to the retained operator.

[0080] S503: Determine a target network model based on the target up-sampling module and the target down-sampling module; wherein the target network model is used for image super-resolution processing on a to-be-processed image.

[0081] It should be noted that after the target up-sampling module and the target down-sampling module are obtained, the target network model can be determined. Specifically, in some embodiments, determining the target network model based on the target up-sampling module and the target down-sampling module can include:

[0082] reconstructing a preset network structure according to the target up-sampling module and the target down-sampling module to obtain a reconstructed network structure;

[0083] training the reconstructed network structure using a preset training set to obtain the target network model; wherein the preset training set includes at least one group of sample images, and each group of sample images includes a low-resolution image and a corresponding high-resolution image.

[0084] That is, the preset network structure is reconstructed according to the target up-sampling module and the target down-sampling module to obtain a reconstructed network structure; and then the reconstructed network structure is trained again using a preset training set, and the target network model for inference can be obtained.

[0085] Further, in some embodiments, after the target network model is obtained, the method can further include: performing inference verification on the target network model using a preset test data set, and observing the inference result, so as to evaluate the generalization performance of the network model through subjective indicators and objective indicators.

[0086] Further, in some embodiments, after obtaining the target network model, the method can further include: obtaining a to-be-processed image; and performing image super-resolution processing on the to-be-processed image by using the target network model to obtain a super-resolution image.

[0087] It should be noted that the target network model can be used to perform image super-resolution processing on the to-be-processed image. For example, assuming that the to-be-processed image is HxW, then the super-resolution image output after the target network model is 2Hx2W.

[0088] In addition, in the embodiments of the present application, in order to search for a more optimal network structure, when constructing the search space, in addition to the up-sampling operator, the down-sampling operator and the identity mapping operator, more lightweight replaceable operators (such as the Channel Shuffle operator) or operators friendly to the hardware of the neural-network processing unit (NPU) can be introduced, so as to further reduce the number of parameters of the network structure and make it easier to deploy.

[0089] The present embodiment provides a network structure searching method, which includes: constructing a search space of a preset network structure; searching and training the preset network structure according to the search space, determining a target operator by using a sharpness-aware minimization method, and obtaining a target up-sampling module and a target down-sampling module according to the target operator; and determining a target network model based on the target up-sampling module and the target down-sampling module; wherein the target network model is used to perform image super-resolution processing on a to-be-processed image. In this way, the preset network structure is searched and trained according to the search space, and after the target operator is determined, the target up-sampling module and the target down-sampling module can be obtained, which not only solves the overfitting problem of the architecture parameters in the network structure search, but also simplifies the network structure search process, thereby improving the processing efficiency of the image super-resolution application, and also enables the network model to have strong generalization performance and migratability.

[0090] It can be understood that based on the network structure searching method described in the foregoing embodiments, a kind of automatic search method of image super-resolution differentiable network structure based on sharpness-aware minimization is specifically proposed. Referring to Figure 7 , which shows a network framework schematic diagram of a target network model provided by an embodiment of the present application. As shown in Figure 7 , the target network model is similar to the NAS-UNET structure, and can include a header module, N target down-sampling modules, a transmission module, N target up-sampling modules and a tail module; wherein N is an integer greater than or equal to 1.

[0091] It should be noted that for the target network model, the connection relationship is as follows:

[0092] The head module is connected with the input of the first target down-sampling module, and the tail module is connected with the output of the first target up-sampling module.

[0093] When i is greater than or equal to 1 and less than N, the output of the i th target down-sampling module is connected with the input of the i+1 th target down-sampling module, and the output of the i th target down-sampling module is also connected with the input of the i th target up-sampling module through an identity mapping operation, and the input of the i th target up-sampling module is also connected with the output of the i+1 th target up-sampling module.

[0094] When i is equal to N, the output of the N th target down-sampling module is connected with the input of the N th target up-sampling module through an identity mapping operation, and the output of the N th target down-sampling module is also connected with the input of the transmission module, and the output of the transmission module is connected with the input of the N th target up-sampling module.

[0095] In the embodiments of the present application, the head module can include a convolution module with a step of 2 and an activation module; the transmission module can include a transpose convolution module with a step of 1 and an activation module; and the tail module can include a convolution module with a step of 1, an activation module and a Pixel Shuffle module. Wherein, the convolution module with a step of 2 is denoted as Convolution with stride=2, the convolution module with a step of 1 is denoted as Convolution with stride=1, and the transpose convolution module with a step of 1 is denoted as Transpose Convolution with stride=1.

[0096] In addition, it should be noted that the activation module can be a Rectified Linear Unit (ReLU) or a Leaky Rectified Linear Unit (Leaky ReLU). Wherein, the ReLU function sets all negative values to zero and keeps the rest unchanged; on the contrary, the Leaky ReLU function gives all negative values a non-zero slope, although the value is negative, but has a small gradient. In the embodiments of the present application, the activation module is specifically set according to the actual situation, and exemplarily, Figure 7 The activation module in the above formula is a Leaky ReLU function, but it is not limited to this.

[0097] It should be further noted that the N target down-sampling modules can be regarded as an encoder network, which has N layers, and each layer corresponds to a target down-sampling module; and the N target up-sampling modules can be regarded as a decoder network, which has N layers, and each layer corresponds to a target up-sampling module.

[0098] For example,Figure 7 For example, assuming N=4 and the input image is H×W, after the input image is input to the head module, its output is connected to the first-layer target downsampling module. The first-layer target downsampling module is connected to the second-layer target downsampling module, and after an identity mapping operation, it is also connected to the first-layer target upsampling module, and so on, until the fourth-layer target downsampling module is connected to the input of the transmission module. After an identity mapping operation, the fourth-layer target downsampling module is also connected to the fourth-layer target upsampling module. The output of the transmission module is connected to the fourth-layer target upsampling module, and the output of the fourth-layer target upsampling module is connected to the input of the third-layer target upsampling module. The output of the third-layer target upsampling module is connected to the input of the second-layer target upsampling module, and the output of the second-layer target upsampling module is connected to the input of the first-layer target upsampling module. Finally, the output of the first-layer target upsampling module is connected to the tail module, and the final output image is a super-resolution image (2H×2W).

[0099] In other words, this application aims to search for a downsampling module cell structure and an upsampling module cell structure that can achieve good performance in image super-resolution tasks. Subsequently, based on these searched cell structures, a compressed network model is reconstructed through multiple stacking operations, as follows: Figure 7 As shown. Additionally... Figure 8 This diagram illustrates a specific implementation architecture of a cell structure provided in an embodiment of this application. For example... Figure 8 As shown, this directed acyclic graph adopts the cell structure construction method of NAS-UNET and DARTS. Specifically, the cell structure includes input nodes, intermediate nodes, and output nodes. There are two input nodes: input node 0 and input node 1, only one output node, and three intermediate nodes: 1, 2, and 3. Moreover, each intermediate node (1, 2, 3) is obtained by weighted summation of its predecessors through edges, and the output node is obtained by concatenating each intermediate node.

[0100] It is also important to note that, for Figure 7 The target network model shown achieves a 2x decrease and a 2x increase in feature map size at the beginning and end of the network structure, respectively, through a combination of convolution / activation operators and a combination of convolution / activation / PixelShuffle operators. Here, the PixelShuffle operator maps a feature map tensor of size r^2×h×w to a feature map tensor of size rw×rh, thus achieving an r-fold upsampling of the image. Figure 9The specific transformation mode of this process is shown, i.e. transforming the original low-resolution image into a high-resolution image (i.e. the "super-resolution image" described in the foregoing embodiment).

[0101] In summary, in the embodiment of the present application, the specific implementation of the image super-resolution network search method based on sharpness perception minimization can include the following steps:

[0102] (1) Automatically search the network framework structure: In terms of network structure, the primary search target of the embodiment of the present application is to search a down-sampling module (or "down-sampling module") and an up-sampling module that are more friendly to the performance of the image super-resolution task. These two modules replace the Conv / ReLU / Pooling structure in the encoder network (Encoding) and the Conv / ReLU / Upsampling structure in the decoder network (Decoding) in the ordinary UNET structure, respectively, so as to be used for constructing the backbone network of image super-resolution processing.

[0103] (2) Search space definition: In terms of search space design, the down-sampling module and the up-sampling module to be searched in the embodiment of the present application are composed of four network nodes and three operators (up-sampling operator, down-sampling operator, identity mapping operator), respectively. The random combination of these nodes and operators constitutes the search space.

[0104] (3) Objective function definition: In terms of search objective function, the optimization objective function with energy consumption constraint is defined as argminL1, i.e. the 1-norm measurement of network loss between the output of the network model and the given true value.

[0105] (4) Metric definition: In the search process, the performance evaluation objective indicators of the network model can adopt peak signal-to-noise ratio and structural similarity as the metric indicators.

[0106] (5) Network parameter optimization: The entire network model contains two sets of parameters, i.e. weight parameters and architecture parameters. The embodiment of the present application adopts the optimization mode of sharpness perception minimization to update the architecture parameters and network weight parameters at the same time until all parameters converge. The optimization mode based on sharpness perception minimization is essentially searching for the minimum value in the neighborhood range, and based on this optimization mode, the loss function convergence region in the network training process is more gentle, which is conducive to improving the generalization performance of the model.

[0107] (6) According to the optimization of the architecture parameter trend and the value selection operator, the embodiment of the present application provides three alternative strategies for selecting the operator:

[0108] Selection strategy 1: select the top-k operators in the same connection as the final retained operators, and splice and fuse the outputs of these operators as the final output, while the remaining operators that are not selected are discarded.

[0109] Selection strategy 2: observe the evolution trend of the architecture parameters in the network search process, select the operators corresponding to the architecture parameters with an upward trend, splice and fuse the outputs of these operators as the final output, while the remaining operators that are not selected are discarded.

[0110] Selection strategy 3: select the operators that meet both selection strategy 1 and selection strategy 2, and splice and fuse the outputs of these operators as the final output of the current connection.

[0111] (7) Model construction and network retraining: based on all the retained operators, determine the target down-sampling module and target up-sampling module searched by the preset network structure search, and reconstruct the network structure. And retrain the reconstructed network structure to obtain the final target network model that can be used for inference.

[0112] (8) Network model inference: based on the target network model obtained in the last step, perform inference verification on the given test data set.

[0113] (9) Verify the generalization performance of the searched network structure: based on the cell structure searched in step (6), retrain on another data set, and observe the inference result, and evaluate the generalization performance of the network through subjective and objective indicators.

[0114] The embodiment provides a network structure search method, and the specific implementation of the foregoing embodiment is described in detail in the foregoing embodiment. As can be seen from the foregoing embodiment, the technical scheme of the foregoing embodiment can solve the problems of architecture parameter overfitting and optimization in the UNET-based image super-resolution network search based on the sharpness-aware minimization optimization method. Moreover, three feasible operator selection strategies are proposed to determine the network structure suitable for image super-resolution tasks. In this way, the technical scheme replaces the complicated differentiable search method in the related art in a simple and effective manner and is applied to the image super-resolution application task. Moreover, the use of sharpness-aware minimization enables the target network model obtained finally to have strong generalization performance and migratability.

[0115] Based on the same inventive concept as the foregoing embodiment, refer to Figure 10 , which shows a component structure schematic diagram of a network structure search device 100 provided by an embodiment of the application. As Figure 10 indicated, the network structure search device 100 can include a construction unit 1001, a search unit 1002, and a determination unit 1003; wherein,

[0116] The constructing unit 1001 is configured to construct a search space of a preset network structure.

[0117] The searching unit 1002 is configured to search and train the preset network structure according to the search space, determine a target operator by using a sharpness-aware minimization method, and obtain a target up-sampling module and a target down-sampling module according to the target operator.

[0118] The determining unit 1003 is configured to determine a target network model based on the target up-sampling module and the target down-sampling module, wherein the target network model is used for image super-resolution processing on a to-be-processed image.

[0119] In some embodiments, referring to Figure 10 The network structure searching apparatus 100 can further include a reconstructing unit 1004 and a training unit 1005, wherein

[0120] The reconstructing unit 1004 is configured to reconstruct the preset network structure according to the target up-sampling module and the target down-sampling module to obtain a reconstructed network structure.

[0121] The training unit 1005 is configured to train the reconstructed network structure by using a preset training set to obtain the target network model, wherein the preset training set includes at least one group of sample images, and each group of sample images includes a low-resolution image and a corresponding high-resolution image.

[0122] In some embodiments, the preset network structure includes an initial up-sampling module and an initial down-sampling module, and the initial up-sampling module and the initial down-sampling module are respectively composed of a plurality of network nodes and a plurality of operators; accordingly, the constructing unit 1001 is specifically configured to determine the search space of the preset network structure according to the combination of the plurality of network nodes and the plurality of operators.

[0123] In some embodiments, the searching unit 1002 is specifically configured to search and train the preset network structure according to the search space, update network parameters in the preset network structure by using a sharpness-aware minimization method, determine optimized network parameters when loss values corresponding to all network parameters in a search neighborhood range satisfy a preset condition, and determine the target operator according to the optimized network parameters.

[0124] In some embodiments, the network parameters include weight parameters and architecture parameters; accordingly, the determining unit 1003 is further configured to determine the target operator by using a preset selection strategy according to the optimized architecture parameters.

[0125] In some embodiments, the determining unit 1003 is further configured to select, based on the first connection between the network nodes, the first k operators in the first connection as the reserved operators, discard the remaining operators other than the reserved operators, and obtain the target operator corresponding to the first connection according to the selected first k operators; and k is an integer greater than zero.

[0126] In some embodiments, the determining unit 1003 is further configured to select, based on the first connection between the network nodes, the operator corresponding to the architecture parameter with an upward trend as the reserved operator, discard the remaining operators other than the reserved operator, and obtain the target operator corresponding to the first connection according to the selected reserved operator.

[0127] In some embodiments, the determining unit 1003 is further configured to select, based on the first connection between the network nodes, the first k operators in the first connection that satisfy the operator corresponding to the architecture parameter with an upward trend as the reserved operators, discard the remaining operators other than the reserved operators, and obtain the target operator corresponding to the first connection according to the selected reserved operators.

[0128] In some embodiments, the determining unit 1003 is further configured to obtain a to-be-processed image, and perform image super-resolution processing on the to-be-processed image by using the target network model to obtain a super-resolution image.

[0129] In some embodiments, the target network model comprises a head module, N target down-sampling modules, a transmission module, N target up-sampling modules, and a tail module, and N is an integer greater than or equal to 1; wherein,

[0130] The head module is connected to the input of the first target down-sampling module, the tail module is connected to the output of the first target up-sampling module, and when i is greater than or equal to 1 and less than N, the output of the i th target down-sampling module is connected to the input of the i+1 th target down-sampling module, and the output of the i th target down-sampling module is also connected to the input of the i th target up-sampling module after being operated by an identity mapping operation, and the input of the i th target up-sampling module is also connected to the output of the i+1 th target up-sampling module; and when i is equal to N, the output of the N th target down-sampling module is connected to the input of the N th target up-sampling module after being operated by an identity mapping operation, and the output of the N th target down-sampling module is also connected to the input of the transmission module, and the output of the transmission module is connected to the input of the N th target up-sampling module.

[0131] In some embodiments, the head module can comprise a convolution module with a step size of 2 and an activation module; the transmission module can comprise a transpose convolution module with a step size of 1 and an activation module; and the tail module can comprise a convolution module with a step size of 1, an activation module, and a pixel shuffle module.

[0132] It can be understood that, in this embodiment, the "unit" can be a partial circuit, a partial processor, a partial program or software, etc., and of course can also be a module, and can also be non-modular. Moreover, the components in this embodiment can be integrated in a processing unit, or can be physically present as individual units, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function module.

[0133] When the integrated unit is realized in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments can be embodied in the form of a software product in essence or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiments. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes or instructions.

[0134] Therefore, the embodiments provide a computer storage medium that stores a computer program, and the computer program, when executed by at least one processor, implements the steps of the network structure search method described in any one of the foregoing embodiments.

[0135] In another embodiment of the present application, based on the components of the foregoing network structure search device 100 and the computer storage medium, referring to Figure 11 , a specific hardware structure schematic diagram of an electronic device provided by the embodiments of the present application is shown. As Figure 11 indicated, the electronic device 110 can include a processor 1101, which can call and run a computer program from a memory to implement the network structure search method described in any one of the foregoing embodiments.

[0136] Optionally, as Figure 11 indicated, the electronic device 110 can further include a memory 1102. The processor 1101 can call and run a computer program from the memory 1102 to implement the network structure search method described in any one of the foregoing embodiments.

[0137] The memory 1102 can be a separate device independent of the processor 1101, or can be integrated in the processor 1101.

[0138] Optionally, as shown in Figure 11 The electronic device 110 can further include a transceiver 1103, which can be controlled by the processor 1101 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices.

[0139] The transceiver 1103 can include a transmitter and a receiver. The transceiver 1103 can further include an antenna, and the number of antennas can be one or more.

[0140] Optionally, the electronic device 110 can be a smart phone, a tablet computer, a palm computer, a notebook computer, a desktop computer, or the like, or a device integrated with the network structure search apparatus 100 according to any one of the foregoing embodiments. Here, the electronic device 110 can implement the corresponding processes described in the various methods according to the embodiments of the present application, and thus will not be described herein again for brevity.

[0141] In yet another embodiment of the present application, based on the composition of the network structure search apparatus 100 and the computer storage medium, in one possible example, referring to Figure 12 , a schematic diagram of the composition structure of a chip is shown. As shown in Figure 12 The chip 120 can include the network structure search apparatus 100 according to any one of the foregoing embodiments.

[0142] In another possible example, referring to Figure 13 , a specific hardware structure schematic diagram of a chip is shown. As shown in Figure 13 The chip 120 can include a processor 1201, which can call and run a computer program from a memory to implement the network structure search method according to any one of the foregoing embodiments.

[0143] Optionally, as shown in Figure 13 The chip 120 can further include a memory 1202. The processor 1201 can call and run a computer program from the memory 1202 to implement the network structure search method according to any one of the foregoing embodiments.

[0144] The memory 1202 can be a separate device independent of the processor 1201, or can be integrated in the processor 1201.

[0145] Optionally, as shown in Figure 13As shown, the chip 120 may also include an input interface 1203. The processor 1201 can control the input interface 1203 to communicate with other devices or chips, specifically, to acquire information or data sent by other devices or chips.

[0146] Optionally, such as Figure 13 As shown, the chip 120 may also include an output interface 1204. The processor 1201 can control the output interface 1204 to communicate with other devices or chips, specifically, to output information or data to other devices or chips.

[0147] Optionally, the chip 120 can be applied to the electronic device described in the foregoing embodiments, and the chip can implement the corresponding processes described in the various methods of the embodiments of this application. For the sake of brevity, these will not be repeated here.

[0148] It should be understood that the chip mentioned in the embodiments of this application may also be called a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc., without any limitation.

[0149] It should be noted that the processor in this application embodiment may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor mentioned above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as execution by a hardware decoding processor, or as execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0150] It is also important to note that the storage medium described in this application can be either volatile or nonvolatile storage media and can include the same as either removable storage or non-removable storage. By way of example, and not limitation, nonvolatile memory can be read only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Electrically EPROM (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which acts as external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), SynchBurst DRAM (SLDRAM), and direct Rambus RAM (DRRAM). It is noted that the system and method described herein are not limited to these and any other suitable type of memory.

[0151] It can be understood that, the embodiments described in the present application can be implemented in hardware, software, firmware, middleware, microcode or combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described in the present application, or a combination thereof. For software implementation, the technical solutions described in the present application can be implemented by modules (for example, processes, functions, etc.) that perform the functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0152] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0154] It should be noted that, in the present application, the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0155] The above-mentioned sequence number of the embodiments of the present application is only for description, not representing the advantages and disadvantages of the embodiments.

[0156] The methods disclosed in the several method embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments.

[0157] The features disclosed in the several product embodiments provided by the present application can be combined arbitrarily without conflict to obtain new product embodiments.

[0158] The features disclosed in the several method or device embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments or device embodiments.

[0159] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A network structure search method characterized by comprising: The method comprises: constructing a search space of a preset network structure; searching and training the preset network structure according to the search space, determining a target operator by using a sharpness perception minimization method, and obtaining a target up-sampling module and a target down-sampling module according to the target operator; determining a target network model based on the target up-sampling module and the target down-sampling module; wherein the target network model is used for image super-resolution processing on a to-be-processed image; the searching and training the preset network structure according to the search space, determining a target operator by using a sharpness perception minimization method, comprises: searching and training the preset network structure according to the search space, updating network parameters in the preset network structure by using a sharpness perception minimization method; when loss values corresponding to all network parameters in a search neighborhood range satisfy a preset condition, determining optimized network parameters; determining the target operator according to the optimized network parameters; wherein, based on a first connection between network nodes in the search space, the target operator is determined based on at least one of the following strategies: (a) selecting the first k operators in the first connection, (b) selecting an operator corresponding to an architecture parameter with an upward trend, and (c) selecting an operator that satisfies both (a) and (b). wherein the network parameters include weight parameters and architecture parameters, and the updating the network parameters in the preset network structure by using a sharpness perception minimization method comprises: simultaneously updating the weight parameters and the architecture parameters by using a sharpness perception minimization method.

2. The method of claim 1, wherein, the determining a target network model based on the target up-sampling module and the target down-sampling module, comprises: reconstructing the preset network structure according to the target up-sampling module and the target down-sampling module to obtain a reconstructed network structure; training the reconstructed network structure by using a preset training set to obtain the target network model; wherein the preset training set comprises at least one group of sample images, and each group of sample images comprises a low-resolution image and a corresponding high-resolution image.

3. The method of claim 1, wherein, The preset network structure comprises an initial up-sampling module and an initial down-sampling module, and the initial up-sampling module and the initial down-sampling module are respectively composed of a plurality of network nodes and a plurality of operators; Correspondingly, the constructing a search space of a preset network structure comprises: determining the search space of the preset network structure according to combinations of the plurality of network nodes and the plurality of operators.

4. The method of claim 1, wherein, The determining the target operator according to the optimized network parameters comprises: determining the target operator by using a preset selection strategy according to the optimized architecture parameters.

5. The method of claim 4, wherein, The determining the target operator by using a preset selection strategy comprises: based on the first connection between network nodes, selecting the first k operators in the first connection as reserved operators, and discarding the remaining operators other than the reserved operators; obtaining the target operator corresponding to the first connection according to the selected first k operators; wherein k is an integer greater than zero.

6. The method of claim 4, wherein, The determining the target operator by using a preset selection strategy comprises: select, based on a first connection between network nodes, an operator corresponding to an architecture parameter with an upward trend as a reserved operator, and discard the remaining operators other than the reserved operator; obtain, according to the selected reserved operator, that the first connection corresponds to the target operator.

7. The method of claim 4, wherein, The method further comprises: obtaining a to-be-processed image; performing image super-resolution processing on the to-be-processed image by using the target network model to obtain a super-resolution image.

8. The method of claim 1, wherein, The target network model comprises a head module, N target down-sampling modules, a transmission module, N target up-sampling modules, and a tail module, where N is an integer greater than or equal to 1; wherein the head module is connected to the input of the first target down-sampling module, and the tail module is connected to the output of the first target up-sampling module; when i is greater than or equal to 1 and less than N, the output of the i-th target down-sampling module is connected to the input of the i+1-th target down-sampling module, and the output of the i-th target down-sampling module is also connected to the input of the i-th target up-sampling module after being operated by an identity mapping operation, and the input of the i-th target up-sampling module is also connected to the output of the i+1-th target up-sampling module; 9. The method according to any one of claims 1 to 8, characterized in that, when i is equal to N, the output of the N-th target down-sampling module is connected to the input of the N-th target up-sampling module after being operated by an identity mapping operation, and the output of the N-th target down-sampling module is also connected to the input of the transmission module, and the output of the transmission module is connected to the input of the N-th target up-sampling module.

10. The method of claim 9, wherein the head module comprises a convolution module with a step size of 2 and an activation module; the transmission module comprises a transpose convolution module with a step size of 1 and an activation module; the tail module comprises a convolution module with a step size of 1, an activation module, and a pixel shuffle module. The network structure searching device comprises a construction unit, a searching unit, and a determination unit; wherein the construction unit is configured to construct a search space of a preset network structure; the searching unit is configured to search and train the preset network structure according to the search space, determine a target operator by using a sharpness-aware minimization method, and obtain a target up-sampling module and a target down-sampling module according to the target operator; 11. A network structure search apparatus characterized by comprising: the determination unit is configured to determine a target network model based on the target up-sampling module and the target down-sampling module, wherein the target network model is used for image super-resolution processing on a to-be-processed image; ​ ​ ​ The searching unit is specifically configured to search and train the preset network structure according to the search space, update network parameters in the preset network structure by using a sharpness-aware minimization method, and determine optimized network parameters when loss values corresponding to all network parameters in a search neighborhood range satisfy a preset condition; determine the target operator according to the optimized network parameters; and determine the target operator based on at least one of the following strategies based on first connections between network nodes in the search space: (a) selecting the first k operators in the first connections, (b) selecting an operator corresponding to an architecture parameter with an upward trend, and (c) selecting an operator that satisfies both (a) and (b). The network parameters include weight parameters and architecture parameters, and the searching unit is further configured to update the weight parameters and the architecture parameters simultaneously by using the sharpness-aware minimization method.

12. A chip, characterized by The chip includes the network structure searching device of claim 11.

13. An electronic device, comprising: The electronic device includes a memory and a processor; wherein The memory is configured to store a computer program capable of running on the processor; The processor is configured to execute the method of any one of claims 1 to 10 when running the computer program.

14. A computer storage medium, characterized in that The computer storage medium stores a computer program, and the computer program is executed by at least one processor to implement the method of any one of claims 1 to 10.

Images