Network model hierarchy division method and device, equipment, medium and product

Abstract

The invention discloses a network model hierarchy division method and device, equipment, a medium and a product. The method comprises the following steps: obtaining a target network model to be subjected to hierarchical division, wherein the target network model comprises at least one convolutional layer; inputting an input image into the target network model, and performing reverse image generation to obtain a target image corresponding to each convolutional layer; the entropy of each target image is calculated; and performing hierarchical division on the target network model according to the entropy of each target image. According to the technical scheme of the invention, the method can achieve the specific deep and shallow classification of the quantification neural network model, and can achieve the reasonable and effective deep and shallow classification of the convolution layers in the neural network model.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of neural network technology, and in particular to a method, device, equipment, medium and product for dividing a network model into levels. Background Art

[0002] Existing deep neural network architectures are difficult to deploy on resource-constrained edge devices, so they need to be pruned reasonably and effectively before deployment.

[0003] The existing pruning schemes usually perform structured pruning or unstructured pruning, using different evaluation criteria to prune weights or filters. However, the network structures are complexly interconnected, and a single weight / filter importance evaluation criterion is biased by humans. The final retained structure / parameters are often difficult to optimize, and the generalization of the compressed model is poor. In addition, there are obvious differences in the features extracted by different layers of convolutional neural networks. It is generally believed that using the same evaluation criterion on each layer of the entire network will destroy the network's ability to extract features. At present, the shallow and deep division methods of the target network have not been fully studied and defined, and are only relative concepts. The existing shallow and deep division methods of the target network are relatively simple and crude. The default method is generally to divide according to the type of layer, such as the input layer and the first few hidden layers of the target neural network as shallow layers, and the fully connected layers close to the output layer as deep layers. There are also only conceptual fuzzy divisions, where the shallow layer extracts edge textures and the deep layer is high-level semantics.

[0004] In summary, the existing shallow-deep network structure division method has the following problems: the shallow-deep division method is not fully defined and is only a relative concept; the division is based only on the layer type, which is too simple; the internal feature extraction of the network is ignored and difficult to explain; there is a lack of a definitive definition of shallow and deep layers. This leads to the following problems in the existing pruning technology: the feature extraction of different layers of the same network is obviously different, the existing weight evaluation standard is relatively single and global, and lacks a local perspective; the existing evaluation standard has artificial bias. Therefore, it is urgent to design a reasonable and effective way to divide the shallow and deep layers. Summary of the invention

[0005] The embodiments of the present invention provide a network model hierarchical division method, device, equipment, medium and product to realize the deep and shallow layer classification method of a specific quantitative neural network model, which can reasonably and effectively classify the deep and shallow layers of the convolutional layers in the neural network model.

[0006] According to one aspect of the present invention, a method for dividing a network model into layers is provided, comprising:

[0007] Obtaining a target network model to be hierarchically divided, wherein the target network model includes at least one convolutional layer;

[0008] Input the input image into the target network model, perform inverse image generation, and obtain the target image corresponding to each convolution layer;

[0009] Calculating the entropy of each of the target images;

[0010] The target network model is hierarchically divided according to the entropy of each target image.

[0011] In some embodiments of the present invention, inputting an input image into the target network model and performing inverse image generation to obtain a target image corresponding to each convolutional layer includes:

[0012] Inputting the input image into the target network model to obtain an activation matrix output by a target layer, wherein the target layer is any convolutional layer in the target network model;

[0013] determining a loss function according to the activation matrix;

[0014] Obtaining a gradient value of the loss function relative to the input image;

[0015] Updating the input image by using the gradient value to obtain an updated image;

[0016] Adding a target regularization term to the activation matrix to obtain an updated loss function;

[0017] The updated image is used as a new input image, and the updated loss function is used as a new loss function. Return to execute the operation of inputting the input image into the target network model to obtain the activation matrix output by the target layer, until the number of iterations reaches a preset number, and the target image corresponding to the target layer is obtained.

[0018] In some embodiments of the present invention, calculating the entropy of each target image includes:

[0019] For each of the target images, convert the target image into a grayscale image, and obtain the number of pixels corresponding to each grayscale pixel in the grayscale image;

[0020] Determine the frequency corresponding to each grayscale pixel according to the number of pixels corresponding to each grayscale pixel;

[0021] Determine the probability distribution corresponding to each grayscale pixel according to the frequency corresponding to each grayscale pixel;

[0022] The entropy of the target image is determined according to the probability distribution corresponding to each grayscale pixel.

[0023] In some embodiments of the present invention, hierarchical division of the target network model is performed according to the entropy of each target image, including:

[0024] If the entropy of the target image is greater than or equal to a preset entropy threshold, the target layer is divided into a shallow layer;

[0025] If the entropy of the target image is less than the entropy threshold, the target layer is divided into a deep layer.

[0026] In some embodiments of the present invention, determining a loss function according to the activation matrix includes:

[0027] Obtain the number of filters and the number of channels corresponding to the target layer;

[0028] A loss function is determined based on the number of filters, the number of channels, and the activation matrix.

[0029] In some embodiments of the present invention, updating the input image by using the gradient value to obtain an updated image includes:

[0030] Determining an update step size according to a preset learning rate, the gradient value, and the input image;

[0031] An updated image is determined according to the input image and the update step size.

[0032] According to another aspect of the present invention, a network model hierarchical division device is provided, the device comprising:

[0033] An acquisition module, used to acquire a target network model to be hierarchically divided, wherein the target network model includes at least one convolutional layer;

[0034] An input module, used to input the input image into the target network model, perform inverse image generation, and obtain a target image corresponding to each convolution layer;

[0035] A calculation module, used for calculating the entropy of each target image;

[0036] A division module is used to divide the target network model into levels according to the entropy of each target image.

[0037] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0038] at least one processor; and

[0039] a memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the network model hierarchical division method described in any embodiment of the present invention.

[0041] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the network model hierarchical division method described in any embodiment of the present invention when executed.

[0042] According to another aspect of the present invention, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the network model hierarchical division method described in any embodiment of the present invention.

[0043] The embodiment of the present invention obtains a target network model including a convolutional layer to be hierarchically divided, inputs an input image into the target network model for reverse image generation, obtains a target image corresponding to each convolutional layer, then calculates the entropy of each target image, performs quantitative analysis on the information amount of the image, and divides the target network model into deep and shallow layers according to the entropy of each target image. Through the technical solution of the present invention, a specific deep and shallow layer classification method of a quantitative neural network model can be implemented, and a reasonable and effective deep and shallow layer classification of the convolutional layers in the neural network model can be performed.

[0044] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 is a flow chart of a network model hierarchical division method in an embodiment of the present invention;

[0047] Figure 2 It is a structural schematic diagram of a network model hierarchical division device in an embodiment of the present invention;

[0048] Figure 3 It is a structural diagram of an electronic device that implements the network model hierarchical division method of an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and their meanings are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0051] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0052] Embodiment 1

[0053] Figure 1 is a flowchart of a network model hierarchical division method in an embodiment of the present invention. This embodiment is applicable to the situation of deep neural network hierarchical structure division. The method can be executed by the network model hierarchical division device in an embodiment of the present invention. The device can be implemented in software and / or hardware, such as Figure 1 As shown, the method specifically comprises the following steps:

[0054] S101, obtaining a target network model to be hierarchically divided.

[0055] Among them, the target network model includes at least one convolutional layer.

[0056] It can be known that the purpose of dividing the network model into layers is to perform personalized pruning, and pruning usually only involves convolutional layers and fully connected layers with more computational complexity. Generally speaking, the fully connected layers of deep neural network models are placed in the last classification layer, so they are usually classified as deep layers. Therefore, the embodiment of the present invention will mainly classify the convolutional layers, that is, mainly divide the convolutional layers into layers. In this embodiment, the target network model can be a deep neural network model containing convolutional layers.

[0057] In actual operation, the target network model can be a trained network model, and the model can be prepared in advance before performing hierarchical division. Specifically, the model preparation stage can be:

[0058] The target network model to be hierarchically divided is pre-trained using the training data. For example, the target network model is a deep neural network f θ , consisting of L layers, the target network model can be formally defined as follows:

[0059] f θ (X) = f L f L-1 …f1(X);

[0060] Among them, X represents the input image or input feature, θ represents the parameters of the model, such as weights and biases, and each layer f l By parameter θ l =(W l ,b l ), usually the network model is composed of multiple layers, such as convolutional layer, activation layer, pooling layer, fully connected layer and other basic layers. The corresponding operations between layers usually include convolution operations, nonlinear activation functions, pooling operations, etc.

[0061] S102: Input the input image into the target network model to perform inverse image generation to obtain the target image corresponding to each convolutional layer.

[0062] The embodiment of the present invention will mainly divide the convolutional layers into levels, that is, the embodiment of the present invention will only reversely generate images for the convolutional layers layer by layer.

[0063] In this embodiment, the input image may be an image input into the target network model for activating specific features of the convolutional layer in the target network model. Preferably, the input image may be a random noise image. In the inverse image generation process, the random noise image is selected instead of the image of the original data set as the input of the trained target network model. The ultimate goal is to optimize the image so that it can activate specific features of the convolutional layer, thereby determining whether the convolutional layer recognizes basic features or high-level semantic features.

[0064] It can be known that reverse image generation generally refers to the process of generating a corresponding image from given data or features. In this embodiment, reverse image generation may refer to the process of activating the convolutional layer in the target network model from a given random noise image to generate a corresponding target image.

[0065] Among them, the target image can be the image finally generated by the convolution layer.

[0066] Specifically, a random noise image is selected as an input image, the input image is input into the target network model, the convolutional layer in the target network model is activated, and reverse image generation is performed to obtain the target image finally generated by each convolutional layer.

[0067] S103: Calculate the entropy of each target image.

[0068] It should be noted that image entropy is the sum of the entropies of the probability distribution of all pixel values ​​in the image.

[0069] It is known that image entropy is an important concept in information theory, which is used to measure the uncertainty or randomness of information in an image. The higher the image entropy, the more information the image contains, the richer the details of the image, and the more complex it looks visually.

[0070] In the implementation process, after obtaining the target image finally generated by each convolution layer, each convolution layer is classified by analyzing the target image reversely generated by each convolution layer, that is, hierarchical division is performed. Specifically, the embodiment of the present invention uses entropy to quantify the amount of information in the reversely generated target image, so as to perform deep and shallow division of each convolution layer.

[0071] S104, hierarchically dividing the target network model according to the entropy of each target image.

[0072] In actual operation, the entropy of an image usually reflects the amount of information in the image. Shallow features usually retain more local details, so the entropy is relatively high, while deep features usually retain abstract information, so the entropy is relatively low. Specifically, each convolutional layer is classified into deep and shallow layers according to the entropy of the target image generated by each convolutional layer, thereby realizing the hierarchical division of the entire target network model.

[0073] The embodiment of the present invention obtains a target network model including a convolutional layer to be hierarchically divided, inputs an input image into the target network model for reverse image generation, obtains a target image corresponding to each convolutional layer, then calculates the entropy of each target image, performs quantitative analysis on the information amount of the image, and divides the target network model into deep and shallow layers according to the entropy of each target image. Through the technical solution of the present invention, a specific deep and shallow layer classification method of a quantitative neural network model can be implemented, and a reasonable and effective deep and shallow layer classification of the convolutional layers in the neural network model can be performed.

[0074] Optionally, the input image is input into the target network model to perform inverse image generation to obtain the target image corresponding to each convolutional layer, including:

[0075] Input the input image into the target network model and get the activation matrix of the target layer output.

[0076] Among them, the target layer is any convolutional layer in the target network model.

[0077] In this embodiment, the input image can be input into the target network model, a standard forward propagation is performed using the current input image, the target layer is selected, and all activation values ​​of the target layer are calculated. The output of the target layer can be expressed as A l (x) represents the activation matrix of the target layer, which contains the activation values ​​of all neurons.

[0078] Determine the loss function based on the activation matrix.

[0079] The loss function may be a loss function of the target network model, which may be represented by L(x). The function of the loss function is to maximize the sum of all activation values ​​of the entire target layer.

[0080] It should be noted that this loss function is the initial loss function and does not contain a regularization term, that is, it directly maximizes the activation value of the target layer. The purpose is to make the input image activate specific layers as much as possible and generate feature images that purely reflect these layers.

[0081] Specifically, the loss function of the target network model is calculated based on the activation matrix output by the convolutional layer.

[0082] Get the gradient of the loss function with respect to the input image.

[0083] In this embodiment, the gradient value of the loss function relative to the input image is calculated through back propagation, and the gradient value represents the influence of each element (ie, pixel point) in the input image on the activation value of the target layer.

[0084] Specifically, the calculation method of the gradient value of the loss function relative to the input image can be expressed as:

[0085]

[0086] Among them, L(x) represents the loss function, x represents the input image, Represents the gradient value of the loss function with respect to the input image.

[0087] Update the input image by the gradient value to obtain the updated image.

[0088] In actual operation, in order to find the input image that maximizes the activation of the target layer, the gradient ascent method can be used to update the input image x so that the activation value of the entire convolutional layer increases.

[0089] Specifically, the input image can be updated by the calculated gradient value, so that the activation value of the entire convolutional layer increases.

[0090] Add the target regularization term to the activation matrix to obtain the updated loss function.

[0091] In this embodiment, the target regularization term may be an L2 regularization term.

[0092] In actual operation, since the initial loss function lacks any constraints, the generated images may contain a lot of high-frequency noise or unnatural details. These noises are generated by the target network model in order to maximize the activation value, but they usually do not conform to people's intuitive understanding of natural images. Therefore, after the initial pure activation maximization, it is necessary to add a regularization term to constrain the image generation process, help control the complexity and structure of the image, and make the generated image more natural and intuitive.

[0093] Exemplarily, in this embodiment, L2 can be selected as the regularization method to constrain the pixel values ​​of the input image, so the entire loss function can be updated as:

[0094]

[0095] Here, λ represents a regularization weight, which can be set by the user according to actual needs or experience values, and is not limited in this embodiment.

[0096] The updated image is used as the new input image, the updated loss function is used as the new loss function, and the input image is input into the target network model to obtain the activation matrix output by the target layer, until the number of iterations reaches the preset number, and the target image corresponding to the target layer is obtained.

[0097] The number of iterations may be the number of times the process of inversely generating an image is repeated after the input image is updated and the loss function is updated. It should be noted that the input image and the loss function need to be updated in each iteration.

[0098] In this embodiment, the preset number of times may be a loop iteration number threshold value pre-set by the user according to actual needs or experience. When the actual number of iterations reaches this threshold, it is considered that the loss function has reached convergence and the iteration is stopped. This embodiment does not limit the specific value of the preset number of times.

[0099] Specifically, after multiple iterations, the final generated image will show the features captured by the entire target layer. These features are the integrated results of all filters in the target layer and can reflect the common response of the target layer to the input image.

[0100] Optionally, calculate the entropy of each target image, including:

[0101] For each target image, convert the target image into a grayscale image, and obtain the number of pixels corresponding to each grayscale pixel in the grayscale image.

[0102] Specifically, if the target image generated by reverse engineering is a color image, it is converted to a grayscale image. The following formula can be used for conversion:

[0103] Gray(i,j)=0.299·R(i,j)+0.578·G(i,j)+0.114·B(i,j);

[0104] Among them, R, G, and B represent the pixel values ​​of the red, green, and blue channels respectively.

[0105] The frequency corresponding to each grayscale pixel is determined according to the number of pixels corresponding to each grayscale pixel.

[0106] Specifically, you can create a histogram array and initialize a one-dimensional array of size 256 (assuming an 8-bit image) to store the number of pixels for each grayscale pixel: hist = [0,0,…0] (length is 256). Then traverse each pixel of the target image, count the number of occurrences of each grayscale pixel, and calculate the frequency corresponding to each grayscale pixel. The calculation method for the frequency corresponding to each grayscale pixel can be expressed as: pixel(i,j):hist[Gray(i,j)]+=1.

[0107] The probability distribution corresponding to each grayscale pixel is determined according to the frequency corresponding to each grayscale pixel.

[0108] Specifically, the frequency corresponding to each grayscale pixel is converted into the probability distribution corresponding to each grayscale pixel: Where hist[i] is the number of pixels with grayscale value i, and H and W are the height and width of the target image, respectively.

[0109] The entropy of the target image is determined based on the probability distribution corresponding to each grayscale pixel.

[0110] In actual operation, the entropy of the target image is calculated according to the probability distribution corresponding to each grayscale pixel: Among them, p(i) is the probability of gray value i.

[0111] Optionally, the target network model is hierarchically divided according to the entropy of each target image, including:

[0112] If the entropy of the target image is greater than or equal to the preset entropy threshold, the target layer is classified as a shallow layer.

[0113] The entropy threshold may be a threshold of the target image entropy pre-set by the user according to actual needs or experience, and this embodiment does not limit the specific value of the entropy threshold.

[0114] For example, in the embodiment of the present invention, the entropy threshold of the target image generated by reverse engineering can be defined as 4, that is, if the entropy value H of the target image generated by reverse engineering of the target layer is entropy ≥4, the target layer is classified as shallow.

[0115] If the entropy of the target image is less than the entropy threshold, the target layer is divided into a deep layer.

[0116] For example, if the entropy value H of the target image generated by the target layer is entropy <4, the target layer is classified as a deep feature.

[0117] Optionally, a loss function is determined based on the activation matrix, including:

[0118] Get the number of filters and channels corresponding to the target layer.

[0119] The number of filters may be the number of filters in the convolution layer, which may be represented by n in this embodiment. The number of channels may be the number of all channels in the convolution layer, which may be represented by j in this embodiment.

[0120] The loss function is determined based on the number of filters, number of channels, and activation matrix.

[0121] Specifically, the way to calculate the loss function based on the number of filters, the number of channels, and the activation matrix can be expressed as:

[0122]

[0123] Among them, L(x) represents the loss function, n represents the number of filters in the convolutional layer, j represents all the channels of the convolutional layer, and the activation matrix of the convolutional layer can be expressed as A l (x). In this embodiment, the loss function can be determined by calculating the sum of activation values ​​of all channels of all filters in the convolution layer.

[0124] Optionally, the input image is updated by the gradient value to obtain an updated image, including:

[0125] The update step size is determined based on the preset learning rate, gradient value, and input image.

[0126] The learning rate may be a value preset by the user according to actual needs or experience, and is used to control the update step size of the input image in each iteration. This embodiment does not limit the specific value of the learning rate.

[0127] It should be noted that the step size can be updated to find the input image that maximizes the activation of the target layer, and the gradient ascent method is used to update the step size of the input image in each iteration.

[0128] Specifically, the method of calculating the update step size according to the preset learning rate, gradient value and input image can be described as: the gradient value is derived from the input image and then multiplied by the learning rate to obtain the result.

[0129] The updated image is determined based on the input image and the update step size.

[0130] Specifically, the formula for updating the image can be expressed as: Where η is the learning rate, which controls the update step size of the image in each iteration, L(x) represents the loss function, x represents the input image, and x old represents the input image before updating, x new represents the updated input image.

[0131] The technical solution of the embodiment of the present invention designs a reasonable and effective way to divide the shallow layer into the deep layer, classifies the main layers (convolutional layers) in the deep neural network model into shallow and deep layers, and specifically quantifies the classification method, filling the ambiguity and gaps in this research. Through the technical solution of the present invention, it is possible to implement a specific quantification method for classifying the deep and shallow layers of the neural network model, and to reasonably and effectively classify the deep and shallow layers of the convolutional layers in the neural network model.

[0132] Embodiment 2

[0133] Figure 2 Schematic diagram of a network model hierarchical division device in an embodiment of the present invention. This embodiment is applicable to the situation of deep neural network hierarchical division. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides the function of network model hierarchical division, such as Figure 2 As shown, the network model hierarchical division device specifically includes: an acquisition module 201, an input module 202, a calculation module 203 and a division module 204.

[0134] The acquisition module 201 is used to acquire a target network model to be hierarchically divided, wherein the target network model includes at least one convolutional layer;

[0135] An input module 202 is used to input an input image into the target network model to perform inverse image generation to obtain a target image corresponding to each convolutional layer;

[0136] A calculation module 203, used for calculating the entropy of each target image;

[0137] The division module 204 is used to divide the target network model into levels according to the entropy of each target image.

[0138] Optionally, the input module 202 includes:

[0139] An input unit, used to input an input image into the target network model to obtain an activation matrix output by a target layer, wherein the target layer is any convolutional layer in the target network model;

[0140] A determination unit, used to determine a loss function according to the activation matrix;

[0141] An acquisition unit, used to acquire a gradient value of the loss function relative to the input image;

[0142] An updating unit, configured to update the input image using the gradient value to obtain an updated image;

[0143] An adding unit, used for adding a target regularization term to the activation matrix to obtain an updated loss function;

[0144] The execution unit is used to use the updated image as a new input image and the updated loss function as a new loss function, return to execute the operation of inputting the input image into the target network model to obtain the activation matrix output by the target layer, until the number of iterations reaches a preset number, and the target image corresponding to the target layer is obtained.

[0145] Optionally, the calculation module 203 is specifically used for:

[0146] For each of the target images, convert the target image into a grayscale image, and obtain the number of pixels corresponding to each grayscale pixel in the grayscale image;

[0147] Determine the frequency corresponding to each grayscale pixel according to the number of pixels corresponding to each grayscale pixel;

[0148] Determine the probability distribution corresponding to each grayscale pixel according to the frequency corresponding to each grayscale pixel;

[0149] The entropy of the target image is determined according to the probability distribution corresponding to each grayscale pixel.

[0150] Optionally, the division module 204 is specifically used for:

[0151] If the entropy of the target image is greater than or equal to a preset entropy threshold, the target layer is divided into a shallow layer;

[0152] If the entropy of the target image is less than the entropy threshold, the target layer is divided into a deep layer.

[0153] Optionally, the determining unit is specifically configured to:

[0154] Obtain the number of filters and the number of channels corresponding to the target layer;

[0155] A loss function is determined based on the number of filters, the number of channels, and the activation matrix.

[0156] Optionally, the updating unit is specifically used for:

[0157] Determining an update step size according to a preset learning rate, the gradient value, and the input image;

[0158] An updated image is determined according to the input image and the update step size.

[0159] The above-mentioned product can execute the network model hierarchical division method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0160] Embodiment 3

[0161] Figure 3 A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0162] like Figure 3As shown, the electronic device 30 includes at least one processor 31, and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 32 or the computer program loaded from the storage unit 38 to the random access memory (RAM) 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32, and the RAM 33 are connected to each other through a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0163] A number of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0164] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 31 performs the various methods and processes described above, such as the network model hierarchical division method:

[0165] Obtaining a target network model to be hierarchically divided, wherein the target network model includes at least one convolutional layer;

[0166] Input the input image into the target network model, perform inverse image generation, and obtain the target image corresponding to each convolution layer;

[0167] Calculating the entropy of each of the target images;

[0168] The target network model is hierarchically divided according to the entropy of each target image.

[0169] In some embodiments, the network model hierarchical division method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the network model hierarchical division method described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to execute the network model hierarchical division method in any other appropriate manner (e.g., by means of firmware).

[0170] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0171] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0172] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0173] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0174] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0175] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0176] In one embodiment, the embodiment of the present invention also includes a computer program product, the computer program product includes a computer program, and when the computer program is executed by a processor, it implements the network model hierarchy division method of any embodiment of the present invention.

[0177] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0178] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0179] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A network model hierarchical division method, characterized in that: include: Obtaining a target network model to be hierarchically divided, wherein the target network model includes at least one convolutional layer; Input the input image into the target network model, perform inverse image generation, and obtain the target image corresponding to each convolution layer; Calculating the entropy of each of the target images; The target network model is hierarchically divided according to the entropy of each target image.

2. The method according to claim 1, characterized in that Input the input image into the target network model, perform reverse image generation, and obtain the target image corresponding to each convolution layer, including: Inputting the input image into the target network model to obtain an activation matrix output by a target layer, wherein the target layer is any convolutional layer in the target network model; determining a loss function according to the activation matrix; Obtaining a gradient value of the loss function relative to the input image; Updating the input image by using the gradient value to obtain an updated image; Adding a target regularization term to the activation matrix to obtain an updated loss function; The updated image is used as a new input image, and the updated loss function is used as a new loss function. Return to execute the operation of inputting the input image into the target network model to obtain the activation matrix output by the target layer, until the number of iterations reaches a preset number, and the target image corresponding to the target layer is obtained.

3. The method according to claim 1, characterized in that: Calculating the entropy of each target image, including: For each of the target images, convert the target image into a grayscale image, and obtain the number of pixels corresponding to each grayscale pixel in the grayscale image; Determine the frequency corresponding to each grayscale pixel according to the number of pixels corresponding to each grayscale pixel; Determine the probability distribution corresponding to each grayscale pixel according to the frequency corresponding to each grayscale pixel; The entropy of the target image is determined according to the probability distribution corresponding to each grayscale pixel.

4. The method according to claim 1, characterized in that The target network model is hierarchically divided according to the entropy of each target image, including: If the entropy of the target image is greater than or equal to a preset entropy threshold, the target layer is divided into a shallow layer; If the entropy of the target image is less than the entropy threshold, the target layer is divided into a deep layer.

5. The method according to claim 2, characterized in that: Determining a loss function according to the activation matrix includes: Obtain the number of filters and the number of channels corresponding to the target layer; A loss function is determined based on the number of filters, the number of channels, and the activation matrix.

6. The method according to claim 2, characterized in that Updating the input image by the gradient value to obtain an updated image includes: Determining an update step size according to a preset learning rate, the gradient value, and the input image; An updated image is determined according to the input image and the update step size.

7. A network model hierarchical division device, characterized in that: include: An acquisition module, used to acquire a target network model to be hierarchically divided, wherein the target network model includes at least one convolutional layer; An input module, used to input the input image into the target network model, perform inverse image generation, and obtain a target image corresponding to each convolution layer; A calculation module, used for calculating the entropy of each target image; A division module is used to divide the target network model into levels according to the entropy of each target image.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the network model hierarchy division method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the network model hierarchical division method described in any one of claims 1-6 when executed.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the network model hierarchy division method according to any one of claims 1-6.

Images