Model lightweight method, device, electronic device and storage medium

Abstract

The present invention relates to the field of artificial intelligence technology, and discloses a model lightweight method, device, electronic device and storage medium, which performs dimensionality reduction processing on the convolution parameters of each convolution filter to obtain the initial feature vector of each convolution filter; according to the initial feature vector of each convolution filter, the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element are determined; according to the basic feature vector and the amplification weight corresponding to each basic vector element, the target amplification feature of each basic vector element is determined; the target amplification feature of each basic vector element is inserted into the basic feature vector to obtain the target feature vector; based on the target feature vector, a target model corresponding to the model to be lightweight is constructed. By determining the basic feature vector and inserting the amplification feature into the basic feature vector, while reducing the parameter amount of the target model, it is largely guaranteed that the performance of the model will not be reduced due to the lightweighting of the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model lightweighting method, device, electronic device and storage medium. Background Art

[0002] At present, traditional convolutional neural network models usually have many parameters and large computational complexity, making them difficult to process and deploy in real time on resource-constrained devices such as mobile terminals. Therefore, how to lightweight convolutional neural network models has become a key research topic.

[0003] In related technologies, model compression technology is usually used to remove redundant parameters in deep neural network models to reduce the size of the model. Or in the process of model application, the model input and operation method are adjusted to reduce the amount of model calculation.

[0004] However, the model lightweighting methods in related technologies are generally difficult to guarantee model performance. Therefore, there is an urgent need for a model lightweighting method that can guarantee model performance, which is of great significance to improving the adaptability of convolutional neural network models to resource-constrained environments. Summary of the invention

[0005] The present application provides a model lightweighting method, device, electronic device and storage medium to solve the defects of the model lightweighting methods in the related art that it is generally difficult to ensure the model performance.

[0006] The first aspect of the present application provides a model lightweight method, comprising:

[0007] Get the convolution parameters of each convolution filter in the model to be lightweight;

[0008] Performing dimensionality reduction processing on the convolution parameters of each of the convolution filters to obtain an initial feature vector of each of the convolution filters;

[0009] Determine, according to the initial feature vectors of each of the convolution filters, a basic feature vector of the model to be lightweight and an amplification weight corresponding to each basic vector element in the basic feature vector; wherein the length of the basic feature vector is less than the length of the initial feature vector;

[0010] Determining a target amplified feature of each basic vector element according to the basic feature vector and an amplification weight corresponding to each basic vector element in the basic feature vector;

[0011] Inserting the target augmented feature of each element of the basic vector into the basic feature vector to obtain a target feature vector;

[0012] Based on the target feature vector, construct a target model corresponding to the model to be lightweight;

[0013] Obtain an image to be recognized;

[0014] Based on the target model, the image to be recognized is recognized to obtain an image recognition result of the image to be recognized.

[0015] In an optional implementation, the performing dimensionality reduction processing on the convolution parameters of each of the convolution filters to obtain the initial feature vector of each of the convolution filters includes:

[0016] For any of the convolution filters, position encoding is performed on the convolution parameters of the convolution filter to obtain a target position encoding of the convolution filter;

[0017] The convolution parameters of the convolution filter and the target position code are concatenated to obtain a target convolution parameter vector of the convolution filter;

[0018] Performing dimensionality reduction processing on the target convolution parameter vector of each of the convolution filters to obtain an initial feature vector of each of the convolution filters.

[0019] In an optional implementation, for any of the convolution filters, position encoding is performed on the convolution parameters of the convolution filter to obtain the target position encoding of the convolution filter, including:

[0020] For any of the convolution filters, obtaining position information of the convolution filter during the convolution calculation process of the model to be lightweighted;

[0021] According to the position information, the convolution parameters of the convolution filter are position-encoded to obtain the target position code of the convolution filter.

[0022] In an optional implementation, the performing dimensionality reduction processing on the target convolution parameter vector of each of the convolution filters to obtain the initial feature vector of each of the convolution filters includes:

[0023] Converting the target convolution parameter vector of each convolution filter into a one-dimensional vector to obtain the target one-dimensional vector of each convolution filter;

[0024] Calculating the similarity between the target one-dimensional vectors of each of the convolution filters to obtain a similarity calculation result;

[0025] According to the similarity calculation result, a similarity matrix is ​​constructed; wherein the similarity matrix is ​​used to uniformly represent the similarity between the target one-dimensional vectors of each convolution filter;

[0026] According to the similarity matrix, converting the target one-dimensional vector of each convolution filter into an initial feature vector;

[0027] The length of the initial feature vector is smaller than the length of the target one-dimensional vector.

[0028] In an optional implementation, converting the target one-dimensional vector of each convolution filter into an initial feature vector according to the similarity matrix includes:

[0029] Performing eigenvalue decomposition on the similarity matrix to obtain an eigenvalue decomposition result of the similarity matrix;

[0030] According to the eigenvalue decomposition result of the similarity matrix, an initial eigenvector corresponding to the target one-dimensional vector of each convolution filter is determined.

[0031] In an optional implementation, determining the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter includes:

[0032] Performing element clustering processing on the initial feature vectors of each convolution filter to obtain a plurality of element cluster centers and a sample size of each element category; wherein the number of the element cluster centers is determined according to a preset amplification feature ratio and the number of channels of the convolution filter;

[0033] According to the multiple element clustering centers and the sample size of each element category, the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector are determined.

[0034] In an optional implementation, determining the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the multiple element cluster centers and the sample size of each element category includes:

[0035] Taking the multiple element cluster centers as basic vector elements, constructing a basic feature vector of the model to be lightweight;

[0036] According to the sample size of each element category, the amplification weight corresponding to each basic vector element in the basic feature vector is determined.

[0037] In an optional implementation, determining the target augmented feature of each basic vector element according to the basic feature vector and the augmented weight corresponding to each basic vector element in the basic feature vector includes:

[0038] Generate a basic feature map according to the basic feature vector;

[0039] Determining the number of amplicon features corresponding to each basic vector element in the basic feature according to the amplification weight corresponding to each basic vector element in the basic feature vector;

[0040] For any of the basic vector elements, mapping the basic vector element as a traversal center to the basic feature map;

[0041] According to the preset offset and the number of amplified sub-features of the basic vector element, a key feature traversal is performed on the traversal center of the basic feature graph to obtain the target amplified feature of the basic vector element.

[0042] In an optional implementation, determining the number of amplicon features corresponding to each basic vector element in the basic feature according to the amplification weight corresponding to each basic vector element in the basic feature vector includes:

[0043] The number of amplicon features corresponding to each basic vector element in the basic feature is determined based on the following formula:

[0044]

[0045] in, represents the first The number of amplicon features per basis vector element, represents the first The augmentation weights of the basis vector elements, Indicates the preset amplification feature ratio, Indicates the number of channels of the convolution filter.

[0046] In an optional implementation, the step of performing a key feature traversal on the traversal center of the basic feature graph according to a preset offset and the number of amplicon features of the basic vector element to obtain a target amplified feature of the basic vector element includes:

[0047] Determining the target traversal round number corresponding to the basic vector element according to the number of amplicon features of the basic vector element; wherein the number of amplicon features is equal to the target traversal round number;

[0048] In the case of performing a key feature first traversal on the traversal center of the basic feature graph, performing a key feature traversal on the traversal center of the basic feature graph according to the preset offset to obtain an amplicon feature of the first traversal;

[0049] In the case where the key feature is not traversed for the first time on the traversal center of the basic feature graph, iteratively adjusting the preset offset according to the current traversal round number to obtain the current round target offset until the current traversal round reaches the target traversal round number;

[0050] According to the target offset of this round, key feature traversal is performed on the traversal center of the basic feature graph to obtain amplicon features that are not traversed for the first time;

[0051] The target amplification feature of the basic vector element is determined according to the amplicon feature of the first traversal and all the amplicon features of the non-first traversal.

[0052] In an optional implementation, inserting the target augmented feature of each of the basic vector elements into the basic feature vector to obtain the target feature vector includes:

[0053] Determining an amplified feature standard deviation and an amplified feature mean of the basic feature vector according to a target amplified feature of each of the basic vector elements;

[0054] For a target amplification feature of any of the basic vector elements, determining a target distribution probability value of the target amplification feature according to the target amplification feature, the amplification feature standard deviation and the amplification feature mean;

[0055] Determining a target insertion position of each of the target amplification features according to a target distribution probability value of each of the target amplification features;

[0056] According to the target insertion position of each target amplified feature, each target amplified feature is inserted into the basic feature vector to obtain a target feature vector.

[0057] In an optional implementation, the target amplification feature for any of the basic vector elements, determining the target distribution probability value of the target amplification feature according to the target amplification feature, the amplification feature standard deviation and the amplification feature mean, comprises:

[0058] For a target amplified feature of any of the basic vector elements, determining an initial distribution probability value of the target amplified feature according to the target amplified feature, the amplified feature standard deviation and the amplified feature mean;

[0059] The initial distribution probability value of each target amplification feature is normalized to obtain the target distribution probability value of each target amplification feature.

[0060] In an optional embodiment, determining the initial distribution probability value of the target amplification feature according to the target amplification feature, the amplification feature standard deviation and the amplification feature mean includes:

[0061] Based on the following formula, the initial distribution probability value of the target amplification feature is determined:

[0062]

[0063] in, represents the first The initial distribution probability value of the target augmented feature corresponding to the basis vector elements, represents the first The target augmented features corresponding to the basis vector elements, represents the standard deviation of the amplification characteristics, represents the mean of the amplification characteristics.

[0064] In an optional embodiment, determining the target insertion position of each target amplification feature according to the target distribution probability value of each target amplification feature includes:

[0065] Determining a cumulative probability distribution of target amplification features according to the target distribution probability value of each of the target amplification features;

[0066] The target insertion position of each of the target amplification features is determined according to the cumulative probability distribution of the target amplification features.

[0067] In an optional embodiment, determining the cumulative probability distribution of the target amplification feature according to the target distribution probability value of each target amplification feature includes:

[0068] The cumulative probability distribution of the target amplification feature is determined based on the following formula:

[0069]

[0070]

[0071] in, represents the cumulative probability distribution, represents the cumulative probability distribution The cumulative probability, represents the total number of target amplification features, Indicates The target distribution probability value of the target amplification feature.

[0072] In an optional embodiment, determining the target insertion position of each target amplification feature according to the cumulative probability distribution of the target amplification feature includes:

[0073] Generate a cumulative probability threshold for each target amplification feature according to a preset threshold generation method;

[0074] For any of the target amplification features, according to the cumulative probability threshold of the target amplification feature, traverse the target cumulative probability in the cumulative probability distribution;

[0075] Determining a target insertion position of the target amplification feature according to an index value of the target cumulative probability in the cumulative probability distribution;

[0076] The target cumulative probability is the minimum cumulative probability in the cumulative probability distribution that is not less than the cumulative probability threshold.

[0077] A second aspect of the present application provides a model lightweight device, comprising:

[0078] A first acquisition module is used to obtain the convolution parameters of each convolution filter in the model to be lightweight;

[0079] A dimensionality reduction module, used for performing dimensionality reduction processing on the convolution parameters of each convolution filter to obtain an initial feature vector of each convolution filter;

[0080] A first determination module is used to determine the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter; wherein the length of the basic feature vector is less than the length of the initial feature vector;

[0081] A second determination module, configured to determine a target amplified feature of each basic vector element according to the basic feature vector and an amplification weight corresponding to each basic vector element in the basic feature vector;

[0082] An insertion module, used for inserting the target augmented feature of each element of the basic vector into the basic feature vector to obtain a target feature vector;

[0083] A lightweight module, used for constructing a target model corresponding to the model to be lightweight based on the target feature vector;

[0084] A second acquisition module is used to acquire an image to be identified;

[0085] The recognition module is used to recognize the image to be recognized based on the target model to obtain an image recognition result of the image to be recognized.

[0086] A third aspect of the present application provides an electronic device, comprising: at least one processor and a memory;

[0087] The memory stores computer-executable instructions;

[0088] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the model lightweighting method described in the first aspect and various possible designs of the first aspect.

[0089] The fourth aspect of the present application provides a non-volatile computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the model lightweighting method or as described in the first aspect and various possible designs of the first aspect is implemented.

[0090] A fifth aspect of the present application provides a computer program product, including computer instructions, which are used to enable a computer to execute the method described in the first aspect and various possible designs of the first aspect.

[0091] The technical solution of this application has the following advantages:

[0092] The present application provides a model lightweight method, device, electronic device and storage medium, the method comprising: obtaining the convolution parameters of each convolution filter in the model to be lightweight; performing dimensionality reduction processing on the convolution parameters of each convolution filter to obtain the initial feature vector of each convolution filter; determining the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter; wherein the length of the basic feature vector is less than the length of the initial feature vector; determining the target amplification feature of each basic vector element according to the basic feature vector and the amplification weight corresponding to each basic vector element in the basic feature vector; inserting the target amplification feature of each basic vector element into the basic feature vector to obtain a target feature vector; based on the target feature vector, constructing a target model corresponding to the model to be lightweight and obtaining an image to be recognized; based on the target model, recognizing the image to be recognized to obtain an image recognition result of the image to be recognized. The method provided by the above scheme determines the basic feature vector with significant features and small data volume in the lightweight model, and inserts the amplified features into the basic feature vector. While reducing the number of parameters of the target model, it ensures to a large extent that the performance of the model will not be reduced due to the lightweight model, thereby improving the accuracy of the image recognition results of the image to be recognized. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technical descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0094] Figure 1 This is a schematic diagram of the structure of the model lightweight system on which the embodiments of the present application are based;

[0095] Figure 2 A schematic diagram of a flow chart of a model lightweighting method provided in an embodiment of the present application;

[0096] Figure 3 A schematic diagram of a flow chart of an exemplary model lightweighting method provided in an embodiment of the present application;

[0097] Figure 4 A schematic diagram of the structure of a model lightweight device provided in an embodiment of the present application;

[0098] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0099] The above drawings have shown clear embodiments of the present application, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0100] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0101] In addition, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the following embodiments, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0102] In related technologies, in today's technological development, the explosive growth of data volume and the limited computing resources have become prominent problems in many application scenarios, which has spawned and promoted the development of lightweight models. With the widespread use of artificial intelligence technology, especially in mobile devices, IoT devices and edge computing scenarios, higher standards have been set for the computing efficiency and storage requirements of models. Traditional deep neural network models usually have many parameters and large computational complexity, making them difficult to process and deploy in real time on resource-constrained devices. For example, when performing image recognition or speech processing on a smartphone, if a large and complex model is used, the device may consume too much energy and respond slowly, affecting the user experience.

[0103] In IoT devices, given the limited hardware resources, lightweight models are more urgently needed to implement intelligent functions. In addition, applications with high real-time requirements such as autonomous driving and industrial automation also require models to give results quickly. Lightweight models can shorten the calculation time and meet the needs of real-time decision-making.

[0104] To meet these challenges, researchers have been working hard to develop various technologies to achieve lightweight models, including model compression, pruning, quantization, knowledge distillation and other methods. These technologies aim to reduce the number of model parameters and reduce computational complexity. For example, model compression technology can reduce the size of the model by removing redundant parameters; pruning technology can delete connections or neurons that have little impact on model performance; quantization technology can convert the parameters of the model from high-precision numerical representation to low-precision, thereby reducing storage space and computational complexity. In short, the development of lightweight models is to adapt to various resource-constrained scenarios, achieve efficient intelligent computing, and promote the application of artificial intelligence technology in a wider range of fields.

[0105] However, the model lightweighting methods in related technologies are generally difficult to guarantee model performance. Therefore, there is an urgent need for a model lightweighting method that can guarantee model performance, which is of great significance to improving the adaptability of convolutional neural network models to resource-constrained environments.

[0106] In response to the above problems, an embodiment of the present application provides a model lightweight method, device, electronic device and storage medium, the method comprising: obtaining convolution parameters of each convolution filter in the model to be lightweight; performing dimensionality reduction processing on the convolution parameters of each convolution filter to obtain an initial feature vector of each convolution filter; determining a basic feature vector of the model to be lightweight and an amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter; wherein the length of the basic feature vector is less than the length of the initial feature vector; determining a target amplification feature of each basic vector element according to the basic feature vector and the amplification weight corresponding to each basic vector element in the basic feature vector; inserting the target amplification feature of each basic vector element into the basic feature vector to obtain a target feature vector; based on the target feature vector, constructing a target model corresponding to the model to be lightweight and obtaining an image to be identified; based on the target model, identifying the image to be identified to obtain an image recognition result of the image to be identified. The method provided by the above scheme determines the basic feature vector with significant features and small data volume in the lightweight model, and inserts the amplified features into the basic feature vector. While reducing the number of parameters of the target model, it ensures to a large extent that the performance of the model will not be reduced due to the lightweight model, thereby improving the accuracy of the image recognition results of the image to be recognized.

[0107] The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0108] First, the structure of the model lightweight system on which this application is based is described:

[0109] The model lightweighting method, device, electronic device and storage medium provided in the embodiments of the present application are suitable for lightweighting a convolutional neural network model to be deployed on a resource-constrained device to reduce the number of model parameters, that is, to reduce the resource occupation of the model on the resource-constrained device. Figure 1 As shown, it is a structural schematic diagram of the model lightweight system based on the embodiment of the present application, which mainly includes a model lightweight device and a resource-constrained device. The model lightweight device is used to take the convolutional neural network model to be deployed to the resource-constrained device as the model to be lightweight, and perform model lightweight processing on the lightweight model to obtain the target model, and finally send the target model to the resource-constrained device so that the resource-constrained device can perform image recognition based on the target model.

[0110] The embodiment of the present application provides a model lightweight method for lightweighting a convolutional neural network model to reduce the number of model parameters. The execution subject of the embodiment of the present application is a model lightweight device, such as a server, a desktop computer, a laptop computer, a tablet computer, and other electronic devices that can be used for model lightweighting.

[0111] like Figure 2 FIG. 1 is a flow chart of a model lightweighting method provided in an embodiment of the present application, and the method includes:

[0112] Step 201, obtaining the convolution parameters of each convolution filter in the model to be lightweight.

[0113] Among them, the model to be lightweight is a trained convolutional neural network model. In the lightweight model structure, the convolution parameters of each convolution filter are obtained by traversing the convolution filters in the convolution layer. The convolution parameters include information such as the weight value of the convolution filter.

[0114] Step 202: Perform dimensionality reduction processing on the convolution parameters of each convolution filter to obtain an initial feature vector of each convolution filter.

[0115] Specifically, the convolution parameters of each convolution filter are firstly vectorized to obtain a convolution parameter vector, and then a series of dimensionality reduction processes are performed on the convolution parameter vector to obtain an initial feature vector of each convolution filter.

[0116] Step 203: Determine the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter.

[0117] Among them, the length of the basic feature vector is smaller than that of the initial feature vector. The basic feature vector has significant features and a small amount of data, that is, the basic feature vector can represent the core features of the model to be lightweight.

[0118] Specifically, the initial feature elements in the initial feature vectors of each convolution filter may be clustered, and based on the clustering results, the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector may be determined.

[0119] Step 204: Determine the target augmented feature of each basic vector element according to the basic feature vector and the augmented weight corresponding to each basic vector element in the basic feature vector.

[0120] Specifically, the target amplification feature of each basic vector element may be determined according to the amplification weight corresponding to each basic vector element and in combination with the model amplification rule.

[0121] Step 205: insert the target augmented feature of each basic vector element into the basic feature vector to obtain the target feature vector.

[0122] It should be noted that by inserting the target amplified features of each basic vector element into the basic feature vector, the number of parameters can be reduced in the process of model lightweighting, and the model performance can be maintained through reasonable amplification.

[0123] Specifically, the target augmented features of each basic vector element may be inserted into the basic feature vector according to a preset fusion rule, so as to obtain the target feature vector through feature fusion.

[0124] Step 206: construct a target model corresponding to the model to be lightweight based on the target feature vector.

[0125] Among them, the target model is reconstructed using the optimized and integrated target feature vector, that is, the original convolution parameters are replaced by the target feature vector, so that the new target model can be lightweight while maintaining a certain performance, so that the target model can be deployed and applied on resource-constrained devices.

[0126] Step 207: Obtain the image to be recognized.

[0127] The image to be recognized may be a face image to be subjected to face recognition, etc.

[0128] Step 208: Based on the target model, the image to be recognized is recognized to obtain an image recognition result of the image to be recognized.

[0129] Among them, taking the image to be recognized as a face image as an example, in face recognition, the backbone network can use various convolutional neural networks to complete the feature extraction work, in which the face recognition model can extract a large number of face feature vectors through the convolutional network, and then complete the face recognition process through the upsampling module. Therefore, whether the algorithm network can generate different features for different faces and generate similar features for the same face will be the focus of the face recognition task, that is, how to maximize the distance between classes and minimize the distance within classes. While lightweighting the model, the embodiment of the present application clusters the basic features that can better reflect the face features and assigns corresponding amplification weights, thereby improving the ability to extract subtle features of the face, so as to improve the accuracy of the image recognition results of the image to be recognized by resource-constrained devices.

[0130] On the basis of the above embodiments, in order to ensure the accuracy of the dimensionality reduction processing result of the convolution parameters of the convolution filter, so as to further reduce the influence of the convolution parameter dimensionality reduction process on the model performance, as an implementable method, in one embodiment, the convolution parameters of each convolution filter are subjected to dimensionality reduction processing to obtain the initial feature vector of each convolution filter, including:

[0131] Step 2021, for any convolution filter, position encoding is performed on the convolution parameters of the convolution filter to obtain a target position encoding of the convolution filter;

[0132] Step 2022, concatenating the convolution parameters of the convolution filter and the target position code to obtain a target convolution parameter vector of the convolution filter;

[0133] Step 2023, performing dimensionality reduction processing on the target convolution parameter vector of each convolution filter to obtain the initial feature vector of each convolution filter.

[0134] Specifically, the convolution filter in the lightweight model is first vectorized to obtain the convolution parameter vector, that is, the expression of a convolution filter is expressed by Convert to , get the array ,in Indicates the number of channels of the input convolution, represents the number of channels of the output convolution, represents the convolution filter size, Represents the convolution parameter vector transformed by the corresponding convolution. The length of this array is , It is the abbreviation of Input, which means input. The abbreviation of Output represents output. The purpose of arrayization is to conveniently measure the value of the filter, classify the properties of the convolution, and then cluster the convolutions with the same properties to facilitate the selection of features for amplification. That is, the convolution parameters of all convolution filters in the model to be lightweight are integrated into a matrix, and each row represents the convolution parameters of a convolution filter.

[0135] Among them, considering directly vectorizing the convolution parameters, that is, flattening the convolution parameters from a 3-dimensional array to a 1-dimensional array (dimensionality reduction), the length of the processed data is The disadvantage of directly using the data for clustering is that after the data is converted into one-dimensional data, the spatiality between the convolution parameters is lost, and the clustering effect is reduced.

[0136] Specifically, in order to solve the technical problem, the embodiment of the present application performs position encoding on the convolution parameters of the convolution filter before performing dimensionality reduction processing to obtain the target position encoding of the convolution filter; the convolution parameters and the target position encoding of the convolution filter are spliced ​​to obtain the target convolution parameter vector of the convolution filter; finally, the target convolution parameter vector of each convolution filter is subjected to dimensionality reduction processing to obtain the initial feature vector of each convolution filter.

[0137] Specifically, in one embodiment, for any convolution filter, the position information of the convolution filter during the convolution calculation process of the model to be lightweight can be obtained; based on the position information, the convolution parameters of the convolution filter are position-encoded to obtain the target position encoding of the convolution filter.

[0138] The position information may refer to the position information of the convolution filter in the convolution layer of the model to be lightweight. In a convolutional neural network, the convolution layer contains multiple convolution filters, which are arranged in a two-dimensional array in space. The position information may specifically refer to the row position and column position of the convolution filter in this array.

[0139] Specifically, for any convolution filter, the position encoding can be determined by using methods such as sine / cosine function encoding or two-dimensional coordinate encoding. For ease of use, two-dimensional coordinate encoding can be used. To improve accuracy, sine and other methods can be used with corresponding modifications to obtain higher accuracy. Among them, it is assumed that the dimension of the position encoding is , for the convolutional filters, whose positions are encoded as a d-dimensional vector, denoted as .

[0140] Furthermore, The position encoding of the convolution filters and the convolution parameters (convolution parameter vector) are concatenated. Assume that the dimension of the convolution parameters is , the convolution is , the dimension of the position encoding is , the concatenated vector (target convolution parameter vector) is recorded as , then the dimension after splicing is The concatenation operation can use a simple connection operation, that is, connecting the position encoding vector and the convolution parameter vector in sequence.

[0141] In an embodiment of the present application, position encoding provides an additional source of information, which can help the model more accurately capture position-related features in the convolution parameters. At the same time, in subsequent processing, the position features are also used as auxiliary information during feature clustering, which is beneficial to enhance the clustering effect.

[0142] Specifically, in one embodiment, the target convolution parameter vector of each convolution filter is converted into a one-dimensional vector to obtain the target one-dimensional vector of each convolution filter; the similarity between the target one-dimensional vectors of each convolution filter is calculated to obtain a similarity calculation result; based on the similarity calculation result, a similarity matrix is ​​constructed; wherein the similarity matrix is ​​used to uniformly represent the similarity between the target one-dimensional vectors of each convolution filter; according to the similarity matrix, the target one-dimensional vector of each convolution filter is converted into an initial feature vector; wherein the length of the initial feature vector is less than the length of the target one-dimensional vector.

[0143] Specifically, the concatenated vector Flatten it into a one-dimensional vector to obtain the target one-dimensional vector of each convolution filter. The flattening operation can convert the multi-dimensional concatenated vector into a one-dimensional vector, making it a vector with a length of A one-dimensional vector of .

[0144] Among them, the target convolution parameter vectors of each convolution filter obtained through the above steps are integrated into a combination formula: ,in, represents the concatenation and flattening operation. This formula takes the positional encoding and convolution parameters as input and outputs a length of A one-dimensional vector of . Among them, For one The matrix of Represents the number of convolution channels, and each row represents the concatenation and flattening result of a convolution.

[0145] Furthermore, the cosine similarity is used to measure the similarity of the target one-dimensional vector of each pair of convolution filters. The similarity measures the cosine value of the angle between each pair of convolutions, and the value range is set to [-1, 1]. The closer the cosine similarity is to 1, the more similar the two vectors are, and the closer it is to 1, the less similar they are. Constructing a similarity matrix of the same structure ,(Right now for The matrix of and The shape of the similarity matrix Gaussian distribution is used for initialization, and the similarity matrix is ​​subsequently calculated based on the results of cosine similarity calculation. Assign values, where Indicates that in the current lightweight model, The target one-dimensional vector of the convolution filter and the The similarity between the target one-dimensional vectors of the convolution filters.

[0146] Specifically, in one embodiment, the similarity matrix may be subjected to eigenvalue decomposition to obtain an eigenvalue decomposition result of the similarity matrix; and based on the eigenvalue decomposition result of the similarity matrix, an initial eigenvector corresponding to the target one-dimensional vector of each convolution filter is determined.

[0147] Specifically, the obtained similarity matrix is ​​first normalized. The purpose of normalizing the similarity matrix is ​​to limit the similarity value to a fixed range and avoid excessive data from interfering with the overall calculation process. Normalization, which maps the value to a probability distribution. For the similarity matrix, the following formula can be used Normalization: , in, is an element in the similarity matrix (similarity) is an exponential function, Represents the sum of all elements. Through this formula, the value of the similarity matrix can be normalized to a probability distribution so that the sum of each element is 1.

[0148] Furthermore, the similarity matrix is ​​subjected to eigenvalue decomposition. Specifically, a method based on decomposition technology (such as QR decomposition and singular value decomposition) can be used to calculate the eigenvalues ​​and eigenvectors of the similarity matrix, and the eigenvector can be obtained. feature vectors, and obtain the initial feature vector .

[0149] Among them, the eigenvector represents the main change pattern of the data (similarity) in a certain direction, and the eigenvalue represents the intensity of change in this direction. The eigenvector corresponding to the larger eigenvalue represents the more important change direction in the data, that is, the data has a higher variance in these directions. By performing eigenvalue decomposition on the similarity matrix through methods such as QR decomposition or singular value decomposition, and selecting the appropriate eigenvector according to the size of the eigenvalue, the convolution parameter array (target one-dimensional vector) can be effectively reduced in dimension, achieving the purpose of simplifying data representation and highlighting the main features, providing a more efficient and meaningful data structure for subsequent model processing.

[0150] On the basis of the above embodiments, in order to further ensure the accuracy of the dimensionality reduction processing results of the convolution parameters of the convolution filter, so as to further reduce the impact of the convolution parameter dimensionality reduction process on the model performance, as an implementable method, in one embodiment, according to the initial eigenvectors of each convolution filter, the basic eigenvector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic eigenvector are determined, including:

[0151] Step 2031, performing element clustering processing on the initial feature vectors of each convolution filter to obtain multiple element clustering centers and the sample size of each element category; wherein the number of element clustering centers is determined according to a preset amplification feature ratio and the number of channels of the convolution filter;

[0152] Step 2032, determining the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the multiple element clustering centers and the sample size of each element category.

[0153] It should be noted that the disadvantage of directly using the initial eigenvector for clustering is that it is difficult to cluster data of this length. Therefore, the embodiment of the present application constructs a similarity matrix, and by performing eigenvalue decomposition on the similarity matrix, the data (the initial eigenvector of each convolution filter) can be reduced to a special vector (basic eigenvector) representation.

[0154] Specifically, taking the face recognition model as an example, since different basic features in the face feature should have different amplification weights, in the clustering process, in addition to using the clustering results obtained by clustering as the basic features, it is also necessary to obtain the amplification weight of each basic feature (basic vector element). Perform element clustering analysis on the obtained initial feature vector, and group these convolution parameters (initial feature vectors) into Y groups. groups, get Represents the basic group, the final basic feature vector It can simply represent the entire convolutional layer.

[0155] Using the optimization method of dimensionality-reducible clustering, the optimized loss function can be described as:

[0156]

[0157] in Indicates convolution parameter groups The mean vector represents the mean of this group, also known as the centroid. The optimization steps are as follows:

[0158] Step 1. Select Initialization samples as the initial cluster centers. equal , that is, the number of element cluster centers , Indicates the preset amplification feature ratio, Indicates the number of channels of the convolution filter.

[0159] Step 2. Calculate it for each sample in the data set The distance between the cluster centers is calculated and the samples are divided into the class corresponding to the cluster center with the smallest distance. Here, a similarity measurement method is selected, such as Euclidean distance, cosine similarity, etc., to measure the distance between the sample and the cluster center.

[0160] Step 3. Recalculate the cluster center for each group .

[0161] Step 4. Repeat steps 2 and 3 above until one of the following termination conditions is reached and the above formula is satisfied The difference is less than 0.1 in 5 iterations, or the number of iterations reaches 50 (number of iterations, minimum error change, etc.).

[0162] Specifically, the above optimization steps are used to perform element clustering on the initial feature vectors of each convolution filter, and the final result is The cluster center is the element cluster center, and the sample size of each element category is the data size of the element category to which each cluster center belongs.

[0163] Specifically, in one embodiment, multiple element cluster centers may be used as basic vector elements to construct a basic feature vector of the model to be lightweight; and the amplification weight corresponding to each basic vector element in the basic feature vector is determined according to the sample size of each element category.

[0164] Specifically, by counting the number of samples divided into each element category , converting the number of categories of each element into a weight. , the weight is proportional to the number of samples, where Indicates The weight of the category, The weights of the basic vector elements can be calculated using different methods. In the embodiment of the present application, the number of categories is divided by the total number of samples, or it is normalized to a range between 0 and 1. The corresponding calculation formula is .

[0165] Among them, by clustering the initial feature vectors and calculating the weight of each cluster, the foundation is laid for the subsequent weighted feature fusion according to the weight value of each feature, so as to achieve the differentiation of different basic features in the amplification process, which has the effect of distinguishing the basic features to a certain extent, and is conducive to ensuring the comprehensive performance of the target model.

[0166] On the basis of the above embodiments, in order to further ensure the accuracy of the dimensionality reduction processing result of the convolution parameter of the convolution filter, so as to further reduce the influence of the convolution parameter dimensionality reduction process on the model performance, as an implementable method, in one embodiment, according to the basic feature vector and the amplification weight corresponding to each basic vector element in the basic feature vector, the target amplification feature of each basic vector element is determined, including:

[0167] Step 2041, generating a basic feature map according to the basic feature vector;

[0168] Step 2042, determining the number of amplicon features corresponding to each basic vector element in the basic feature according to the amplification weight corresponding to each basic vector element in the basic feature vector;

[0169] Step 2043, for any basic vector element, map the basic vector element as the traversal center to the basic feature map;

[0170] Step 2044, according to the preset offset and the number of amplicon features of the basic vector element, perform key feature traversal on the traversal center of the basic feature graph to obtain the target amplified feature of the basic vector element.

[0171] Among them, the basic feature map is another form of expression of the basic feature vector, so as to facilitate the subsequent key feature traversal. The embodiment of the present application can reduce the original network structure framework by amplifying the basic feature vector after clustering. It reduces the number of network parameters by 2 times, while maintaining effective feature amplification effects and improving the target model’s ability to extract subtle features of the face.

[0172] Specifically, in the embodiment of the present application, the ratio of the amplification features (the preset amplification feature ratio) is , that is, when the number of channels is In the case of, for the convenience of representation, the number of basic vector elements included in the basic eigenvector is , the basic eigenvector can be expressed as ,So is the number of expanded features.

[0173] Specifically, in the process of generating augmented features, it is necessary to ensure that the number of augmented features is an integer, that is, the augmented features of each basic vector element are integers and there are no surpluses. You can round up or down as needed to make the initial values ​​all integers. When making adjustments, is the total number of augmented features, if The initial value can be Reduce one by one until the total is equal to , or increase the initial value one by one until the total is equal to .

[0174] The calculation formula for the number of amplicon features corresponding to each basic vector element is as follows:

[0175]

[0176] in, represents the first The number of amplicon features per basis vector element, represents the first The augmentation weights of the basis vector elements, Indicates the preset amplification feature ratio, Indicates the number of channels of the convolution filter.

[0177] Specifically, in one embodiment, the target number of traversal rounds corresponding to the basic vector element can be determined according to the number of amplicon features of the basic vector element; wherein the number of amplicon features is equal to the target number of traversal rounds; in the case of performing the first traversal of the key features on the traversal center of the basic feature map, the key feature traversal is performed on the traversal center of the basic feature map according to a preset offset to obtain the amplicon features of the first traversal; in the case of performing the non-first traversal of the key features on the traversal center of the basic feature map, the preset offset is iteratively adjusted according to the current number of traversal rounds to obtain the target offset of this round until the current traversal round reaches the target number of traversal rounds; the key feature traversal is performed on the traversal center of the basic feature map according to the target offset of this round to obtain the amplicon features of the non-first traversal; the target amplification features of the basic vector element are determined according to the amplicon features of the first traversal and all the amplicon features of the non-first traversal.

[0178] Specifically, the amplification weight corresponding to the basic feature is ,in , that is, the sum of the amplification weights is 1, and the number of channels available for amplification of the basic feature vector is , the number of times each feature (element of the basis vector) should be expanded is The amplification feature obtained is G= , where the number of amplicon features per basis vector element is Among them, the offset during the traversal process can be expressed as , then the target amplification feature is obtained can be described as:

[0179]

[0180] in, Represents the basic vector elements After a The features (amplicon features) after the step shift (key feature traversal). The specific translation operation can be shift translation. Equal to the target traversal round number. During the traversal process, the offset increases with the traversal round number. When the key feature is first traversed at the traversal center of the basic feature graph, the preset offset is used. , and so on, until the current traversal round number Among them, the target amplification feature Through The amplicon features were fused.

[0181] Based on the above embodiment, it is assumed that the currently obtained basic feature vector can be expressed as , So The fill position of the augmented feature is usually inserted into However, this method is prone to overfitting. In subsequent processing, the model tends to ignore the amplified features, which is not conducive to ensuring the accuracy of the model. To solve this technical problem, as an implementable method, in one embodiment, the target amplified features of each basic vector element are inserted into the basic feature vector to obtain the target feature vector, including:

[0182] Step 2051, determining the amplified feature standard deviation and the amplified feature mean of the basic feature vector according to the target amplified feature of each basic vector element;

[0183] Step 2052, for a target augmented feature of any basic vector element, determine a target distribution probability value of the target augmented feature according to the target augmented feature, the augmented feature standard deviation, and the augmented feature mean;

[0184] Step 2053, determining the target insertion position of each target amplification feature according to the target distribution probability value of each target amplification feature;

[0185] Step 2054 , inserting each target augmented feature into the basic feature vector according to the target insertion position of each target augmented feature to obtain a target feature vector.

[0186] Among them, the standard deviation of the amplified feature reflects the degree of dispersion of the amplified feature, and the mean of the amplified feature reflects the average level. Through these two statistics, a probability distribution framework of the amplified feature can be constructed to arrange the insertion position of the amplified feature and avoid problems such as overfitting caused by simple insertion.

[0187] Specifically, in one embodiment, for the target amplification feature of any basic vector element, the initial distribution probability value of the target amplification feature can be determined based on the target amplification feature, the amplification feature standard deviation and the amplification feature mean; the initial distribution probability values ​​of each target amplification feature are normalized to obtain the target distribution probability value of each target amplification feature.

[0188] The initial distribution probability value of the target amplification feature may specifically adopt a Gaussian distribution probability.

[0189] Specifically, in one embodiment, the initial distribution probability value of the target amplification feature may be determined based on the following formula:

[0190]

[0191] in, represents the first The initial distribution probability value of the target augmented feature corresponding to the basis vector elements, represents the first The target augmented features corresponding to the basis vector elements, represents the standard deviation of the amplification characteristics, represents the mean of the amplification features.

[0192] Furthermore, the initial distribution probability value of each target amplification feature can be normalized based on the following formula to obtain the target distribution probability value of each target amplification feature:

[0193]

[0194] in, Indicates The target distribution probability value of the target amplification feature, Indicates The initial distribution probability value of the target amplification feature, represents the sum of the initial distribution probability values ​​of all target amplification features, It actually means normalized, and the sum of the target distribution probability values ​​of all target amplified features is equal to 1, so as to accurately represent the relative probability of each target amplified feature in the whole.

[0195] Specifically, in one embodiment, the cumulative probability distribution of the target amplification features may be determined according to the target distribution probability value of each target amplification feature; and the target insertion position of each target amplification feature may be determined according to the cumulative probability distribution of the target amplification features.

[0196] Among them, the cumulative probability distribution represents the overall probability accumulation as the target amplification features are accumulated one by one.

[0197] Specifically, in one embodiment, the cumulative probability distribution of the target amplification feature may be determined based on the following formula:

[0198]

[0199]

[0200] in, represents the cumulative probability distribution, represents the cumulative probability distribution The cumulative probability, represents the total number of target amplification features, Indicates The target distribution probability value of the target amplification feature. , , , and so on.

[0201] Specifically, in one embodiment, a cumulative probability threshold for each target amplification feature can be generated according to a preset threshold generation method; for any target amplification feature, the target cumulative probability is traversed in the cumulative probability distribution according to the cumulative probability threshold of the target amplification feature; and the target insertion position of the target amplification feature is determined according to the index value of the target cumulative probability in the cumulative probability distribution.

[0202] The target cumulative probability is the minimum cumulative probability in the cumulative probability distribution that is not less than the cumulative probability threshold.

[0203] Specifically, the cumulative probability threshold of each target amplification feature can be generated according to the preset threshold generation method, and the cumulative probability threshold A randomly generated number between 0 and 1, for example, can be evenly divided into intervals to generate a cumulative probability threshold. Then, for any target augmentation feature, according to its cumulative probability threshold, traverse the cumulative probability distribution to find the target cumulative probability. The target cumulative probability is the minimum cumulative probability in the cumulative probability distribution that is not less than the cumulative probability threshold. Finally, according to the index value of this target cumulative probability in the cumulative probability distribution, determine the target insertion position of the target augmentation feature. Determine the index value The calculation process is: , Indicates The cumulative probability threshold of target amplification features, that is, Start checking the cumulative probability , find satisfaction the smallest Value, this The value is what we want to determine The index of the insertion position in the array (the underlying feature vector) For example, if the target cumulative probability corresponding to a target augmented feature has an index of 3 in the cumulative probability distribution, it means that its target insertion position is between the 3rd basis vector element and the 4th basis vector element in the basic feature vector.

[0204] Furthermore, the array The elements in are inserted into the array according to the index value of the determined insertion position. The target feature vector is obtained .

[0205] For example, Figure 3 As shown, it is a flow chart of an exemplary model lightweight method provided in an embodiment of the present application. Taking the model to be lightweight as a face super-resolution model as an example, first, the face super-resolution model is constructed, and then the trained model is used as the model to be lightweight, and deep dimensionality reducible clustering processing and weighted feature amplification processing are performed on it in turn to obtain a lightweight face super-resolution model (target model). Among them, in the deep dimensionality reducible clustering processing stage, the convolution parameters are first embedded with position parameters, and then feature decomposition and amplification weight generation are performed to obtain the basic feature vector and the amplification weight of each basic vector element in the basic feature vector. By adding weight terms in the process of feature amplification, each basic vector element performs a different number of amplification channels according to different weight values, that is, a different number of amplified features are generated. The model lightweight method provided in the embodiment of the present application can achieve the effect of model distillation, but there is no need to build a small distillation model architecture.

[0206] The model lightweight method provided by the embodiment of the present application obtains the convolution parameters of each convolution filter in the model to be lightweight; performs dimensionality reduction processing on the convolution parameters of each convolution filter to obtain the initial feature vector of each convolution filter; determines the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter; wherein the length of the basic feature vector is less than the length of the initial feature vector; determines the target amplification feature of each basic vector element according to the basic feature vector and the amplification weight corresponding to each basic vector element in the basic feature vector; inserts the target amplification feature of each basic vector element into the basic feature vector to obtain the target feature vector; constructs the target model corresponding to the model to be lightweight based on the target feature vector, obtains the image to be identified; identifies the image to be identified based on the target model to obtain the image recognition result of the image to be identified. The method provided by the above scheme determines the basic feature vector with significant features and small data volume in the lightweight model, and inserts the amplification feature into the basic feature vector, while reducing the parameter amount of the target model, it largely ensures that the performance of the model will not be reduced due to the lightweight model, thereby improving the accuracy of the image recognition result of the image to be identified. Moreover, by using cluster analysis to obtain the basic feature vector, and then using the parameter-free shift operation to amplify the basic feature vector, the amount of calculation is significantly reduced. Only part of the basic features are involved in the convolution operation, and the rest only need to perform the parameter-free shift operation, which significantly reduces the calculation. In addition, cluster analysis can better visualize the feature map and achieve better convergence effect than traditional convolution. Moreover, the dimensionality reduction of the embodiment of the present application can still maintain a good clustering effect and obtain a high-quality basic feature vector. The proposed weighted feature amplification method amplifies different basic vector elements according to weights, which is more in line with the expression of lightweight models such as face recognition models. The Gaussian distributed embedding method adds more randomness, making the final target model more generalizable.

[0207] like Figure 4 , which is a schematic diagram of the structure of a model lightweight device provided in an embodiment of the present application. The model lightweight device 40 comprises: a first acquisition module 401, a dimensionality reduction module 402, a first determination module 403, a second determination module 404, an insertion module 405, a lightweight module 406, a second acquisition module 407 and an identification module 408.

[0208] Among them, the first acquisition module is used to obtain the convolution parameters of each convolution filter in the model to be lightweight; the dimension reduction module is used to perform dimension reduction processing on the convolution parameters of each convolution filter to obtain the initial feature vector of each convolution filter; the first determination module is used to determine the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter; wherein the length of the basic feature vector is less than the length of the initial feature vector; the second determination module is used to determine the target amplification feature of each basic vector element according to the basic feature vector and the amplification weight corresponding to each basic vector element in the basic feature vector; the insertion module is used to insert the target amplification feature of each basic vector element into the basic feature vector to obtain the target feature vector; the lightweight module is used to construct the target model corresponding to the model to be lightweight based on the target feature vector; the second acquisition module is used to obtain the image to be identified; the identification module is used to identify the image to be identified based on the target model to obtain the image recognition result of the image to be identified.

[0209] Regarding the model lightweight device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0210] The model lightweight device provided in the embodiment of the present application is used to execute the model lightweight method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0211] An embodiment of the present application provides an electronic device for executing the model lightweighting method or image recognition method provided in the above embodiments.

[0212] like Figure 5 FIG. 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 50 includes: at least one processor 51 and a memory 52.

[0213] The memory stores computer-executable instructions; at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the model lightweight method or image recognition method provided in the above embodiment.

[0214] The electronic device provided in the embodiment of the present application is used to execute the model lightweight method or image recognition method provided in the above-mentioned embodiment. Its implementation method and principle are the same and will not be repeated here.

[0215] An embodiment of the present application provides a non-volatile computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, a model lightweight method or an image recognition method provided in any of the above embodiments is implemented.

[0216] The storage medium containing computer executable instructions provided in the embodiments of the present application can be used to store computer executable instructions of the model lightweight method or image recognition method provided in the aforementioned embodiments. The implementation method and principle are the same and will not be repeated here.

[0217] An embodiment of the present application provides a computer program product, including computer instructions, which are used to enable a computer to execute the model lightweight method or image recognition method provided in the aforementioned embodiment.

[0218] The embodiments of the present application provide a computer program product that can be used to execute computer instructions of the model lightweight method or image recognition method provided in the aforementioned embodiments. The implementation method and principle are the same and will not be repeated here.

[0219] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0220] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0221] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0222] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0223] A part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0224] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0225] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A model lightweight method, characterized in that: include: Get the convolution parameters of each convolution filter in the model to be lightweight; Performing dimensionality reduction processing on the convolution parameters of each of the convolution filters to obtain an initial feature vector of each of the convolution filters; Determine, according to the initial feature vectors of each of the convolution filters, a basic feature vector of the model to be lightweight and an amplification weight corresponding to each basic vector element in the basic feature vector; wherein the length of the basic feature vector is less than the length of the initial feature vector; Determining a target amplified feature of each basic vector element according to the basic feature vector and an amplification weight corresponding to each basic vector element in the basic feature vector; Determining an amplified feature standard deviation and an amplified feature mean of the basic feature vector according to a target amplified feature of each of the basic vector elements; For a target amplification feature of any of the basic vector elements, determining a target distribution probability value of the target amplification feature according to the target amplification feature, the amplification feature standard deviation and the amplification feature mean; Determining a target insertion position of each of the target amplification features according to a target distribution probability value of each of the target amplification features; Inserting each of the target augmented features into the basic feature vector according to the target insertion position of each of the target augmented features to obtain a target feature vector; Based on the target feature vector, construct a target model corresponding to the model to be lightweight; Obtain an image to be recognized; Based on the target model, the image to be identified is identified to obtain an image recognition result of the image to be identified; The step of determining a target augmented feature of each basic vector element according to the basic feature vector and the augmented weight corresponding to each basic vector element in the basic feature vector comprises: Generate a basic feature map according to the basic feature vector; Determining the number of amplicon features corresponding to each basic vector element in the basic feature according to the amplification weight corresponding to each basic vector element in the basic feature vector; For any of the basic vector elements, mapping the basic vector element as a traversal center to the basic feature map; According to the preset offset and the number of amplified sub-features of the basic vector element, a key feature traversal is performed on the traversal center of the basic feature graph to obtain the target amplified feature of the basic vector element.

2. The method according to claim 1, characterized in that The step of performing dimensionality reduction processing on the convolution parameters of each convolution filter to obtain an initial feature vector of each convolution filter includes: For any of the convolution filters, position encoding is performed on the convolution parameters of the convolution filter to obtain a target position encoding of the convolution filter; The convolution parameters of the convolution filter and the target position code are concatenated to obtain a target convolution parameter vector of the convolution filter; Performing dimensionality reduction processing on the target convolution parameter vector of each of the convolution filters to obtain an initial feature vector of each of the convolution filters.

3. The method according to claim 2, characterized in that For any of the convolution filters, position encoding is performed on the convolution parameters of the convolution filter to obtain the target position encoding of the convolution filter, including: For any of the convolution filters, obtaining position information of the convolution filter during the convolution calculation process of the model to be lightweighted; According to the position information, the convolution parameters of the convolution filter are position-encoded to obtain the target position code of the convolution filter.

4. The method according to claim 2, characterized in that: The step of performing dimensionality reduction processing on the target convolution parameter vector of each convolution filter to obtain an initial feature vector of each convolution filter includes: Converting the target convolution parameter vector of each convolution filter into a one-dimensional vector to obtain the target one-dimensional vector of each convolution filter; Calculating the similarity between the target one-dimensional vectors of each of the convolution filters to obtain a similarity calculation result; According to the similarity calculation result, a similarity matrix is ​​constructed; wherein the similarity matrix is ​​used to uniformly represent the similarity between the target one-dimensional vectors of each convolution filter; According to the similarity matrix, converting the target one-dimensional vector of each convolution filter into an initial feature vector; The length of the initial feature vector is smaller than the length of the target one-dimensional vector.

5. The method according to claim 4, characterized in that The step of converting the target one-dimensional vector of each convolution filter into an initial feature vector according to the similarity matrix includes: Performing eigenvalue decomposition on the similarity matrix to obtain an eigenvalue decomposition result of the similarity matrix; According to the eigenvalue decomposition result of the similarity matrix, an initial eigenvector corresponding to the target one-dimensional vector of each convolution filter is determined.

6. The method according to claim 1, characterized in that The step of determining the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter includes: Performing element clustering processing on the initial feature vectors of each convolution filter to obtain a plurality of element cluster centers and a sample size of each element category; wherein the number of the element cluster centers is determined according to a preset amplification feature ratio and the number of channels of the convolution filter; According to the multiple element clustering centers and the sample size of each element category, the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector are determined.

7. The method according to claim 6, characterized in that The step of determining the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the multiple element clustering centers and the sample size of each element category includes: Taking the multiple element cluster centers as basic vector elements, constructing a basic feature vector of the model to be lightweight; According to the sample size of each element category, the amplification weight corresponding to each basic vector element in the basic feature vector is determined.

8. The method according to claim 1, characterized in that The step of determining the number of amplicon features corresponding to each basic vector element in the basic feature according to the amplification weight corresponding to each basic vector element in the basic feature vector comprises: The number of amplicon features corresponding to each basic vector element in the basic feature is determined based on the following formula: in, represents the first The number of amplicon features per basis vector element, represents the first The augmentation weights of the basis vector elements, Indicates the preset amplification feature ratio, Indicates the number of channels of the convolution filter.

9. The method according to claim 1, characterized in that: The step of performing key feature traversal on the traversal center of the basic feature graph according to the preset offset and the number of amplicon features of the basic vector element to obtain the target amplified feature of the basic vector element includes: Determining the target traversal round number corresponding to the basic vector element according to the number of amplicon features of the basic vector element; wherein the number of amplicon features is equal to the target traversal round number; In the case of performing a key feature first traversal on the traversal center of the basic feature graph, performing a key feature traversal on the traversal center of the basic feature graph according to the preset offset to obtain an amplicon feature of the first traversal; In the case where the key feature is not traversed for the first time on the traversal center of the basic feature graph, iteratively adjusting the preset offset according to the current traversal round number to obtain the current round target offset until the current traversal round reaches the target traversal round number; According to the target offset of this round, key feature traversal is performed on the traversal center of the basic feature graph to obtain amplicon features that are not traversed for the first time; The target amplification feature of the basic vector element is determined according to the amplicon feature of the first traversal and all the amplicon features of the non-first traversal.

10. The method according to claim 1, characterized in that The target amplification feature for any of the basic vector elements, determining a target distribution probability value of the target amplification feature according to the target amplification feature, the amplification feature standard deviation and the amplification feature mean, comprises: For a target amplified feature of any of the basic vector elements, determining an initial distribution probability value of the target amplified feature according to the target amplified feature, the amplified feature standard deviation and the amplified feature mean; The initial distribution probability value of each target amplification feature is normalized to obtain the target distribution probability value of each target amplification feature.

11. The method according to claim 10, characterized in that Determining the initial distribution probability value of the target amplification feature according to the target amplification feature, the amplification feature standard deviation and the amplification feature mean value includes: Based on the following formula, the initial distribution probability value of the target amplification feature is determined: in, represents the first The initial distribution probability value of the target augmented feature corresponding to the basis vector elements, represents the first The target augmented features corresponding to the basis vector elements, represents the standard deviation of the amplification characteristics, represents the mean of the amplification characteristics.

12. The method according to claim 1, characterized in that Determining the target insertion position of each target amplification feature according to the target distribution probability value of each target amplification feature includes: Determining a cumulative probability distribution of target amplification features according to the target distribution probability value of each of the target amplification features; The target insertion position of each of the target amplification features is determined according to the cumulative probability distribution of the target amplification features.

13. The method according to claim 12, characterized in that Determining the cumulative probability distribution of the target amplification feature according to the target distribution probability value of each target amplification feature includes: The cumulative probability distribution of the target amplification feature is determined based on the following formula: in, represents the cumulative probability distribution, represents the cumulative probability distribution The cumulative probability, represents the total number of target amplification features, Indicates The target distribution probability value of the target amplification feature.

14. The method according to claim 12, characterized in that Determining the target insertion position of each target amplification feature according to the cumulative probability distribution of the target amplification feature includes: Generate a cumulative probability threshold for each target amplification feature according to a preset threshold generation method; For any of the target amplification features, according to the cumulative probability threshold of the target amplification feature, traverse the target cumulative probability in the cumulative probability distribution; Determining a target insertion position of the target amplification feature according to an index value of the target cumulative probability in the cumulative probability distribution; The target cumulative probability is the minimum cumulative probability in the cumulative probability distribution that is not less than the cumulative probability threshold.

15. A model lightweight device, characterized in that: include: A first acquisition module is used to obtain the convolution parameters of each convolution filter in the model to be lightweight; A dimensionality reduction module, used for performing dimensionality reduction processing on the convolution parameters of each convolution filter to obtain an initial feature vector of each convolution filter; A first determination module is used to determine the basic feature vector of the model to be lightweight and the amplification weight corresponding to each basic vector element in the basic feature vector according to the initial feature vector of each convolution filter; wherein the length of the basic feature vector is less than the length of the initial feature vector; A second determination module, configured to determine a target amplified feature of each basic vector element according to the basic feature vector and an amplification weight corresponding to each basic vector element in the basic feature vector; An insertion module is used to determine the amplification feature standard deviation and the amplification feature mean of the basic feature vector according to the target amplification feature of each basic vector element; for the target amplification feature of any basic vector element, determine the target distribution probability value of the target amplification feature according to the target amplification feature, the amplification feature standard deviation and the amplification feature mean; determine the target insertion position of each target amplification feature according to the target distribution probability value of each target amplification feature; and insert each target amplification feature into the basic feature vector according to the target insertion position of each target amplification feature to obtain a target feature vector; A lightweight module, used for constructing a target model corresponding to the model to be lightweight based on the target feature vector; A second acquisition module is used to acquire an image to be recognized; A recognition module, used for recognizing the image to be recognized based on the target model to obtain an image recognition result of the image to be recognized; The second determining module is specifically used to: Generate a basic feature map according to the basic feature vector; Determining the number of amplicon features corresponding to each basic vector element in the basic feature according to the amplification weight corresponding to each basic vector element in the basic feature vector; For any of the basic vector elements, mapping the basic vector element as a traversal center to the basic feature map; According to the preset offset and the number of amplified sub-features of the basic vector element, a key feature traversal is performed on the traversal center of the basic feature graph to obtain the target amplified feature of the basic vector element.

16. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the model lightweighting method according to any one of claims 1 to 14.

17. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the model lightweighting method according to any one of claims 1 to 14 is implemented.

18. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the model lightweighting method according to any one of claims 1 to 14.

Images