Model training method, object processing method and device, electronic device, and medium

Abstract

The embodiment of the present disclosure relates to a model training method, an object processing method and device, electronic equipment and storage medium, and relates to the technical field of computers. The model training method comprises: obtaining an original model; if the original model converges, performing sensitivity analysis on the original model to determine a sensitivity, and obtaining an actual clustering number of each layer of the original model according to the sensitivity; clustering the weight values of each layer according to the actual clustering number of each layer to compress the original model through parameter sharing, and training the compressed original model based on the clustering result to obtain a trained original model. The technical solution of the present disclosure can accurately determine the actual clustering number and compress the model by introducing sensitivity analysis of each layer of the original model to calculate the actual clustering number of each layer, thereby improving the model precision of the compressed model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular, to a model training method and device, an object processing method and device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Neural network algorithms have been widely applied to various types of tasks in the fields of machine vision, autonomous driving, etc. However, due to the complex structure, a large amount of power consumption and computing resources are occupied even at inference, which seriously affects the deployment on mobile and embedded platforms with limited power and resources, and thus the model needs to be compressed.

[0003] In the related art, all weight values of each layer can be clustered and the center values of each cluster of all layers are trained until the model converges, or the weight values of each layer are divided into multiple weight value groups according to certain rules and then the model is trained, or the compressed model is trained by regularization.

[0004] In the above manner, the number of clusters of each layer needs to be manually or empirically globally set, or the number of clusters of each layer is adjusted multiple times according to the accuracy of the compressed model, so it is difficult to balance the compression rate and accuracy. In addition, the number of clusters of each layer is the same, so the accuracy is poor and the flexibility is poor, which has certain limitations. Moreover, regularization needs to modify the training code, which is low in operation efficiency and computer operation performance, and consumes a large amount of computing resources.

[0005] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The purpose of the present disclosure is to provide a model training method, an object processing method and device, an electronic device and a storage medium, thereby at least partially overcoming the problem of low model compression accuracy caused by the limitations and defects of the related art.

[0007] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0008] According to one aspect of the present disclosure, a model training method is provided, comprising: obtaining an original model; if the original model converges, performing sensitivity analysis on the original model to determine a sensitivity, and obtaining an actual number of clusters of each layer in the original model according to the sensitivity; clustering the weight values of each layer according to the actual number of clusters of each layer to compress the original model by parameter sharing, and training the compressed original model based on the clustering result to obtain a trained original model.

[0009] According to an aspect of the present disclosure, a method for processing an object is provided, comprising: obtaining an object to be processed; performing convolution operation on the object to be processed by a trained original model to obtain a prediction result corresponding to the object to be processed; wherein the original model is trained according to the model training method in any of the above aspects.

[0010] According to an aspect of the present disclosure, a model training device is provided, comprising: a model obtaining module configured to obtain an original model; a sensitivity analysis module configured to, if the original model converges, perform sensitivity analysis on the original model to determine a sensitivity, and obtain an actual cluster number of each layer of the original model according to the sensitivity; and a compressed model training module configured to cluster the weight of each layer according to the actual cluster number of each layer to compress the original model by parameter sharing, and train the compressed original model based on the clustering result to obtain a trained original model.

[0011] According to an aspect of the present disclosure, a device for processing an object is provided, comprising: an object obtaining module configured to obtain an object to be processed; and a prediction module configured to perform convolution operation on the object to be processed by a trained original model to obtain a prediction result corresponding to the object to be processed; wherein the original model is trained according to the model training method in any of the above aspects.

[0012] According to an aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the model training method in any of the above aspects or the object processing method in any of the above aspects by executing the executable instructions.

[0013] According to an aspect of the present disclosure, a computer readable storage medium is provided, having a computer program stored thereon, wherein the computer program is executed by a processor to implement the model training method in any of the above aspects or the object processing method in any of the above aspects.

[0014] In the model training method, the model training apparatus, the object processing method, the object processing apparatus, the electronic device, and the computer-readable storage medium provided in the embodiments of the present disclosure, on the one hand, the sensitivity is obtained by performing sensitivity analysis on the original model before training the model, and the actual clustering number of each layer in the original model is obtained according to the sensitivity, thereby avoiding the problem that in the related art, the clustering number of each layer can only be determined in a manual manner, the actual clustering number of each layer is determined in a targeted manner instead of being fixed, the accuracy and flexibility of the actual clustering number are improved, the accuracy and precision of the compressed model are improved, the limitation is avoided, the generality is improved, and the accuracy of model training is improved. On the other hand, the model compression is implemented by calculating the sensitivity, the training code does not need to be modified, only the step of performing sensitivity analysis on each layer needs to be added, the steps required to be performed by the computer during training of the model are reduced, the processing speed and operation performance of the computer are improved, the power consumption is reduced, the required computing resources are reduced, and the required computing resources are reduced.

[0015] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0017] Figure 1 A schematic diagram of a system architecture to which the model training method or the object processing method of the embodiments of the present disclosure can be applied is shown.

[0018] Figure 2 A schematic diagram of a model training method in the embodiments of the present disclosure is schematically shown.

[0019] Figure 3 A flowchart of a first sensitivity analysis in the embodiments of the present disclosure is schematically shown.

[0020] Figure 4 A flowchart of calculating a current loss function value in the embodiments of the present disclosure is schematically shown.

[0021] Figure 5 A flowchart of a second sensitivity analysis in the embodiments of the present disclosure is schematically shown.

[0022] Figure 6 A schematic diagram of parameter sharing in the embodiments of the present disclosure is schematically shown.

[0023] Figure 7 Fig. 1 schematically shows a flowchart of training a first model according to an embodiment of the present disclosure.

[0024] Figure 8 Fig. 2 schematically shows a flowchart of training a second model according to an embodiment of the present disclosure.

[0025] Figure 9 Fig. 3 schematically shows a flowchart of training a compressed model according to an embodiment of the present disclosure.

[0026] Figure 10 Fig. 4 schematically shows a flowchart of an object processing method according to an embodiment of the present disclosure.

[0027] Figure 11 Fig. 5 schematically shows a block diagram of a model training apparatus according to an embodiment of the present disclosure.

[0028] Figure 12 Fig. 6 schematically shows a block diagram of an object processing apparatus according to an embodiment of the present disclosure.

[0029] Figure 13 Fig. 7 schematically shows a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0031] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and serve to explain the principles of the present disclosure. The drawings presented herein are not to be construed as limiting the present disclosure in any manner. In the drawings, like reference numerals designate like parts throughout the various figures thereof, and the repetitions of like reference numerals are omitted hereafter. Further, some of the blocks in the flowcharts and block diagrams can be implemented in software, firmware, hardware, or a combination thereof. In the case of a software implementation, program instructions might be used to implement some of the blocks in the flowcharts and block diagrams. Program instructions can be stored in memory, such as random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or Flash memory), or any other form of computer-readable memory. Further, the program instructions might be downloaded to the computer from an external source via a computer network or a broadcast medium, or they might be supplied to the computer on one or more computer-readable media. The computer-readable media might include a computer program product, an article of manufacture, and / or a computer-usable medium having computer readable program instructions thereon for causing a processor to carry out operations described herein. The media and program instructions might be those specially designed and constructed for the purposes of the present disclosure, or they might be of the kind available to those having ordinary skill in the computer software art. Examples of computer- readable media include, but are not limited to, magnetic discs, magnetic tapes, optical discs, magneto-optical discs, semiconductor memories, and the like. The program instructions might include: machine language codes, machine language codes created by a compiler, higher-level programming languages, script languages, machine, or micro-processor firmware, and / or interpreter languages. The program instructions might be executed by a single computer or by a combination of a plurality of computers. The computer might be a personal computer, a workstation, a server, a device, a processor, a microprocessor, a micro-controller, a programmable logic device, a micro-processor, a digital signal processor, a portable computer, a portable data assistant, a smart phone, a personal digital assistant, a mobile phone, a mobile device, a mobile terminal, a mobile station, a mobile unit, a mobile object, a terminal, a remote station, a remote terminal, a user terminal, a user agent, a user device, a user equipment, a user station, a user unit, a user node, a user machine, a subscriber station, a subscriber terminal, a subscriber unit, a subscriber number, a wireless terminal, a wireless unit, a wireless device, a wireless object, a wireless node, a wireless

[0032] The related art mainly includes the following schemes: Mode one, all weights in each layer of the pre-trained neural network are respectively clustered using the K-Means algorithm (for example, the CNN model AlexNet uses 256 classes for all convolutional layers and 32 classes for all fully connected layers; the number of classes for other models can be set as needed), the center value of the cluster to which each weight belongs is used to replace the original value, and then the center values of all layers of each cluster are trained for a period of time until the model converges, and the model at this time is passed to the subsequent stage for processing. However, this mode needs to manually set the number of clusters for each layer or each type of layer globally based on experience, or adjust multiple times according to the accuracy performance of the compressed neural network, which cannot guarantee the optimal balance between compression rate and accuracy. In a convolutional neural network, even the same type of layer, different layers at different positions play completely different roles and have different sensitivity to compression, but this mode globally applies the same number of clusters, resulting in that the layers with high sensitivity are compressed too much, the expression ability is greatly lost, and the layers with low sensitivity are not compressed enough.

[0033] Mode two, each layer of weights is evenly divided into several weight groups according to a certain rule. For example, in a convolutional neural network, a four-dimensional weight tensor with a size of [h, w, c, n] can be divided into n three-dimensional [h, w, c] tensors according to the output channel, each of which contains the same number of floating-point weights, h×w c. After regarding each weight group as a vector in a high-dimensional space, the K-Means clustering algorithm is run on these vectors, and the complete weight group is then taken as the minimum unit for updating in the subsequent training. In this way, although the code table needs more space to store the center values of each cluster, the space for storing the index is reduced (multiple weights are combined into a single index value). Wherein, hwcn represent the vertical direction of the convolution kernel, the horizontal direction of the convolution kernel, the input channel and the output channel dimension respectively. This mode may not have higher accuracy than the scheme of clustering single weights at the same compression rate. Moreover, it is no longer applicable to all networks, and has poor universality.

[0034] The third mode, the clustering of the above two modes is one-time, once the analysis of the pre-training NN weight value at the beginning is passed, an index value is assigned to each weight (group), and thereafter the index value is not changed, and the training only updates the weight value corresponding to the code table index value, that is, the center value of the belonging cluster. The one-time clustering assignment leads to the decrease of the representativeness of the cluster center value, which affects the accuracy of the compressed NN. Therefore, before the actual execution of the clustering, the target function needs to be modified in the training, and a regularization term is added to promote the distribution of the weights to be as close as possible to the limited number of center points under the premise of ensuring a certain accuracy (for some regularization methods, the number of center points is pre-set; in other regularization methods, the center points are naturally formed after training). This mode may cause the distribution of the weights to be different from the result of the unregularized training, and may not be suitable for occasions with high accuracy requirements (such as denoising, super-resolution, and other pixel-level tasks in the image field). In addition, some regularization methods require complex operations such as eigenvalue decomposition on the weight tensor, which not only requires modification of the original NN training code, but also seriously prolongs the training time and increases a large amount of resource consumption for deep neural networks with large parameter quantities, and the practicability is poor.

[0035] To solve the above technical problems, a model training method provided in the embodiments of the present disclosure can be applied to various types of application scenarios such as information extraction, face recognition, image classification, etc.

[0036] Figure 1 A schematic diagram of a system architecture to which the model training method and device or object processing method and device of the embodiments of the present disclosure can be applied is shown.

[0037] As shown in Figure 1 The system architecture 100 can include a client 101, a network 102, and a server 103. The client can be a client, such as a smartphone, a computer, a tablet computer, a smart speaker, etc. The network 102 is a medium for providing a communication link between the client 101 and the server 103. The network 102 can include various connection types, such as wired communication links, wireless communication links, etc. In the embodiments of the present disclosure, the network 102 between the client 101 and the server 103 can be a wired communication link, such as a serial port connection line to provide a communication link, or a wireless communication link to provide a communication link through a wireless network. The server 103 can be a server with computing function or a client, such as a portable computer, a desktop computer, a smartphone, etc. The server 103 is used to process the data or model sent by the client.

[0038] The model training method can be applied to the training scene of the original model. Referring to Figure 1As shown in the figure, the client 101 can send the original model to the server 103 through the network 102, and the server 103 first judges whether the received original model converges, and if so, subsequent processing is performed; if not, the training is continued until the model converges. Further, the sensitivity analysis is performed on the received original model to calculate the actual cluster number of each layer in the original model to realize the compression of the original model based on parameter sharing to obtain the compressed original model, the weight of each layer is clustered according to the actual cluster number, and the model training is performed on the compressed original model based on the clustering result. After the training of the compressed original model is completed, the client sends the object to be processed to the server, and the server calls the trained original model to process the object to be processed to obtain the prediction result corresponding to the object to be processed. And the prediction result can be sent to the client 101 for display and other processing operations. The server can be the same as the client, that is, the client and the server are both intelligent devices, for example, can be a smart phone. The server can also be different from the client, which is not specially limited here.

[0039] It should be noted that the model training method and the object processing method provided by the embodiments of the present disclosure can be completely executed by the server or the client. Correspondingly, the model training device and the object processing device can be arranged in the server. It can also be completely executed by the client.

[0040] Next, reference is made to Figure 2 The model training method in the embodiments of the present disclosure is described in detail.

[0041] In step S210, the original model is obtained.

[0042] In the embodiments of the present disclosure, the original model can be any type of model, for example, can be a machine learning model, a deep learning model, or a neural network model, etc. The original model can be a floating point model (a model with floating point parameters); can also be a fixed point model with any bit width, for example, 16-bit, 12-bit, 11-bit, 10-bit, 8-bit, 7-bit, 6-bit, 4-bit, etc. fixed point model; can also be an integer model. When the original model is a neural network model, it can be any one of a convolutional neural network, a deep neural network, and a recurrent neural network. The original model can include multiple layers of structures such as convolutional layers, pooling layers, fully connected layers, activation layers, RNN cycle units (RNN cells), etc. Among them, the number of layers that can perform weight clustering in the original model is n, and since there can be some layer types without weights, the number of layers that can perform weight clustering can be all layers of the original model or some layers that support sharing, that is, 0 < n < NN. The layers that can perform weight clustering can include, but are not limited to, convolutional layers, fully connected layers, RNN cycle units (RNN cells), etc.

[0043] Further, each layer for weight clustering supports setting the number of clusters as any one of a set K = {k0,..., k b-1}, where b e N* is the total number of supported clusters, and k0<... < k b–1 increases. The set K of the number of clusters of each layer can be the same or different, and the elements in each set can be any positive integer. When compressing the neural network model, the storage space and access bandwidth can be reduced, and the compression process is reflected in the number of bits required for each (group) weight. For example, assuming that the running platform supports quantization bit width of 8 bits, 6 bits or 4 bits, k can be set to 256, 64 or 16 accordingly, so that the index value can be expressed with a corresponding limited bit width. In order to fully utilize the given compression ratio, the number of clusters should take the maximum allowed value under the same bit width, so as to improve the expression ability of the neural network model, and therefore k can be directly taken as an integer power of 2, in addition to which k can also be any positive integer, which is not specifically limited here.

[0044] In the embodiments of the present disclosure, if the obtained original model has converged, subsequent processing is continued. If the obtained original model has not converged, the original model is trained until the original model converges or a training condition is reached. The training condition can be, for example, a maximum step limit, in addition to which it can also be other limiting conditions. When training the original model, the model parameters of the original model can be adjusted according to the input and the label corresponding to the input until it converges, so as to obtain a converged original model.

[0045] After obtaining the original model, if the application scenario in which the original model is located supports adjusting the weight distribution, the weight distribution of the original model can be adjusted through a pre-processing manner. The pre-processing manner can be any type of adjustment manner as long as it can adjust the weight distribution of the original model. Specifically, the weight distribution of the original model can be adjusted based on the retraining of spectrally relaxed K-Means regularization, so that the weight distribution of the original model is more concentrated. It should be noted that the pre-processing manner itself does not perform K-Means clustering, but performs training with a regularization loss term according to the needs of subsequent clustering, that is, driving each weight to concentrate on K center values, so as to complete clustering and reduce the number of weights located in the fuzzy zone between multiple clusters. Each center is naturally formed according to data distribution, rather than being artificially specified in advance. Adjusting the weight distribution of the original model through the pre-processing manner to make it more concentrated can improve the clustering accuracy.

[0046] In step S220, if the original model converges, sensitivity analysis is performed on the original model to determine sensitivity, and the actual clustering number of each layer in the original model is obtained according to the sensitivity.

[0047] In the embodiments of the present disclosure, if the original model converges, sensitivity analysis can be performed on the original model. Sensitivity analysis is used to study how the uncertainty in the numerical or other outputs of a mathematical model or system is distributed to different sources of uncertainty in its inputs. For parameter sharing in the field of artificial intelligence, sensitivity analysis is used to determine the contribution of each weight (group) of each layer to the accuracy loss of the original model after compression. The higher the sensitivity, the greater the accuracy loss caused by compression, and the compression rate needs to be correspondingly reduced (achieved by increasing the clustering number), while the sensitivity of the lower layer can be appropriately increased. The compression rate, and finally meet the global compression rate requirement. By performing sensitivity analysis on the original model, the actual clustering number of each layer in the original model can be obtained, which can more accurately identify important layers and unimportant layers compared to the global uniform clustering number or the parameter sharing relying on empirical modeling in related technologies, and improve the expression ability and accuracy of the compressed original model.

[0048] When performing sensitivity analysis on the original model to obtain the actual clustering number of each layer in the original model, two methods can be used. Method one is to determine the actual clustering number by using temporary copies. Method two is to obtain the actual clustering number by using a loop. Figure 3 FIG. 2 schematically shows a flowchart of sensitivity analysis using temporary copies, and FIG. 3 schematically shows a flowchart of sensitivity analysis using a loop. Figure 3 As shown in FIG. 2, the method mainly includes the following steps:

[0049] In step S310, the loss function value of the original model on the data set is calculated, and the current loss function value is calculated by using multiple temporary copies of the original model.

[0050] In this step, the data set can be the entire training set, a fixed part of the training set, or a part of the training set randomly extracted each time, and can also be a separate data set. The data sets used in multiple loops of the same step can not be the same, and the data sets used in different steps can also be the same or different.

[0051] The data set can include multiple samples. For each sample in the data set, the sample can be input into the original model, and the convolution operation is performed through the convolution layer and the full connection layer of the original model to obtain the output corresponding to the sample. The output can be compared with the real value represented by the input to obtain the loss corresponding to the sample. This is repeated to calculate the loss of each sample to obtain the loss function value of the original model on the data set. For example, the original loss can be obtained according to the original model.

[0052] In the embodiments of the present disclosure, in order to facilitate the determination of the sensitivity of the original model, a plurality of temporary copies of the converged original model can be constructed in the memory. The plurality of temporary copies can be exactly the same as the original model, and can be obtained by copying the original model, that is, the temporary copies can be the converged original model. The number of temporary copies is determined according to the number of layers in the original model that need to be clustered, that is, the number of temporary copies is the same as the number of layers that need to be clustered. For example, the number of layers in the original model that can be clustered is n, and n temporary copies of the converged original model are constructed in the memory.

[0053] After the temporary copies are constructed, the current loss function value can be calculated based on the plurality of temporary copies. The current loss function value refers to the loss function value at the current time, which can be the loss function value calculated according to each temporary copy, that is, the current loss function value and the temporary copy are one-to-one corresponding. For the plurality of temporary copies, there is one current loss function value corresponding to each temporary copy, and the current loss function value corresponding to each temporary copy can be the same or different, which is not specially limited here. For example, the current loss function value corresponding to the temporary copy 1 is loss 1, the current loss function value corresponding to the temporary copy 2 is loss 2, the current loss function value corresponding to the temporary copy 3 is loss 3, and the current loss function value corresponding to the temporary copy n is loss n, and so on.

[0054] Figure 4 The flowchart for calculating the current loss function value is shown in the middle of the figure, and Figure 4 is a specific implementation process of step S310, and calculating the current loss function value mainly includes step S410 and step S420, wherein:

[0055] In step S410, the weights of the i-th layer in the i-th temporary copy of the plurality of temporary copies are clustered to determine the clustering type, the original positions of the i-th layer are determined as the center values of the clustering type, and the weights of other layers are kept unchanged, to obtain the compressed i-th temporary copy.

[0056] In the embodiments of the present disclosure, the i-th temporary copy can be each temporary copy of the plurality of temporary copies. For the i-th temporary copy, the weights of the i-th layer can be clustered into a certain number of clustering types. The weights of the i-th layer can be determined according to the i-th temporary copy, and the layers that need to be clustered can be the same as the temporary copies that need to be processed. For example, the weights of the first layer of the first temporary copy are clustered, the weights of the second layer of the second temporary copy are clustered, and so on, until the weights of the n-th layer of the n-th temporary copy are clustered.

[0057] The number of clusters in clustering the weights of the i-th layer of the i-th temporary copy can be any one of the initial number of clusters of the i-th layer, the minimum supported number of clusters k0, and the target number of clusters in the set K determined according to other rules. The target number of clusters can be any one of the values in the set K, or can be determined according to the priority relationship between a plurality of numbers of clusters. The priority relationship can be that the initial number of clusters is greater than the minimum supported number of clusters k0 and the target number of clusters. For example, if there is an initial number of clusters, the weights of the i-th layer of the i-th temporary copy are clustered into the initial number of cluster types; if there is no initial number of clusters, the weights of the i-th layer are clustered into the minimum supported number of clusters or the target number of clusters. Since the initial number of clusters of each layer can be the same or different, each layer can be clustered into the same or different cluster types.

[0058] In the clustering process, the weights of the i-th layer of the i-th temporary copy can be clustered according to a weight clustering unit to obtain a cluster type. The weight clustering unit can be a single weight, or a relatively more structured group of weights (such as a weight bar, a filter, a feature map, etc.). Further, in the clustering process, the values of the original positions of the i-th layer can be determined as the center values of the corresponding cluster types, and the weights of all other layers remain unchanged, so as to compress the i-th temporary copy in this way to obtain a compressed i-th temporary copy. The original position refers to the original position of the grid of the i-th layer in the original model. Based on this, the weights of the i-th layer of the i-th temporary copy are clustered to obtain a cluster type corresponding to the initial number of clusters. Among them, the values of the original positions of the i-th layer are the center values of the cluster types corresponding to the original positions, and the weights of other layers remain unchanged. For example, if the original position 1 of the i-th layer of the original model is cluster type 1, the value at the original position 1 is the center value of cluster type 1; if the original position 2 is cluster type 3, the value at the original position 1 is the center value of cluster type 3; if the cluster type of the original position 4 is 4, the original position 4 is the center value of the cluster type 4 corresponding to the original position.

[0059] It should be noted that, in clustering the weights of the i-th layer of the i-th temporary copy, the clustering algorithm can be a K-Means clustering algorithm, or any other type of clustering algorithm as long as it can achieve clustering.

[0060] In step S420, the data set is forwardly passed through the compressed i-th temporary copy, and the current loss function value is calculated.

[0061] In the embodiments of the present disclosure, by clustering the weights of the i-th layer of the i-th temporary copy, the temporary copy can be compressed and adjusted, so that the current loss function value can be calculated based on a plurality of compressed temporary copies.

[0062] Specifically, the data set can be forwardly passed through the compressed i-th temporary copy, so as to calculate the current loss function value corresponding to the compressed i-th temporary copy. Specifically, a sample in the data set can be input into the compressed i-th temporary copy to determine the output of the first layer, the output of the last layer is taken as the input of the next layer to calculate the output of the next layer, the above steps are repeated until the output layer is calculated, and the output corresponding to the sample is obtained. Further, the loss of the i-th layer of the sample is obtained according to the difference between the output and the real value represented by the input, and the above steps are repeatedly executed to calculate the loss of each sample in the i-th layer, so as to obtain the current loss function value corresponding to the data set forwardly passing through the compressed i-th temporary copy.

[0063] With reference back to Figure 3 In step S320, the sensitivity of each layer is determined according to the loss function value and the current loss function value.

[0064] In this step, after the loss function value of the original model and the current loss function value corresponding to the compressed i-th temporary copy are calculated, each current loss function value can be compared with the loss function value, and the current loss function value is subtracted from the loss function value to obtain the increment of the current loss function value relative to the loss function value. Further, the sensitivity of each layer in the original model can be determined according to the increment and the correlation between the increment and the sensitivity. The correlation is that the increment between the current loss function value and the loss function value is positively correlated with the sensitivity, that is, the greater the loss increment, the higher the sensitivity, and the smaller the loss increment, the lower the sensitivity. The positive correlation can include but is not limited to linear, quadratic, cubic, exponential and logarithmic correlation. For example, the sensitivity of the i-th layer can be determined according to the increment between the current loss function value and the loss function value obtained by the i-th temporary copy, and the correlation between the increment and the sensitivity. By traversing all temporary copies and repeating the above steps S310 and S320, the sensitivity of each layer in the original model can be obtained. It should be noted that the current loss function value of the i-th layer can be greater than the loss function value. If the current loss function value is less than the loss function value, it is considered that the current loss function value has no change, and the increment is zero.

[0065] After determining the sensitivity of each layer in the original model, the sensitivity of each layer can be standardized to convert the sensitivity of each layer to a sensitivity between 0 and 1. Based on this, the sensitivity of the layer with no loss increase is 0, and the sensitivity of the layer with more loss increase is 1.

[0066] In step S330, the actual cluster number of each layer in the original model is determined based on the sensitivity of each layer.

[0067] In this step, the actual cluster number of each layer can be calculated based on the condition constraint of the global compression rate and the correlation between the sensitivity and the compression rate of each layer. The condition constraint of the global compression rate can be the global target compression rate r, which can be determined by dividing the original size of the original model by the compressed size. Here, the size can be the size of the parameter quantity. The correlation between the sensitivity and the compression rate of each layer is negatively correlated (i.e., positively correlated with the cluster number). Based on this, the actual cluster number of each layer can be calculated under the constraint condition represented by the global target compression rate. The actual cluster number is used to represent the number of weights that actually participate in clustering. Each actual cluster number is a positive integer in the cluster number set K, and the actual cluster number can be less than or equal to the initial cluster number.

[0068] Figure 3 The technical solution in the method, by establishing multiple temporary copies, and by clustering one layer of each temporary copy to compress the temporary copy, and then calculating the current loss function value according to the compressed temporary copy, and determining the sensitivity of each layer in the original model according to the current loss function value and the increment between the loss function values. Then, according to the sensitivity of each layer, the actual cluster number of each layer is determined. The actual cluster number of each layer can be determined according to the different characteristics of each layer in the original model and the different influences on the network expression ability, which can reasonably determine the cluster number of each layer and improve the accuracy. In addition, the calculation of the sensitivity is based on the real inference performance of a large amount of training data, rather than using empirical rules to approximate, which can improve the accuracy of the sensitivity calculation and maximize and explain the impact of weight clustering on the expression ability of the original model.

[0069] In addition, a loop method can also be used, in which the weights of all layers of the original model are first cached, then only one layer is clustered at the beginning of each loop, the original model weights are restored from the cache after the loss function operation is completed, and the next loop is entered, so that the sensitivity can be analyzed layer by layer without creating all temporary copies in advance. Through the above method, the memory for storing the temporary copies of the original model changes can be saved, and the memory consumption can be reduced.

[0070] Figure 5 The flowchart for determining the sensitivity using the loop method is shown in Figure 5 As shown in

[0071] In step S510, the loss function of all temporary copies is calculated forward to obtain the current loss function value of each temporary copy.

[0072] In step S520, the increment of each current loss function value with respect to the loss function value of the original model is back-propagated to the cluster number in each layer of each temporary copy to obtain the actual cluster number of each layer in the next iteration.

[0073] In step S530, forward loss calculation is performed according to the actual cluster numbers of the layers in the next iteration until the actual cluster numbers of the layers are all converged to obtain the actual cluster number of each layer.

[0074] In the embodiments of the present disclosure, the cluster number of each layer can be taken as a trainable parameter, and the increment between the current loss function value and the loss function value can be taken as a gradient, which is back-propagated to the parameter represented by the cluster number, so as to realize the training process of the cluster number. It should be noted that the temporary copies here refer to the compressed temporary copies. Specifically, the cluster type can be determined by clustering the weight of the i th layer of the i th temporary copy, the original positions of the i th layer are determined as the center values of the cluster type to which they belong, and the weight of the other layers is kept unchanged, to obtain the compressed i th temporary copy.

[0075] Specifically, in the sensitivity calculation process, after completing the forward loss calculation of all the compressed temporary copies in one round, the increment of the loss function value of each compressed temporary copy compared with the loss function value of the original model is back-propagated to the initial clustering strategy of each temporary copy to obtain the cluster number of each layer in the next iteration. The initial clustering strategy refers to that each temporary copy is clustered into an equal number of categories in the respective target layer (the layer that needs to be clustered). Based on this, the cluster number of the layer with high sensitivity is increased, and vice versa, and the cluster number of each layer begins to be different from the initial value. On this basis, the forward loss calculation is performed again, and the cycle is repeated until the cluster numbers of all layers are converged. That is, the loss function of all the compressed temporary copies is forward calculated again, the increment of the loss function value of each compressed temporary copy compared with the loss function value of the original model is back-propagated to the initial clustering strategy of each temporary copy to obtain the cluster number of each layer in the next iteration. The above forward calculation and back-propagation process is repeated until the cluster numbers of all layers are converged to train the actual cluster number of each layer.

[0076] By taking the cluster number as a training parameter to determine the actual cluster number of each layer, the accuracy of the actual cluster number can be improved through iteration, the operation of converting the sensitivity into the actual cluster number is avoided, and the operation efficiency can be improved.

[0077] In step S230, the weight of each layer is clustered according to the actual cluster number of each layer to compress the original model through parameter sharing, and the compressed original model is trained based on the clustering result to obtain a trained original model.

[0078] In the embodiments of the present disclosure, according to the actual number of clusters of each layer of the original model calculated, clustering of weights is performed in each layer according to the weight clustering unit. The weight clustering unit can be a single weight, or a relatively more structured group of weights, such as a weight strip, a filter, a feature map, etc. The weight clustering range can be layer-by-layer clustering or axis-by-axis clustering, which is not specifically limited here. The clustering algorithm can be a K-means clustering algorithm or other algorithms. Based on this, the weights of each layer can be clustered according to the actual number of clusters of each layer, so as to compress the original model through parameter sharing, thereby obtaining a compressed original model.

[0079] Parameter sharing refers to that weights with close sizes or weight groups with small distances in the original model are clustered into a few clusters through a clustering algorithm. The mean of all weights in the same cluster is taken as the center value of the cluster, replacing the original values of all weights (groups) in the cluster. The clustering algorithm can be a K-means clustering algorithm or a K-Means algorithm, etc. For the network corresponding to the compressed original model, only the index value (generally integer, smaller bit width) of the cluster to which each weight belongs is stored at the original position of the weight instead of the original weight (generally floating point, larger bit width), and a code table stored in the original bit width is used to achieve the purpose of compressing the storage space and the memory reading bandwidth.

[0080] For example, the weight belonging to the 0th cluster is stored as 0, and the 0th value is searched from the code table during calculation; the weight belonging to the 1st cluster is stored as 1, and so on. The weight tensor of an original model has n weights in all layers, and the original bit width of each weight is b bits. After parameter clustering of k clusters is performed, the compression ratio achieved can be calculated as follows:

[0081]

[0082] After training, all weights in the same cluster are updated synchronously. Specifically, only the center value of each cluster is updated, and the index value is not updated, that is, the weight originally belonging to the ith cluster still belongs to the ith cluster after training, but the ith code table value corresponding to the weight may be updated during training, Through the above operations, the number of effective parameters in the compressed original model is significantly reduced compared to the original model, thereby reducing the degrees of freedom of the network parameters and achieving a regularization effect to avoid overfitting. In addition, converting the data stored in each weight bit from floating-point weights to integer index values ​​through a clustering algorithm reduces the number of non-repeating data points in the weight tensor, improves data regularity, and facilitates further reduction of storage space and transmission bandwidth requirements using existing lossless compression coding techniques (such as Huffman coding). By clustering the weights of each layer according to the actual number of clusters in each layer of the original model, parameter sharing can be used to compress the original model, resulting in a compressed original model and improving the accuracy of model compression.

[0083] refer to Figure 6 As shown, when training and updating weights, the gradients of the same clusters can be accumulated and updated to the center point. The accumulated values ​​are used to obtain the dimensionality-reduced values. Then, the dimensionality-reduced values ​​are added to the cluster centers to obtain the fine-tuned cluster centers.

[0084] Furthermore, the compressed original model can be trained based on the clustering results to obtain a trained original model. It should be noted that the compressed original model can be trained by calculating gradients and modifying the computation graph.

[0085] Figure 7 The diagram illustrates a flowchart of training the compressed original model by calculating gradients. (See reference...) Figure 7 As shown, the main steps include:

[0086] In step S710, the gradient for each weight bit is calculated, and the gradients corresponding to all weight bits belonging to the same cluster are accumulated to update the center points of the same cluster.

[0087] In this step, the weights refer to each dimension of the high-dimensional clustering space. First, the gradient of each weight can be calculated. Then, the gradients of all weights belonging to the same cluster in the clustering results are accumulated to update the centroids of the same cluster. For example, refer to... Figure 6 The diagram illustrating parameter sharing shows that different cluster types have different identifiers (e.g., different font sizes or font styles). For example, weight bits 1, 8, 10, and 13 belong to the same cluster 1; weight bits 2, 7, 9, and 12 belong to the same cluster 2. Let's take cluster 1 as an example. First, the gradient of each weight bit can be calculated separately, for example, gradients of -0.03, 0.12, 0.02, and -0.07 respectively. Then, the gradients of each weight bit belonging to the same cluster can be accumulated. This accumulated value can then be merged with the cluster centers to update the cluster centers. (Reference) Figure 6As shown in FIG. 2, the gradient after accumulation of the same cluster, i.e., cluster 1, is 0.04. The value is fused with the cluster center 2.00 (i.e., the cluster center is subtracted from the accumulated value) to update the cluster center of cluster 1 to 2.00-0.04=1.96.

[0088] The above steps are repeated to accumulate the gradients corresponding to the weight bits of all the same clusters in the clustering result and subtract the cluster centers of the same clusters to update the center points of all the same clusters.

[0089] In step S720, the compressed original model is trained based on the center points until the compressed original model converges to obtain the trained original model.

[0090] In this step, after updating the center points of the same clusters, the compressed original model can be trained based on the center points. That is, only the center points are trained instead of training each weight or weight group itself. By taking the center points as the training parameters, the model parameters of the compressed original model are adjusted until the end condition is met. The end condition can be that the compressed original model converges or that the maximum step limit is reached.

[0091] For example, taking the convolution operation of input feature map x and weight w (both are tensors) as an example (bias is not involved in parameter sharing, which is omitted below), if the network does not perform parameter sharing compression, the output y of the layer can be simply denoted as: y=conv(x,w). The operation graph contains many layers of such operations. If parameter sharing is used for network compression, the weight w is decomposed into two parts after compression: an index tensor i (which is completely consistent with the shape of w) and a center value lookup table tensor c (whose shape depends on how many clusters the layer is clustered into, and is basically unrelated to w), and the compressed weight tensor w'=c[i], where the square brackets represent the index operation.

[0092] Reference Figure 6 As shown in FIG. 2, Figure 6 The weight w in FIG. 2 is clustered to form a cluster index i and a cluster center c. If the left lower corner is indexed, w'[0,0]=c[i[0,0]]=c[3]=2.00, w'[0,1]=c[i[0,1]]=c[1]=0.00, and so on.

[0093] In the training of the compressed original model, if the operation graph is not to be modified, w' directly participates in the operation instead of w, that is, y = conv(x, w'), the calculated gradient is the gradient for each original weight bit, and thus the gradient of each same cluster needs to be accumulated according to the index i to obtain the updated cluster center c. Further, the compressed original model can be trained based on the updated cluster center to obtain the trained original model.

[0094] Figure 8 The flowchart of model training through the operation graph is schematically shown in FIG. 1, and the steps are described in detail as follows. Figure 8

[0095] In step S810, each weight bit is redirected to the center point of the cluster to which the weight bit belongs.

[0096] In step S820, the compressed original model is trained according to the forward inference and the backward propagation update of the center point until the compressed original model converges to obtain the trained original model.

[0097] In the embodiments of the present disclosure, for a general convolution, the convolution operation is implemented by inputting the feature map and adding the bias. In this process, each convolution weight bit needs to be updated and accumulated to update the center point. In order to reduce the operation steps, the operation graph can be modified, and a required weight size can be reconstructed by taking the center point to replace the original weight bit tensor, for example, the weight A is replaced by B by taking the center point. Based on this, the center point is used to replace the original weight of the weight bit to modify the operation graph, and the center point is used as the training parameter instead of the weight itself. After the center point is used as the training parameter, the convolution operation can be performed according to the center point to implement the forward inference, and the center point is adjusted by the error through the backward propagation to adjust the model parameters of the compressed original model until the compressed original model converges or reaches the maximum step limit to obtain the trained original model.

[0098] For example, taking the convolution operation of the input feature map x and the weight w (both are tensors) as an example. If the operation graph is modified, y = conv(x, c[i]), where the index i is fixed after the clustering is completed, and the cluster center c becomes the parameter of the compressed network, and thus the gradient in the training is directly for the cluster center c. Compared with the steps in the prior art, Figure 7 Compared with the steps in the prior art, Figure 1 The steps of grouping and dimension reduction in the prior art are omitted, and the model training efficiency is improved.

[0099] ​It should be noted that the loss function of the compressed original model can be the same as that of the original model, or can be reselected or more subterms can be added according to experience, or can be obtained during the training process. Among them, in order to reduce the precision loss, the loss function can be determined according to the error of the compressed original model and the expression ability of the original model and / or other models, and the error can be calculated by the output layer and / or the intermediate layer. For the loss function, it can be determined according to the task type of the original model, and the original models corresponding to different task types can be the same or different. For example, when the task type of the original model is classification, the loss function thereof can be cross-entropy loss function; when the task type of the original model is pixel-level task, the loss function thereof can be L1 loss function, and so on.

[0100] After the model training is completed, the trained original model can be preprocessed. The preprocessing can include but is not limited to re-compression (such as Huffman coding), model format conversion, and end-side deployment, and the like.

[0101] In order to achieve the maximum benefit of parameter sharing, the required deployment platform can have support at the runtime library level or even the hardware level (such as low-bit quantization access, using indexed values in the code table as weights during forward inference, etc.). The deployment platform can also be a general-purpose platform that does not have the above conditions. When it is a general-purpose platform, a conventional model format can be used, and the center point to which each weight belongs can be directly stored in the original weight bit. Whether the index plus code table form of compression format is used during final deployment or not, as long as the compression and / or regularization in the form of parameter sharing are involved, it belongs to the scope of the present disclosure.

[0102] The technical solutions in the embodiments of the present disclosure can determine different actual cluster numbers for each layer of the original model by introducing sensitivity analysis according to the influence of each layer of the original model after compression on the inference ability of the model on actual data, and can obtain a compressed original model configuration suitable for the current training target under the same global compression rate setting, thereby improving the rationality and the precision of the compressed original model. By calculating the actual cluster number of each layer through sensitivity analysis, the difference caused by determining based on the experience rule lacking theoretical basis in the related art is avoided, and the compression decision has stronger interpretability and higher accuracy. By performing sensitivity analysis on the original model, without modifying the model definition and the core training process, only one step needs to be added before the model training, so that higher compressed precision can be obtained, the situation that the training process changes and cannot be aligned with the original model is avoided, resource consumption is reduced, efficiency is improved, and cost is reduced.

[0103] Figure 9 The overall flowchart of model training is schematically shown in FIG. 1, and the main steps are shown in FIG. 2. Figure 9 The main steps include the following steps:

[0104] In step S901, an original model is acquired. The original model can be a neural network.

[0105] In step S902, it is determined whether the original model converges. If yes, go to step S904. If no, go to step S903.

[0106] In step S903, the original model is trained.

[0107] In step S904, sensitivity analysis is performed. Specifically, the following steps are included:

[0108] In step S9041, input data is input.

[0109] In step S9042, the input data is input into the original model and temporary copies 1 to n, respectively.

[0110] In step S9043, the original loss and losses 1, 2 to n are acquired.

[0111] In step S9044, the current loss function value is converted for sensitivity to obtain the sensitivity of each layer.

[0112] In step S905, the actual cluster number of each layer is calculated according to the sensitivity of each layer, and the original model of each layer is clustered according to the actual cluster number to obtain a compressed original model.

[0113] In step S906, the compressed original model is trained.

[0114] In step S907, it is determined whether the compressed original model converges or whether the maximum step limit is reached. If yes, go to step S908. If no, go to step S906 to continue training the compressed original model.

[0115] In step S908, subsequent processing is performed.

[0116] The technical solutions in the embodiments of the present disclosure can be applied to compression of neural networks in the fields of image, voice, natural language processing, control, etc. By introducing sensitivity analysis before training the neural network, different actual cluster numbers are determined for each layer of the original model according to the influence of the compression of each layer on the inference ability of the model on actual data, and under the same global compression rate setting, a compressed original model configuration suitable for the current training target can be obtained, thereby improving the rationality and the accuracy of the compressed original model. The actual cluster number of each layer in the original model is calculated through sensitivity analysis, thereby avoiding the difference caused by the determination based on the experience rule lacking theoretical basis in the related art, and the compression decision has stronger interpretability and higher accuracy. Through sensitivity analysis of the original model, the model definition and the core training process do not need to be modified, only one step needs to be added before model training, and higher accuracy of the compressed model can be achieved, thereby avoiding the situation that the training process changes and cannot be aligned with the original model, reducing resource consumption, improving efficiency, and reducing cost.

[0117] For parameter sharing in the field of artificial intelligence, the technical solutions provided in the embodiments of the present disclosure are that the higher the sensitivity of a layer is, the greater the accuracy loss caused by compression is, and the compression rate needs to be correspondingly reduced (achieved by increasing the cluster number), while the compression rate of a layer with lower sensitivity can be appropriately increased, and finally the global compression rate requirement is met. Compared with the traditional global unified cluster number or the parameter sharing depending on experience modeling, the unimportant layers can be more accurately identified, and the expression ability and accuracy of the compressed model are improved; and all types of models can be compressed, thereby improving the universality and practicality, and avoiding the limitation of intelligent application to part of the models or part of the scenarios.

[0118] In the embodiments of the present disclosure, an object processing method is also provided, which is shown in Figure 10 and mainly includes the following steps:

[0119] In step S1010, a to-be-processed object is obtained.

[0120] In step S1020, a convolution operation is performed on the to-be-processed object by using the trained original model, to obtain a prediction result corresponding to the to-be-processed object; wherein the original model is trained according to the model training method in any one of the above embodiments.

[0121] In the embodiments of the present disclosure, the to-be-processed object can be determined according to an application scenario, for example, a to-be-processed image or any type of text, data, corpus, etc. In the embodiments of the present disclosure, the to-be-processed object is taken as a to-be-processed image for example. In actual application, only the to-be-processed object needs to be input into the trained original model for convolution operation to extract feature data of the to-be-processed object, and then the feature data is predicted and processed to obtain a prediction result corresponding to the to-be-processed object. The prediction result can be determined according to an actual application scenario, for example, a face recognition result, an image category, a classification result, etc. For example, a to-be-processed image is input into a trained original model to obtain a face recognition result corresponding to the to-be-processed image.

[0122] In the embodiments of the present disclosure, by performing sensitivity analysis on the original model before training the model and obtaining the actual cluster number of each layer in the original model according to the sensitivity, the problem that the cluster number of each layer can only be determined in a manual manner in the related art is avoided, so that the actual cluster number of each layer is determined specifically rather than fixed, the accuracy and flexibility of the actual cluster number are improved, the accuracy and precision of the compressed model are improved, the limitation is avoided, the generality is improved, and the accuracy of model training is improved. The model compression is realized by calculating the sensitivity, without modifying the training code, only the step of performing sensitivity analysis on each layer needs to be added, the steps required to be performed by the computer during training of the model are reduced, so that the processing speed and operation performance of the computer are improved, the power consumption is reduced, and the required computing resources are reduced. Since the original model is compressed, the required memory is reduced, the processing efficiency of the to-be-processed object is improved, the required amount of calculation during prediction and classification of the to-be-processed object is reduced, and the resource consumption is reduced.

[0123] In the embodiments of the present disclosure, a model training device is provided, which can include Figure 11 as shown in the above

[0124] The model obtaining module 1101 is configured to obtain an original model.

[0125] The sensitivity analysis module 1102 is configured to, if the original model converges, perform sensitivity analysis on the original model to obtain a sensitivity, and obtain an actual cluster number of each layer in the original model according to the sensitivity.

[0126] The compressed model training module 1103 is configured to cluster the weight values of each layer according to the actual cluster number of each layer to compress the original model through parameter sharing, and train the compressed original model based on a clustering result to obtain a trained original model.

[0127] In an example embodiment of the present disclosure, the sensitivity analysis module comprises: a loss calculation module configured to calculate a loss function value of the original model on the data set, and calculate a current loss function value by a plurality of temporary copies of the original model; a sensitivity calculation module configured to determine the sensitivity of each layer according to the loss function value and the current loss function value; and a cluster number calculation module configured to determine the actual cluster number of each layer in the original model based on the sensitivity of each layer.

[0128] In an example embodiment of the present disclosure, the loss calculation module comprises: a weight clustering module configured to cluster the weight of the i-th layer in the i-th temporary copy of the plurality of temporary copies to determine a cluster type, determine each original position of the i-th layer as a center value of the cluster type, and keep the weight of other layers unchanged, to obtain the compressed i-th temporary copy; and a loss determination module configured to forward the data set through the i-th temporary copy to calculate the current loss function value.

[0129] In an example embodiment of the present disclosure, the cluster number calculation module is configured to calculate the actual cluster number of each layer based on the conditional constraint of the global compression rate, in combination with the correlation between the sensitivity and the compression rate of each layer.

[0130] In an example embodiment of the present disclosure, the sensitivity analysis module comprises: a loss calculation module configured to forward calculate the loss function of all compressed temporary copies to obtain a current loss function value of each compressed temporary copy; an actual cluster number determination module configured to propagate the increment of each current loss function value relative to the loss function value back to the cluster number of each layer of each compressed temporary copy to obtain the actual cluster number of each layer in the next iteration; and an iteration module configured to forward calculate the loss based on the actual cluster number of each layer in the next iteration until the actual cluster number of each layer converges, to obtain the actual cluster number of each layer.

[0131] In an example embodiment of the present disclosure, the compressed model training module comprises: a center point updating module configured to calculate the gradient of each weight position, and perform accumulation operation on the gradients corresponding to all weight positions belonging to the same cluster in the clustering result to update the center point of the same cluster; and a training control module configured to train the compressed original model based on the center point until the compressed original model converges, to obtain the trained original model.

[0132] In an example embodiment of the present disclosure, the compressed model training module comprises: a redirection module configured to redirect each weight bit to a center point of a cluster to which the weight bit belongs; and a center point training module configured to perform forward inference and back propagation update according to the center points, train the compressed original model until the compressed original model converges, and obtain the trained original model.

[0133] It should be noted that the specific details of each module in the model training apparatus described above have been described in detail in the corresponding model training method, and therefore will not be described here.

[0134] The present disclosure also provides an object processing apparatus. As shown in Figure 12 , the object processing apparatus 1200 can include:

[0135] An object acquisition module 1201 configured to acquire an object to be processed.

[0136] A prediction module 1202 configured to perform convolution operation on the object to be processed by using the trained original model, and acquire a prediction result corresponding to the object to be processed. The original model is trained according to any one of the model training methods described above.

[0137] Figure 13 A schematic diagram of an electronic device suitable for implementing the example embodiments of the present disclosure is shown. The terminal of the present disclosure can be configured in the form of an electronic device as shown in Figure 13 , however, it should be noted that Figure 13 the electronic device shown is only an example and should not limit the functions and use range of the embodiments of the present disclosure.

[0138] The electronic device of the present disclosure at least includes a processor and a memory, and the memory is used to store one or more programs, when the one or more programs are executed by the processor, the processor can implement the method of the example embodiments of the present disclosure.

[0139] Specifically, as Figure 13As shown, the electronic device 1300 can include a processor 1310, an internal memory 1321, an external memory interface 1322, a universal serial bus (USB) interface 1330, a charging management module 1340, a power management module 1341, a battery 1342, an antenna 1, an antenna 2, a mobile communication module 1350, a wireless communication module 1360, an audio module 1370, a speaker 1371, a receiver 1372, a microphone 1373, an earphone interface 1374, a sensor module 1380, a display screen 1390, a camera module 1391, an indicator 1392, a motor 1393, a key 1394, and a subscriber identification module (SIM) card interface 1395, etc. The sensor module 1380 can include a depth sensor, a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor, etc.

[0140] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 1300. In other embodiments of the present application, the electronic device 1300 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0141] The processor 1310 can include one or more processing units, for example: the processor 1310 can include an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors. In addition, the processor 1310 can also be provided with a memory for storing instructions and data. The object processing method in the present exemplary embodiment can be executed by an application processor, a graphics processor, or an image signal processor. When the method involves neural network related processing, an NPU can be used to execute.

[0142] The internal memory 1321 can be used to store computer executable program codes, including instructions. The internal memory 1321 can include a program storage area and a data storage area. The external memory interface 1322 can be used to connect an external memory card, such as a Micro SD card, to realize the expansion of the storage capacity of the electronic device 1300.

[0143] The communication function of the mobile terminal 1300 can be implemented by a mobile communication module, an antenna 1, a wireless communication module, an antenna 2, a modem processor, and a baseband processor, etc. The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals. The mobile communication module can provide a mobile communication solution applied to the mobile terminal 1300, such as 2G, 3G, 4G, 5G, etc. The wireless communication module can provide a wireless communication solution applied to the mobile terminal 200, such as a wireless local area network, Bluetooth, near field communication, etc.

[0144] The display screen is used to implement a display function, such as displaying a user interface, an image, a video, etc. The camera module is used to implement a shooting function, such as shooting an image, a video, etc. The audio module is used to implement an audio function, such as playing an audio, collecting a voice, etc. The power module is used to implement a power management function, such as charging a battery, supplying power for the device, monitoring a battery state, etc.

[0145] The application further provides a computer readable storage medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device.

[0146] The computer readable storage medium may, for example, be— but is not limited to— an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, apparatus, or device.

[0147] The computer readable storage medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0148] The computer readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the method described in the above embodiments.

[0149] Through the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0150] In addition, the above-described drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended for limiting purposes. It is easy to understand that the processes shown in the above-described drawings do not indicate or limit the time sequence of the processes. In addition, it is also easy to understand that the processes can be executed synchronously or asynchronously, for example, in a plurality of modules.

[0151] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. Indeed, according to the embodiments of the present disclosure, the features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functionalities of one module or unit described above can be further divided into embodied by a plurality of modules or units.

[0152] Other embodiments of the present disclosure will be apparent to those skilled in the art upon consideration of the specification and practice of the concepts disclosed herein. The application is intended to cover any variations, uses, or adaptations of the present disclosure following, in general, the principles of the present disclosure and including such departures from the present disclosure that come within known or customary practice in the art to which the present disclosure pertains. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims. It is to be understood that the present disclosure is not limited to the precise construction herein described and as shown in the attached drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A model training method, characterized in that, The application is applied to application scenarios of information extraction, face recognition, and image classification, and includes: obtaining an original model; if the original model converges, performing sensitivity analysis on the original model to determine sensitivity, and obtaining actual clustering numbers of each layer in the original model according to the sensitivity; performing clustering on weight values of each layer according to the actual clustering numbers of each layer to compress the original model through parameter sharing, and training the compressed original model based on a clustering result to obtain a trained original model; the trained original model is used for processing a to-be-processed object to obtain a prediction result corresponding to the to-be-processed object; the to-be-processed object is an image, a text, data, or a corpus; wherein the sensitivity analysis on the original model to determine the sensitivity and the obtaining of the actual clustering numbers of each layer in the original model according to the sensitivity include: calculating a loss function value of the original model on a data set, and calculating a current loss function value through multiple temporary copies of the original model; determining the sensitivity of each layer according to the loss function value and the current loss function value; determining the actual clustering numbers of each layer in the original model based on the sensitivity of each layer; or, performing forward calculation on loss functions of all temporary copies to obtain current loss function values of each temporary copy; propagating increments of each current loss function value relative to the loss function value back to clustering numbers in each layer of each temporary copy to obtain actual clustering numbers of each layer in the next iteration; performing forward loss calculation according to the actual clustering numbers of each layer in the next iteration until the actual clustering numbers of each layer converge, to obtain the actual clustering numbers of each layer.

2. The model training method of claim 1, wherein, the calculation of the current loss function value through the multiple temporary copies of the original model includes: performing clustering on weight values of an i-th layer in an i-th temporary copy of the multiple temporary copies to determine a clustering type, determining each original position of the i-th layer as a center value of the clustering type, and keeping weight values of other layers unchanged, to obtain a compressed i-th temporary copy; forwarding a data set through the compressed i-th temporary copy to calculate the current loss function value.

3. The model training method of claim 1, wherein, the determination of the actual clustering numbers of each layer in the original model based on the sensitivity of each layer includes: based on a conditional constraint of a global compression rate, combining an association relationship between the sensitivity and a compression rate of each layer, and calculating the actual clustering numbers of each layer.

4. The model training method of claim 1, wherein, the training of the compressed original model based on the clustering result to obtain the trained original model includes: calculating a gradient of each weight bit, and performing accumulation operation on gradients corresponding to all weight bits belonging to the same cluster in the clustering result to update a center point of the same cluster; training the compressed original model based on the center point until the compressed original model converges, to obtain the trained original model.

5. The model training method of claim 1, wherein, the training of the compressed original model based on the clustering result to obtain the trained original model includes: redirecting each weight bit to a center point of a cluster to which each weight bit belongs. According to the center point, forward reasoning and back propagation update are performed, and the compressed original model is trained until the compressed original model converges, so as to obtain the trained original model.

6. An object processing method characterized by comprising: The application scenarios include information extraction, face recognition, and image classification. An object to be processed is obtained. A convolution operation is performed on the object to be processed by using the trained original model, and a prediction result corresponding to the object to be processed is obtained; the original model is trained according to the model training method in any one of claims 1-5; and the object to be processed is an image, a text, data, or a corpus.

7. A model training apparatus characterized by comprising: The application scenarios include information extraction, face recognition, and image classification. A model obtaining module is configured to obtain an original model. A sensitivity analysis module is configured to, if the original model converges, perform sensitivity analysis on the original model to determine a sensitivity, and obtain an actual cluster number of each layer in the original model according to the sensitivity. A compressed model training module is configured to cluster weights of each layer according to the actual cluster number of each layer to compress the original model by parameter sharing, and train the compressed original model to obtain a trained original model. The trained original model is used to process an object to be processed to obtain a prediction result corresponding to the object to be processed; the object to be processed is an image, a text, data, or a corpus. The sensitivity analysis on the original model to determine the sensitivity and the obtaining of the actual cluster number of each layer in the original model according to the sensitivity include: A loss function value of the original model on a data set is calculated, and a current loss function value is calculated through multiple temporary copies of the original model. The sensitivity of each layer is determined according to the loss function value and the current loss function value. The actual cluster number of each layer in the original model is determined based on the sensitivity of each layer. Alternatively, a forward calculation is performed on the loss functions of all temporary copies to obtain a current loss function value of each temporary copy. An increment of each current loss function value with respect to the loss function value is back propagated to the cluster number of each temporary copy in each layer to obtain actual cluster numbers of each layer in the next iteration. Forward loss calculation is performed according to the actual cluster numbers of each layer in the next iteration until the actual cluster numbers of each layer converge, so as to obtain the actual cluster numbers of each layer.

8. An object processing apparatus characterized by comprising: The application scenarios include information extraction, face recognition, and image classification. An object obtaining module is configured to obtain an object to be processed. A prediction module is configured to perform a convolution operation on the object to be processed by using a trained original model to obtain a prediction result corresponding to the object to be processed; the original model is trained according to the model training method in any one of claims 1-5; and the object to be processed is an image, a text, data, or a corpus.

9. An electronic device, comprising: A processor and a memory are included. The memory is configured to store executable instructions of the processor. The processor is configured to execute the model training method in any one of claims 1-5 or the object processing method in claim 6 by executing the executable instructions. ​ ​ 10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the model training method of any one of claims 1-5 or the object processing method of claim 6.

Images